Published: Mon, 14 Jul 2025 12:50:09 GMT
Position: Director of Machine Learning Engineering
Company: Chalice Custom Algorithms (chalice.ai)
About Us:
Chalice Custom Algorithms (chalice.ai) is a leading AI solution for brands looking to utilize their own data and analytics for real-time decision making in ad buying. Our platform automates data ingestion, predictive analytics, and custom bidding logic deployment across major DSPs, Meta, and YouTube. We have been recognized as “Best Demand Side Tech” by AdExchanger and powered AdWeek’s “Best Use of Programmatic” in the 2023 Media Plan of the Year awards.
About the Role:
We are seeking a highly skilled and experienced Director of Machine Learning Engineering (MLE) to lead the design, deployment, and optimization of our machine learning infrastructure and models. Reporting directly to the VP of Data Science and Analytics, this role will oversee our Senior MLE(s) and work closely with the Engineering Architect to develop and scale robust, low-latency enterprise pipelines. The Director will also mentor junior software engineers in machine learning engineering best practices, fostering growth and excellence within the team.
This is a critical position for a leader who excels at the intersection of ML infrastructure, production systems, and delivering business value. You will help shape the technical vision, ensure the delivery of scalable AI solutions, and drive cross-functional initiatives that shape the evolution of our platform.
Key Responsibilities:
• Lead the design, development, and deployment of scalable machine learning systems using Databricks, PySpark, Rust, and Kubernetes for both low-latency applications and offline training pipelines.
• Oversee LLMs and neural network predictive modeling pipelines, ensuring performance, observability, and alignment with product strategy.
• Own the full ML lifecycle – from data ingestion and feature engineering to model deployment and performance monitoring.
• Collaborate with the Engineering Architect to build robust, fault-tolerant, and maintainable ML pipelines that support enterprise scale.
• Manage and mentor a growing team of 3 senior and junior MLEs and collaborate closely with software engineers, fostering technical excellence, code quality, and architectural rigor.
• Partner with product, data science, and engineering teams to translate business objectives into technical solutions that drive measurable outcomes.
• Guide adoption of emerging technologies and techniques (e.g., LoRA, quantization, vector stores, Rust microservices for model serving) to improve model efficiency and latency.
• Ensure compliance with audit, privacy, and security standards in all ML engineering efforts.
Qualifications:
• Minimum of 8 years of experience in machine learning engineering, with at least 3 years leading ML engineering teams.
• Extensive experience building low-latency machine learning systems using Kubernetes, Rust, PySpark, and Databricks, Pytorch, Tensorflow, and state-of-the-art neural network packages.
• Proven track record in designing and deploying LLM- and neural network-based solutions at scale.
• Experience with CI/CD Integrations and ML Data and Model versioning.
• Mastery of MLOps tools and processes (MLflow, Prometheus, Grafana, Airflow, etc.).
• Strong experience with cloud platforms (AWS, Databricks) and production data engineering at scale.
• Ability to influence architecture and strategy while being hands-on when necessary.
• Experience in mentoring and developing technical talent across seniority levels.
• Excellent communication and collaboration skills to partner across product, engineering, and executive teams.
• Advertising technology or performance marketing experience is a strong plus.
Why Join Us:
At Chalice, you will be at the forefront of AI-driven advertising innovation. We offer:
• A collaborative, high-performance environment where your work shapes the future of custom AI for brands.
• Competitive compensation with performance incentives.
• Comprehensive benefits including medical, dental/vision, 401k options, unlimited PTO, 11 company holidays, and year-end closure.
• A culture that values transparency, non-conformity, and diversity of thought. Apply link