Job Details

Weekly Hours: 40

Role Number: 200641736-3337

Summary

Apple Services Engineering (ASE) powers many AI and LLM features across App Store, Music, Video and more. As these systems increasingly rely on human-in-the-loop evaluation systems, the quality of our decisions is constrained by the quality of our evaluation systems. We believe that to build exceptional LLMs, you need exceptional mechanisms to validate the signals used to train and evaluate them.

Description

As the Senior Manager of Meta Evaluation & Quality Assurance, you will lead a specialized team of Data Scientists and Machine Learning Engineers who evaluate the evaluators. You will move beyond basic validation to lead the strategy and technical development of ML-based validation frameworks and automated data quality validation pipelines. You will set strategy, guide execution, and work cross functionally to deliver a cohesive quality system that combines machine learning with human-in-the-loop processes to ensure our metrics are trustworthy, robust, and decision-ready.

Minimum Qualifications

• 8+ years of experience in Data Science, Machine Learning, or Evaluation Science, with 3+ years leading technical teams

• Strong background in Meta Evaluation, AI/ML measurement, statistics, or quality assurance methodologies.

• Demonstrated success in designing Human-in-the-Loop (HITL) machine learning systems or active learning pipelines.

• Masters degree in Statistics, Data Science, Machine Learning or related field.

Preferred Qualifications

• PhD in Statistics, Computer Science, Machine Learning, or related field

• Deep understanding of evaluation pipelines, calibration techniques, and statistical process control

• Experience building ML models specifically designed for quality estimation, anomaly detection, or disagreement modeling

• Proficiency in Python or R for statistical analysis and reasoning about evaluation data

• Experience defining governance gates or certification processes for AI systems

• Proven ability to manage complex methodological and technical programs in dynamic, fast-paced environments.

• Exceptional communication, organizational, and analytical skill.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant (https://www.eeoc.gov/sites/default/files/2023-06/22-088_EEOC_KnowYourRights6.12ScreenRdr.pdf) .