About the Role
This role sits at the intersection of technical program management, data operations, and vendor coordination for an early-stage AI/ML infrastructure company. You will own complex data and evaluation programs for frontier AI labs and internal teams, turning ambiguous technical asks into well-structured, executable programs. Your work directly improves the quality of LLM and agent evaluation at scale.
What You'll Do
-
Own data and evaluation programs end-to-end, from initial scoping and requirements gathering through production, quality assurance, delivery, and retrospective.
-
Translate ambiguous requests from AI labs and internal teams into clear specifications with defined milestones, owners, dependencies, and acceptance criteria.
-
Maintain and improve data quality procedures using quantitative signals and qualitative inspection, including analysis of task coverage, difficulty, and reward distributions.
-
Manage external vendors throughout the delivery lifecycle and improve delivery pipelines at scale.
-
Streamline data production logistics by improving workflows, documentation, automation, and cross-functional handoffs.
-
Partner with research, platform, and marketplace teams to translate learnings and feedback into product improvements.
What We're Looking For
-
3+ years owning complex technical programs in AI/ML, data operations, support engineering, applied research, or similarly cross-functional environments.
-
Demonstrated ability to translate ambiguous technical requirements into executable specifications with clear milestones, owners, dependencies, and acceptance criteria.
-
Strong quantitative and qualitative judgment about data, with the ability to move between aggregate metrics and individual examples to assess dataset quality.
-
Experience maintaining and improving complex data quality procedures using both quantitative metrics and qualitative inspection methods.
-
Experience managing external vendors and scaling delivery pipelines.
-
Background supporting frontier AI labs, technical enterprise customers, or research teams with urgent and evolving requirements.
-
Experience with RLHF, reinforcement-learning environments, agent evaluations, human-in-the-loop data pipelines, annotation workflows, or expert data collection.
-
Strong written and verbal communication skills, with the ability to turn complex technical information into clear decisions and next steps.
-
Early-stage startup experience and comfort working independently in fast-paced environments.
Compensation & Benefits
Competitive compensation package including equity. Full benefits include 100% employer-covered medical, dental, and vision insurance, a 401k, commuter benefits, and unlimited access to leading AI productivity tools. Visa sponsorship and relocation support are available for strong full-time candidates.
Location
On-site in San Francisco, CA or Singapore. Remote candidates who can maintain 70 to 80% time zone overlap with either office location are also welcome to apply.