We are looking for a talented Senior Machine Learning Engineer - LLM Systems & Evaluation.
This is an opportunity to work on next-generation AI systems, including large language models, retrieval-augmented generation, agents, and AI safety-focused evaluation.
Essential functions:
- Own machine learning projects from problem definition through implementation
- Design and implement evaluation methodologies for AI and machine learning systems
- Create datasets, benchmarks, and metrics to measure model and product performance
- Evaluate and improve LLM-based systems, including RAG applications, agents, safety systems, and end-to-end AI products
- Analyse model behaviour, identify failure modes, and recommend practical improvements
- Build and maintain ML pipelines, tooling, and evaluation infrastructure
- Collaborate with product, engineering, and research teams to translate business goals into measurable ML objectives
- Prototype and iterate rapidly to solve business and product challenges
- Communicate findings, trade-offs, and recommendations to both technical and non-technical stakeholders
Qualifications:
- 5+ years of experience in Machine Learning Engineering or a related field.
- Strong understanding of machine learning fundamentals and model evaluation
- Strong Python programming skills and experience with modern ML frameworks such as PyTorch, TensorFlow, or JAX
- Understanding of retrieval-augmented generation (RAG), agentic systems, and LLM safety concepts
- Experience training, fine-tuning, or adapting machine learning models
- Experience working with Large Language Models beyond simple API integration
- Experience evaluating AI systems and translating results into actionable recommendations
- Experience building and maintaining machine learning systems and pipelines
- Ability to work effectively in ambiguous problem spaces with incomplete requirements and limited data
- Strong written and verbal communication skills
Would be a plus:
- Experience designing benchmarks, evaluation frameworks, or automated evaluation systems
- Experience with distributed training or large-scale model inference
- Experience building reusable ML tooling and internal platforms
- Experience with cloud platforms and modern MLOps practices
- Experience working on user-facing AI products at scale
- Research experience or publications in machine learning or AI-related fields