Required Skills
Python
API design
Distributed systems
SQL
Database management
Debugging
Automated testing
Production troubleshooting
Software engineering principles
AI/ML
GenAI
Model inference
Prompt engineering
AI evaluation workflows
Data pipelines
Technical leadership
Preferred Skills
LLMs
SLMs
VLMs
OCR
Document AI
RAG
Embeddings
Vector databases
AI model-serving platforms
Cloud infrastructure
GPU infrastructure
Kubernetes
MLOps platforms
Model registries
Observability tools
Enterprise AI security
PII handling
SOC 2
ISO 27001
SaaS architecture
About the Role
We are looking for a highly experienced Senior AI/ML Engineer who can lead the design, development, and delivery of production-grade AI systems. This role requires strong engineering judgment, hands-on experience implementing AI/ML solutions, expertise in model evaluation, and the ability to build safe, scalable, and maintainable AI workflows for enterprise use cases.
- The Senior AI/ML Engineer will work closely with platform, product, application, and infrastructure teams to deliver reliable AI capabilities. The role will also involve mentoring junior and mid-level engineers, establishing engineering best practices, and making technical decisions that balance accuracy, cost, latency, security, and reliability.
Requirements
Programming and Engineering
- Advanced proficiency in Python.
- Strong experience with API design and integration.
- Understanding of distributed systems and scalable application architecture.
- Strong SQL and database fundamentals.
- Experience with debugging, automated testing, and production troubleshooting.
- Strong understanding of software engineering principles, code quality, and maintainability.
AI/ML and GenAI
- 6–10+ years of relevant software engineering and/or AI/ML experience.
- Hands-on experience building and supporting production AI/ML or GenAI systems.
- Strong understanding of model inference and model behavior analysis.
- Experience with prompt engineering and structured AI outputs.
- Experience designing and implementing AI evaluation workflows.
- Ability to evaluate models based on measurable quality, performance, cost, and reliability metrics.
Architecture and Production Engineering
- Experience designing AI workflows, service integrations, and data pipelines.
- Understanding of queues, asynchronous processing, monitoring, and operational reliability.
- Experience integrating AI services and models into production applications.
- Ability to design scalable and maintainable AI services and workflows.
- Strong understanding of API contracts, data validation, error handling, and system reliability.
Leadership
- Proven experience mentoring or technically guiding junior and mid-level engineers.
- Strong technical decision-making and problem-solving skills.
- Ability to communicate complex technical concepts clearly to technical and non-technical stakeholders.
- Experience documenting architecture, engineering standards, and technical decisions.
- Ability to collaborate effectively across Product, Engineering, Platform, Infrastructure, QA, and Security teams.
Responsibilities
Technical Leadership
- Lead the design and implementation of AI-powered features and model-integration patterns.
- Define engineering standards for prompts, schemas, model evaluation, monitoring, and AI workflows.
- Mentor Junior and Mid-Level AI/ML Engineers and provide technical guidance.
- Make pragmatic trade-offs between model accuracy, cost, latency, scalability, and safety.
- Review technical designs and provide recommendations for production AI architecture.
- Establish reusable engineering patterns for AI application development.
Production AI Systems
- Design and implement reliable AI workflows with validation, fallback, and human-review mechanisms.
- Evaluate and select AI models and services based on measurable performance, quality, latency, cost, and reliability criteria.
- Build, review, and improve APIs for model inference, extraction, validation, and analytics.
- Design workflows for model routing based on workload, complexity, cost, and performance requirements.
- Improve the reliability and maintainability of production AI systems.
- Support troubleshooting and resolution of AI-related production issues.
Model Evaluation and Optimization
- Design evaluation datasets and methodologies for AI/ML and GenAI systems.
- Establish measurable criteria for model accuracy, consistency, latency, cost, and reliability.
- Analyze model failures, hallucinations, edge cases, and recurring error patterns.
- Establish regression testing and release gates for AI models and workflows.
- Develop feedback loops that continuously improve model performance and output quality.
- Identify opportunities for model, prompt, workflow, and infrastructure optimization.
Governance, Safety, and Operations
- Define responsible AI standards for model usage, validation, monitoring, and release readiness.
- Design guardrail strategies covering PII handling, prompt injection, hallucination mitigation, and human oversight.
- Ensure model outputs are appropriately bounded, schema-validated, and auditable before they can trigger business actions.
- Establish model, prompt, dataset, and version governance practices.
- Drive tenant-isolation strategies across data, logs, artifacts, and processing results.
- Establish monitoring for quality drift, recurring failure modes, unsafe outputs, latency, and cost anomalies.
- Review data de-identification, redaction, retention, and access-control practices.
- Create and maintain incident-response procedures for AI failures, privacy issues, incorrect outputs, and automation errors.