Job Description
Key Skills Required
- Core Engineering
- Strong Python; solid OOP, typing, packaging
- Test automation frameworks: pytest, Playwright/Cypress, Pact (contract testing)
- Git, GitHub Actions / FluxCD, GitOps workflows
- Kubernetes, Docker, Helm basics
- AI / LLM Engineering
- Hands-on with LLM APIs (OpenAI, Anthropic, Azure OpenAI) and prompt engineering
- RAG pipelines, embeddings, vector stores (FAISS, pgvector, or similar)
- LangChain / LlamaIndex or equivalent orchestration frameworks
- Fine-tuning and model adaptation basics (LoRA / PEFT awareness)
- Evaluation & Observability
- LLM evaluation frameworks: Ragas, DeepEval, promptfoo, LangSmith
- Metrics: groundedness, faithfulness, accuracy, latency, token cost
- Golden dataset design and regression harness setup
- Observability: OpenTelemetry, LangFuse, MLflow
- Guardrails & Trust
- NeMo Guardrails, Guardrails AI, Llama Guard, or custom policy engines
- Output schema validation (Pydantic, JSON schema)
- PII / IP leakage detection, hallucination checks, rate-limit and cost controls
- Quality & DevOps
- SonarQube, static analysis, code review automation
- CI/CD integration of AI tests and evals
- Familiarity with Speckit-driven and agentic delivery workflows
- Soft Skills
- Collaborative mindset — works across GenAI Architect, domain developers, and QA
- Clear documentation and evidence-based reporting
- Comfortable operating in a hybrid Zensar + Vanderlande POD environment