We are seeking a Senior AI/LLM Application Engineer with strong production experience building and optimizing LLM-powered applications, RAG pipelines, AI safety, and multi-step LLM workflows. This is a hands-on engineering role focused on production AI quality, accuracy, latency, evaluation, and safety.
Must-Have Skills
- 10+ years total IT experience
- 4–7 years software engineering experience with 2+ years building LLM applications in production
- Strong Python development experience
- Expert LLM prompt engineering
- Expert experience with LangChain, LlamaIndex, or custom LLM orchestration
- Strong RAG pipeline design and optimization
- Experience with embeddings, vector search, chunking, retrieval, and reranking
- Hands-on RAG evaluation using RAGAS, TruLens, or equivalent
- Experience with AI safety, guardrails, prompt injection, and jailbreak testing
- Experience optimizing multi-step LLM chains for latency, cost, and accuracy
- Experience with LLM evaluation, benchmarking, regression testing, and production monitoring
- Strong experience with multi-turn conversations, streaming responses, and session memory
- Experience building AI APIs and production-ready AI services
Responsibilities
- Build and optimize production-grade LLM and RAG applications.
- Design multi-step LLM orchestration and conversational workflows.
- Develop automated RAG/LLM evaluation frameworks and quality gates.
- Implement AI safety, guardrails, hallucination mitigation, and prompt-injection protection.
- Optimize model selection, latency, cost, retrieval accuracy, and response quality.
- Develop feedback loops to improve retrieval and model performance.
- Monitor production AI quality and identify performance or accuracy degradation.
- Work closely with product, engineering, and security teams to deliver reliable AI solutions.
Application Question(s):
- Do you have expert-level Python experience for AI/ML application development?
Experience:
- LLM applications in production: 3 years (Required)
Work Location: Hybrid remote in Santa Clara, CA 95056