Job Details
- Job Title: LLM Engineer
- Location: Cincinnati, OH
- Work Location: Remote - USA
- Duration: 1 year
- Experience Required: 8+ years
- Role Category: AI and Automation
- Education: Any degree
- Start Date: 15-September-2026
Role Overview
- Design, build, optimize, deploy, and operate Large Language Model (LLM) and Small Language Model (SLM) capabilities.
- Build secure, reliable, reusable, and enterprise-ready AI capabilities.
- Support:
- Agentic AI workflows
- AI for SDLC
- Knowledge retrieval
- Model evaluation
- Private AI hosting
- AgentOps
- Work closely with:
- Principal AI Architect
- AI Engineering Lead
- Platform Engineers
- Security teams
- Enterprise Architecture
- Product Owners
- Domain teams
Must-Have Technical Skills
LLM & Generative AI
- Large Language Models (LLMs)
- Small Language Models (SLMs)
- Prompt Engineering
- Context Engineering
- Retrieval-Augmented Generation (RAG)
- Embeddings
- Semantic Search
- Agentic AI Patterns
- Multi-Agent Workflows
- Tool Calling
- Function Calling
- Model Evaluation
- LLM Observability
Model Engineering
- Fine-Tuning
- Supervised Fine-Tuning
- LoRA
- QLoRA
- Quantization
- Distillation
- Model Compression
- Synthetic Data Generation
- Model Benchmarking
- Model Selection
- Model Routing
Model Hosting & Serving
- Private LLM Hosting
- On-Prem Model Deployment
- GPU-Based Inference
- Model Serving APIs
- High-Availability Inference
- Autoscaling
- Load Balancing
- Caching
- Batch and Real-Time Inference
AI Infrastructure & Frameworks
- Kubernetes
- Docker
- Kubeflow
- KServe
- Ray Serve
- MLflow
- Hugging Face
- Transformers
- PyTorch
- PEFT
- DeepSpeed
- NVIDIA NIM
- Triton Inference Server
- TensorRT-LLM
- vLLM
- TGI
- SGLang
Programming & Engineering
- Python
- TypeScript or JavaScript
- REST APIs
- Microservices
- CI/CD
- GitHub or Azure DevOps
- API Design
- Distributed Systems
- Cloud-Native Engineering
- Test Automation
Data & Knowledge Systems
- Vector Databases
- Knowledge Graphs
- Document Processing
- Metadata Management
- Data Pipelines
- Object Storage
- Enterprise Search
- Structured and Unstructured Data Integration
Roles & Responsibilities
LLM Application Engineering
- Build enterprise-grade LLM-powered applications and intelligent agent capabilities.
- Design reusable LLM patterns, services, APIs, and accelerators.
- Develop model interaction patterns for:
- Reasoning
- Summarization
- Classification
- Extraction
- Planning
- Decision support
- Build reusable prompt, context, retrieval, memory, and evaluation components.
- Support AI-for-SDLC agents across:
- Requirements
- Design
- Coding
- Testing
- Security Review
- Deployment
- Operations
- Convert AI use cases into scalable production solutions.
Agent Factory Intelligence Layer
- Build core intelligence services for enterprise agents.
- Develop reusable capabilities for:
- Planning
- Task decomposition
- Reasoning
- Tool usage
- Agent collaboration
- Enable agent-to-agent interaction and multi-agent orchestration.
- Integrate LLMs with:
- Agent runtimes
- Tool registries
- Workflow engines
- MCP-based gateways
- Support human-in-the-loop, approval, escalation, and feedback workflows.
- Improve agent quality, accuracy, safety, and task completion.
Prompt & Context Engineering
- Design reusable prompt engineering standards, templates, and libraries.
- Create:
- System prompts
- Task prompts
- Role prompts
- Guardrail prompts
- Evaluation prompts
- Develop context engineering strategies for better grounding, relevance, and personalization.
- Optimize:
- Token usage
- Context windows
- Memory injection
- Retrieval inputs
- Establish prompt versioning, testing, and governance practices.
Retrieval-Augmented Generation (RAG)
- Design and implement enterprise RAG architectures.
- Build retrieval pipelines using:
- Enterprise documents
- Knowledge repositories
- Structured data
- Metadata
- Optimize:
- Chunking
- Embeddings
- Indexing
- Ranking
- Reranking
- Retrieval strategies
- Improve grounding, citation quality, precision, recall, and factual accuracy.
- Build reusable retrieval services for agents and business domains.
- Partner with data and knowledge management teams to onboard trusted data sources.
LLM / SLM Model Engineering
- Evaluate, build, fine-tune, deploy, and optimize LLMs and SLMs.
- Support domain-specific model development using approved datasets.
- Build supervised fine-tuning and model adaptation pipelines.
- Apply:
- LoRA
- QLoRA
- Distillation
- Quantization
- Model compression
- Evaluate commercial, open-source, and internally hosted models.
- Select models based on:
- Accuracy
- Latency
- Cost
- Data residency
- Security
- Operational requirements
Private AI & On-Prem Hosting
- Build and support private AI capabilities for LLM/SLM hosting.
- Deploy models across:
- On-premises
- Hybrid
- Private cloud environments
- Support GPU-enabled model hosting.
- Optimize latency, throughput, concurrency, resiliency, and GPU utilization.
- Build secure inference endpoints for internal applications and agents.
- Support air-gapped and restricted AI environments.
- Partner with infrastructure and platform teams on private AI hosting.
Model Serving & Inference Optimization
- Implement scalable model serving using modern inference frameworks.
- Build high-availability inference architectures.
- Optimize:
- Token throughput
- Response latency
- Cost efficiency
- Inference performance
- Implement:
- Model routing
- Load balancing
- Caching
- Fallback strategies
- Support batch and real-time inference.
- Develop reusable deployment templates for different model families.
LLMOps / ModelOps / AgentOps
- Build operational practices for managing models and agents throughout their lifecycle.
- Implement observability for:
- Prompts
- Retrieval
- Model responses
- Latency
- Cost
- Failures
- Develop evaluation pipelines for regression testing and continuous quality improvement.
- Monitor:
- Model drift
- Response quality
- Hallucination indicators
- Safety risks
- Support CI/CD and release management for:
- Prompts
- Models
- Agents
- Retrieval pipelines
- Build dashboards and metrics for AI quality, reliability, adoption, and operational readiness.
AI Evaluation & Benchmarking
- Define and implement LLM evaluation frameworks.
- Measure:
- Accuracy
- Groundedness
- Relevance
- Hallucination rate
- Toxicity risk
- Safety compliance
- Task completion
- User satisfaction
- Build automated test suites for prompts, agents, tools, and RAG pipelines.
- Benchmark models across enterprise use cases.
- Compare cloud, open-source, and on-prem models based on performance, cost, quality, and risk.
- Establish quality gates for production AI releases.
Responsible AI, Security & Governance
- Implement Responsible AI controls in LLM applications and agent workflows.
- Develop guardrails for:
- Safe output
- Tool usage
- Data access
- Enterprise policy compliance
- Support:
- Model risk management
- Auditability
- Transparency
- Traceability
- Ensure sensitive data is handled according to security and privacy requirements.
- Collaborate with Security, Enterprise Architecture, Risk, and Compliance teams.
- Support model and agent approval and production-readiness governance.
Preferred / Additional Experience
- Experience deploying open-source models such as:
- Llama
- Mistral
- Mixtral
- Phi
- Gemma
- Qwen
- DeepSeek
- Granite
- Falcon
- Domain-specific models
- Experience with GPU infrastructure such as:
- NVIDIA H100
- H200
- B200
- B300
- A100
- L40S
- GH200
- AMD MI300X
- Experience with:
- Private AI
- Hybrid AI
- Air-gapped AI environments
- Experience in regulated industries such as:
- Healthcare
- Financial Services
- Insurance
- Experience building:
- Enterprise copilots
- AI assistants
- Agent platforms
- Experience with:
- MCP
- Tool registries
- Agent runtimes
- Enterprise integration patterns
- Experience with Responsible AI, model governance, model risk management, and AI compliance.
- Experience optimizing AI workloads for:
- Cost
- Performance
- Latency
- Security
Role / Skills Summary
- Role: LLM Engineer
- Essential Skill: LLM
- Primary Skill: AI and Automation
- Experience: 8-10+ years
- Work Location: Remote USA
- Duration: 1 year
- Core Technologies: LLM, SLM, RAG, Generative AI, Agentic AI, Python, Kubernetes, MLflow, Hugging Face, PyTorch, vLLM, KServe, Docker, Vector Databases