Job Title: Databricks AI Engineer
Location: Palm Beach, FL
Job Description Overview
We are seeking a Databricks AI Engineer with strong expertise in Databricks, PySpark, Generative AI, and Agentic AI to design and build intelligent, scalable AI solutions. The ideal candidate will have hands-on experience developing AI agents, RAG applications, and enterprise-grade data and AI platforms using Databricks.
Key Responsibilities
- Develop scalable data pipelines and AI solutions using Databricks, PySpark, Python, and SQL.
- Design and implement Generative AI applications leveraging LLMs such as GPT, Claude, and Llama.
- Build Agentic AI solutions using frameworks such as LangChain, LangGraph, CrewAI, AutoGen, or Semantic Kernel.
- Develop and optimize RAG (Retrieval-Augmented Generation) architectures using vector databases and embeddings.
- Utilize Databricks Mosaic AI, MLflow, Unity Catalog, and Vector Search for AI application development and governance.
- Integrate AI agents with enterprise systems, APIs, databases, and knowledge repositories.
- Implement LLMOps/MLOps practices for deployment, monitoring, and continuous improvement.
Required Skills
- Strong experience with Databricks, PySpark, Python, and SQL
- Expertise in Generative AI, LLMs, Prompt Engineering, and RAG
- Experience building AI Agents and Multi-Agent Systems
- Hands-on experience with LangChain, LangGraph, CrewAI, AutoGen, or similar frameworks
- Knowledge of Vector Databases (Pinecone, ChromaDB, FAISS, Weaviate)
- Experience with Azure, AWS, or GCP Cloud Platforms