Databricks Resident Solutions Architect (RSA)
Location: 100% Remote Work
Duration: Long term(Open for Contract or Full Time)
Looking for a Databricks RSA who is a dedicated, post-sales technical leader embedded with client teams to design, implement, and optimize enterprise Lakehouse and AI architectures. Unlike pre-sales architects, an RSA focuses on long-term delivery, governance, code-level optimizations, and enablement.
Core Technical Requirements
- Databricks Lakehouse & Spark: Deep expertise in Apache Spark runtime internals, execution plan analysis (DAGs), memory tuning, Photon engine, and Delta Lake (ACID transactions, liquid clustering, vacuuming, data versioning).
- Languages: Advanced proficiency in PySpark and Python, with strong complex analytical SQL and optional Scala.
- Data Governance & Security: Hands-on setup of Unity Catalog (metastore architecture, data lineages, access controls, row/column-level masking, external storage credentials, system tables).
- Data Engineering & Orchestration: Production pipelines with Databricks Workflows, Lakeflow / Delta Live Tables (DLT), Auto Loader, and streaming architectures (Structured Streaming).
- Cloud Platforms: Production architecture experience across at least one major provider—AWS, Azure, or GCP—including VPC peering, IAM, Private Link, and cloud storage (S3, ADLS Gen2, GCS).
- DevOps & MLOps: Databricks Asset Bundles (DABs), CI/CD pipelines (GitHub Actions, Azure DevOps, or GitLab), Terraform for infrastructure as code, and MLflow / Mosaic AI serving.
Total Experience
- 8–12 years in data engineering, distributed systems, or enterprise solutions architecture.
Databricks Experience
- 3–5+ years of hands-on production design, deployment, and performance tuning.
Consulting Skills
Proven track record guiding cross-functional teams, running architectural review boards, and advising VP/C-suite stakeholders.
Certifications (High-Priority)
- Databricks: Databricks Certified Data Engineer Professional or Databricks Certified Solutions Architect (often mandatory or strongly preferred).
- Cloud: Cloud Architect/Data Engineer credentials (e.g., AWS Solutions Architect Professional, Azure Solutions Architect Expert, or GCP Professional Data Engineer).