Role: AWS Databricks Lead
Location: Hartford, CT (Onsite)
Job Description:
We are looking for an experienced AWS Databricks Lead to design, develop, and lead cloud-based data engineering solutions. The role involves building scalable data pipelines, implementing Databricks solutions on AWS, and providing technical leadership to the data engineering team.
Key Responsibilities:
- Lead the design and development of data solutions using Databricks on AWS.
- Build and optimize ETL/ELT pipelines using PySpark and SQL.
- Develop Delta Lake and Databricks workflows for large-scale data processing.
- Integrate Databricks with AWS services such as S3, Glue, Lambda, and Redshift.
- Implement data security, governance, performance tuning, and monitoring.
- Lead technical discussions, code reviews, and mentor team members.
- Collaborate with architects, business teams, and other engineering teams.
Required Skills:
- Strong hands-on experience with AWS and Databricks.
- Expertise in PySpark, Python, SQL, and Delta Lake.
- Experience with AWS S3, Glue, IAM, Lambda, and related services.
- Strong knowledge of data engineering, ETL/ELT, and cloud architecture.
- Experience leading data engineering teams/projects.
- Excellent troubleshooting and communication skills.