Job Title: ETRM Databricks Data Engineer
Location: Houston, Texas
Job Description:
Job Summary: ETRM Databricks Data Engineer to design, develop, and maintain scalable data solutions that support business intelligence, analytics, and data-driven decision-making with understanding of Data Domain & Data Assets in ETRM Domain. The ideal candidate will have strong expertise in Databricks, Apache Spark, Azure Cloud platforms, and modern data engineering practices.
Key Responsibilities:
- Design, develop, and optimize data pipelines using Databricks & ADF.
- Must possess good understanding of Trading Domain knowledge.
- Build and maintain scalable ETL/ELT processes for structured and unstructured data.
- Integrate data from multiple sources into enterprise data platforms.
- Develop and manage data lakes, lakehouses, and data warehouses.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Ensure data quality, integrity, security, and governance across data platforms.
- Monitor and troubleshoot data processing jobs and optimize performance.
- Implement CI/CD practices and automate data engineering workflows.
- Create and maintain technical documentation, data models, and process flows.
- Support cloud-based data solutions and migration initiatives.
Required Qualifications:
• Hands-on experience with Databricks and Azure.
• Strong proficiency in Python, SQL, and PySpark.
• Experience with cloud platforms such as Azure
• Knowledge of data warehousing, data modeling, and ETL/ELT concepts.
• Experience with version control tools such as Git.
• Strong analytical, problem-solving, and communication skills.
• Trading Domain experience related for Crude, Refined Products, Natural gas is a must.
Preferred Qualifications:
• Experience with Azure Data Factory, Azure Synapse Analytics, Delta Lake, or similar technologies.
• Knowledge of streaming data processing and real-time analytics.
• Familiarity with DevOps practices and CI/CD pipelines.
• Databricks, Azure, or related cloud certifications.