Role Overview:
We are seeking a Data Engineer with 3+ years of experience to design, build, and maintain scalable data pipelines and infrastructure across Azure's data ecosystem, ensuring reliable, high-quality data delivery for analytics and downstream applications. This role is fully remote.
Key Responsibilities:
- Design and develop ETL/ELT pipelines using Azure Data Factory and Azure Databricks
- Build and optimize large-scale data processing workflows using PySpark and Python
- Work within Microsoft Fabric and manage data storage on ADLS Gen2
- Implement and manage Delta Lake architectures for reliable, versioned data storage
- Design and maintain incremental load strategies and CDC (Change Data Capture) processes
- Write efficient, complex SQL queries for data transformation and validation
- Establish and enforce data quality checks and validation frameworks
- Design logical and physical data models to support analytics and reporting needs
- Integrate real-time data streams using Kafka or Azure Event Hubs
- Manage infrastructure as code using Terraform
- Collaborate using Git for version control and CI/CD practices, primarily in a remote/async environment
Required Skills & Experience:
- 3+ years of professional experience in data engineering or a related role
- Strong SQL and Python programming skills
- Hands-on experience with PySpark for distributed data processing
- Proficiency with Azure Data Factory, Azure Databricks, and Microsoft Fabric
- Experience working with ADLS Gen2 and Delta Lake
- Solid understanding of ETL/ELT design patterns
- Experience implementing incremental loads and CDC
- Strong grasp of data quality principles and data modeling techniques
- Familiarity with Kafka or Azure Event Hubs for streaming data
- Working knowledge of Git and Terraform
- Comfortable working independently in a remote, distributed team environment
Pay: ₹1,200,000.00 - ₹1,800,000.00 per year
Benefits:
- Health insurance
- Internet reimbursement
- Paid sick time
- Paid time off
- Provident Fund
- Work from home
Experience:
- Data Engineer: 3 years (Preferred)
Work Location: Remote