We are looking for an experienced Azure & Purview Data Engineer to join a project for a US-based client. The ideal candidate will have a strong background in Azure data engineering and hands-on experience building scalable data pipelines while integrating data governance and compliance practices through Microsoft Purview.
Responsibilities
- Design, build, and maintain robust data pipelines across the Microsoft Azure ecosystem.
- Develop and manage pipelines using Azure Data Factory and/or Microsoft Fabric.
- Build and maintain data processing workflows using Azure Databricks and/or Azure Synapse Analytics notebooks.
- Design and implement Delta Lake and Medallion Architecture layers within Azure Data Lake Storage (ADLS).
- Ingest and integrate data from multiple sources, including databases, files, and APIs.
- Implement secure, automated, and monitored data pipeline deployments, ensuring reliability and scalability.
- Implement and operate Microsoft Purview as part of the data platform and governance strategy.
- Register and scan data sources in Microsoft Purview.
- Configure and maintain data classifications and sensitivity labels.
- Curate and maintain the data glossary, catalog, and lineage.
- Integrate governance, compliance, and access policies into data pipelines to ensure cataloging and compliance remain current as new data is ingested.
- Collaborate with technical and business stakeholders to ensure data solutions meet governance, security, and business requirements.
Requirements
- 4+ years of professional experience in Data Engineering or similar positions.
- Strong hands-on experience with Microsoft Azure data services.
- Experience with Azure Data Factory and/or Microsoft Fabric pipelines.
- Experience with Azure Databricks and/or Azure Synapse Analytics.
- Solid knowledge of Delta Lake and Medallion Architecture.
- Experience working with Azure Data Lake Storage (ADLS).
- Hands-on experience with Microsoft Purview, including source registration, scanning, classification, sensitivity labeling, glossary management, and data lineage.
- Experience integrating data from databases, files, and APIs.
- Understanding of data governance, security, access control, and compliance best practices.
- Experience implementing automated and monitored data pipeline deployments.
- Strong troubleshooting and problem-solving skills.
- English proficiency is required, as the role involves working directly with a US-based client.