Job Title: Senior Data Engineer / Data Engineering Lead
Experience: 15+ Years
Location: new york
Employment Type: W2
Job Description
We are looking for an experienced Senior Data Engineer / Data Engineering Lead with 15+ years of experience in designing, developing, and implementing scalable data solutions. The ideal candidate will have strong expertise in data engineering, cloud platforms, ETL/ELT, data warehousing, big data technologies, and modern data pipelines.
Key Responsibilities
- Design and develop scalable, high-performance data pipelines and ETL/ELT processes.
- Develop and maintain enterprise-level data warehouses, data lakes, and data platforms.
- Work with business and technical teams to understand data requirements and translate them into technical solutions.
- Develop data integration solutions using tools such as Informatica, Talend, SSIS, Azure Data Factory, or similar technologies.
- Build and optimize data pipelines using Python, SQL, Spark, and PySpark.
- Design data solutions on cloud platforms such as AWS, Azure, or GCP.
- Implement data ingestion, transformation, cleansing, validation, and reconciliation processes.
- Work with large datasets using Apache Spark, Databricks, Kafka, or other Big Data technologies.
- Develop and optimize complex SQL queries, stored procedures, and data models.
- Design dimensional and relational data models for analytical and reporting requirements.
- Implement data quality, governance, security, and performance standards.
- Troubleshoot data pipeline failures and resolve performance and data-quality issues.
- Mentor junior and mid-level data engineers and provide technical leadership.
- Participate in architecture discussions, code reviews, technical documentation, and production support.
- Collaborate with DevOps, Data Science, BI, and application development teams.
Required Skills
- 15+ years of overall IT experience, with strong experience in Data Engineering.
- Strong expertise in SQL and relational databases.
- Hands-on experience with Python / PySpark.
- Strong knowledge of ETL/ELT concepts and data integration.
- Experience with Data Warehousing, Data Lakes, and Data Lakehouse architecture.
- Strong experience with one or more cloud platforms:
- AWS
- Microsoft Azure
- Google Cloud Platform (GCP)
- Experience with Apache Spark / Databricks.
- Strong knowledge of data modeling and database architecture.
- Experience with Kafka or other streaming technologies.
- Experience with CI/CD, Git, and DevOps practices.
- Strong understanding of data security, governance, and quality.
- Excellent communication, analytical, and problem-solving skills.
Preferred Skills
- Snowflake
- Azure Synapse / Microsoft Fabric
- AWS Redshift / Glue / EMR
- Databricks
- Apache Airflow
- Kafka
- Informatica
- Terraform
- Docker / Kubernetes
- Power BI / Tableau
- NoSQL databases such as MongoDB or Cassandra
- Experience with Agile/Scrum methodology
Education
Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
Ideal Candidate
The ideal candidate should be a hands-on Data Engineering Lead/Architect capable of owning end-to-end data solutions, working with modern cloud and Big Data technologies, and providing technical leadership across complex enterprise data projects.