AI Jobs Map

N2S.Global · Sydney, New South Wales, Australia

Lead Data Engineer - GCP & MLOps

seniorfull timePosted Aug 13
Apply on LinkedInOpens the original posting. AI Jobs Map never asks for your details.

Stack mentioned

gcpmlopsetldevopspythonkubeflowci/cddjangoserverlesscryptographybigquerymicroservicesdata-engineeringapache-beamartificial-intelligencemachine-learningrest-api

Key Skills & Achievements

- Designed and implemented a customized Data Pipeline Framework using GCP Dataflow and Apache Beam.

- Delivered multiple AI/ML solutions on Google Cloud Platform (GCP) across various business domains.

- Implemented DevOps and MLOps practices for end-to-end automation, monitoring, and governance of data pipelines and machine learning systems.

- Led MIG implementation for a Reconciliation Dashboard solution.

- Developed a Dataflow Flex Template Framework and domain-specific Python libraries to accelerate data engineering workflows.

- Built and automated Kubeflow Pipelines for streamlined ML model training and deployment.

- Applied DevOps best practices to ML systems, ensuring scalability, reliability, and continuous delivery.

- Automated continuous deployment of Django applications on GCP.

- Implemented and automated Cloud Run deployments and serverless application management.

- Configured Google Secret Manager and Cloud KMS with deterministic encryption for secure data processing.

- Developed automated CI/CD workflows using Google Cloud Build.

- Performed BigQuery data extraction, preprocessing, and automated job execution for large-scale analytics workloads.

- Managed model lifecycle and maintenance using GCP Model Registry.

- Developed microservices and REST APIs for scalable cloud-native applications.

- Built customized web applications for AI Platform resource provisioning, orchestration, and management.

- Implemented automated continuous deployment pipelines for Google Cloud Functions.

- Deployed and managed analytics dashboards using R Shiny and Python Django.

- Optimized application code, cloud resources, and deployment processes to improve performance, maintainability, and operational efficiency.

More jobs at N2S.Global