Job

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL processes on Google Cloud Platform (GCP).
• Implement and optimize data storage solutions using BigQuery, Cloud Storage, and other GCP services.
• Collaborate with data scientists, machine learning engineers, data engineers, and other stakeholders to integrate and deploy machine learning models into production environments.
• Develop and maintain custom deployment solutions for machine learning models using tools such as Kubeflow, AI Platform, and Docker.
• Write clean, efficient, and maintainable code in Python and PySpark for data processing and transformation tasks.
• Ensure data quality, integrity, and consistency through data validation and monitoring processes.
• Deep understanding of Medallion architecture.
• Develop metadata-driven pipelines and ensure optimal processing of data.
• Use Terraform to manage and provision cloud infrastructure resources on GCP.
• Troubleshoot and resolve production issues related to data pipelines and machine learning models.
• Stay up-to-date with the latest industry trends and best practices in data engineering, machine learning, and cloud technologies, including data lifecycle management, data pruning, model drift, and model optimizations.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

Skills & Experience:

• Proven experience as a Data Engineer with a focus on Google Cloud Platform (GCP).
• Strong proficiency in Python and PySpark for data processing and transformation.
• Hands-on experience with machine learning model deployment and integration on GCP.
• Familiarity with GCP services such as BigQuery, Cloud Storage, Dataflow, and AI Platform.
• Experience with Terraform for infrastructure as code.
• Experience with containerization and orchestration tools like Docker and Kubernetes.
• Strong problem-solving skills with the ability to troubleshoot complex issues.
• Excellent communication and collaboration skills

Gcp Data Engineer position available in Gauteng, Gauteng. This job position was posted by MECS Africa. The job has been posted as a char8000 ad on 2025-04-11 at 01:26:38 in the Trade Artisans Technical category

Click Go Apply to apply online!

You might also like these jobs in the same area.

More related positions