Job Description
Cape Town – Western Cape – South Africa
- Design & Build Data Pipelines: Design, develop, and deploy scalable, production-grade data pipelines on Google Cloud Platform using services like Cloud Dataflow, Cloud Dataproc, and Cloud Functions to ingest, process, and transform large volumes of structured and unstructured data.
- Architect Data Warehouses & Lakes: Develop and maintain their core data warehouse architecture in BigQuery and their data lake on Google Cloud Storage, ensuring data integrity, security, performance, and cost-optimization.
- Data Modeling & ETL/ELT: Design and implement robust data models and schemas within BigQuery. Build and manage complex ETL/ELT processes using SQL, Dataflow, and other GCP-native tools to support analytics and reporting.
- Custom API Integration: Design, build, and maintain custom RESTful APIs using Python (e.g., Flask, FastAPI) and PHP to handle data ingress from and egress to various internal systems, bot platforms, and third-party services.
- Integration & Streaming: Develop and maintain data integration solutions to connect various internal and external data sources using Pub/Sub for real-time streaming and Cloud Data Fusion for batch integration.
- Monitoring & Optimization: Proactively monitor, troubleshoot, and optimize data pipelines and BigQuery performance. Utilize Cloud Monitoring and Cloud Logging to ensure reliability, identify bottlenecks, and resolve issues.
- Data Governance: Implement and manage data quality and governance frameworks within the GCP ecosystem, leveraging tools like Dataplex for data discovery, metadata management, and policy enforcement.
- Collaboration: Work closely with data scientists, analysts, and software engineers to understand data requirements, translate them into technical specifications, and integrate data solutions into production applications.
- Innovation & Best Practices: Stay current with the latest GCP data services and industry best practices. Champion and implement innovative solutions to continuously improve their data platform.
- Documentation: Create and maintain comprehensive documentation for data pipelines, architectures, and processes to ensure knowledge sharing and long-term maintainability.
- Proven Experience: Demonstrated experience as a Data Engineer, with a strong portfolio of designing and building data solutions specifically on Google Cloud Platform.
- Programming Proficiency: Strong proficiency in backend programming with Python and PHP for data processing, API development, and infrastructure automation. Ability to write clean, efficient, and maintainable code.
- SQL Mastery: Expert-level SQL skills, with extensive experience writing complex queries, optimizing performance, and modeling data in BigQuery.
- Core GCP Services: Hands-on, in-depth experience with core GCP data services, including BigQuery, Cloud Storage, Cloud Dataflow, and Pub/Sub.
- Data Architecture: Solid understanding of modern data warehousing concepts (e.g., Kimball, Inmon), data lake architectures, distributed systems, and data governance best practices within a cloud context.
- Data Integration: Proficiency in various data integration techniques, including batch processing, real-time streaming, and API-based data ingestion on GCP.
- Orchestration: Experience with workflow orchestration tools, preferably Cloud Composer (Managed Apache Airflow).
- Infrastructure as Code (IaC): Familiarity with managing GCP resources using IaC tools like Terraform.
- Database Knowledge: Experience with relational databases like Cloud SQL (MySQL, PostgreSQL) and NoSQL databases.
- Machine Learning: Familiarity with ML concepts and experience with GCP’s ML services, such as Vertex AI or BigQuery ML.
- Data Visualization: Experience building reports and dashboards using Looker or Looker Studio.
- Security & Compliance: Knowledge of GCP security best practices, IAM roles, and data compliance standards (e.g., GDPR, POPIA).
- Software Engineering Practices: Understanding of CI/CD principles and experience using tools like Cloud Build and version control systems like Git.
- Problem-Solving: Excellent analytical and problem-solving skills, with the ability to work effectively in a fast-paced, agile environment.
- Communication: Strong verbal and written communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
GO APPLY NOW
Safe & secure application process
Explore More Opportunities
Get Similar Job Alerts
Job Seeker Tip
Dress professionally for interviews, even if the company has a casual dress code.
How to Apply
Click “GO APPLY NOW” to visit the company’s application page.
Follow their instructions carefully.
JVR Jobs connects you with employers – we don’t process applications directly.
Latest Job Opportunities
Western Cape: Estimator posted by Execustaff South Africa (Pty) Ltd
Requirements: - Grade 12/Matric with Mathematics and/or accounting as subjects- Estimating experience: minimum 3 – 5 years- Relevant Degree or…
View JobSouth Africa: EXTERNAL SALES REPRESENTATIVE (KIMBERLEY / BLOEMFONTEIN) posted by MPRTC Recruitment
Requirements:Matric / Grade 12.Technical background in electronics/electrical (qualification or practical experience preferred).Previous sales experience...
View JobSouth Africa: Code 10 Driver with PDP posted by Greys Recruitment
We are looking for a skilled Code 10 Driver with a valid PDP to join our team. As a Driver,…
View JobWestern Cape: Extrusion Manager posted by GAP Consulting
Minimum Requirements:Must have a minimum of 5 years experience within the Plastic | Extrusion Manufacturing IndustryDiploma | Degree in Mechanical…
View JobUmdloti: 1x SENIOR Beauty Therapist URGENTLY needed for a busy Spa/Salon in Durban- Umdloti posted by Spa Sense
SENIOR Beauty Therapist needed in Durban- Umdloti VACANCY REQUIREMENTS:2–3 years in Beauty/ Somatology Qualification.ITEC, CIDESCO, or SAASHP is a BONUS.3-5...
View JobJohannesburg North: C# Integration Developer posted by Network Finance
Our client in Johannesburg is looking for a technically strong and solution-oriented C# Integration Developer to join their team. The…
View Job
Browse Employers
Job Alerts