Cape Town: Data Engineer (CPT Onsite) (3-month Contract to go perm) posted by Datafin
Posted on 2025-07-15 00:00:00
| Employer | Datafin |
|---|---|
| Category | Other IT/Computer |
| Location | Western Cape / Cape Town |
Job Summary
Cape Town – Western Cape – South Africa
- Design & Build Data Pipelines: Design, develop, and deploy scalable, production-grade data pipelines on Google Cloud Platform using services like Cloud Dataflow, Cloud Dataproc, and Cloud Functions to ingest, process, and transform large volumes of structured and unstructured data.
- Architect Data Warehouses & Lakes: Develop and maintain their core data warehouse architecture in BigQuery and their data lake on Google Cloud Storage, ensuring data integrity, security, performance, and cost-optimization.
- Data Modeling & ETL/ELT: Design and implement robust data models and schemas within BigQuery. Build and manage complex ETL/ELT processes using SQL, Dataflow, and other GCP-native tools to support analytics and reporting.
- Custom API Integration: Design, build, and maintain custom RESTful APIs using Python (e.g., Flask, FastAPI) and PHP to handle data ingress from and egress to various internal systems, bot platforms, and third-party services.
- Integration & Streaming: Develop and maintain data integration solutions to connect various internal and external data sources using Pub/Sub for real-time streaming and Cloud Data Fusion for batch integration.
- Monitoring & Optimization: Proactively monitor, troubleshoot, and optimize data pipelines and BigQuery performance. Utilize Cloud Monitoring and Cloud Logging to ensure reliability, identify bottlenecks, and resolve issues.
- Data Governance: Implement and manage data quality and governance frameworks within the GCP ecosystem, leveraging tools like Dataplex for data discovery, metadata management, and policy enforcement.
- Collaboration: Work closely with data scientists, analysts, and software engineers to understand data requirements, translate them into technical specifications, and integrate data solutions into production applications.
- Innovation & Best Practices: Stay current with the latest GCP data services and industry best practices. Champion and implement innovative solutions to continuously improve their data platform.
- Documentation: Create and maintain comprehensive documentation for data pipelines, architectures, and processes to ensure knowledge sharing and long-term maintainability.
- Proven Experience: Demonstrated experience as a Data Engineer, with a strong portfolio of designing and building data solutions specifically on Google Cloud Platform.
- Programming Proficiency: Strong proficiency in backend programming with Python and PHP for data processing, API development, and infrastructure automation. Ability to write clean, efficient, and maintainable code.
- SQL Mastery: Expert-level SQL skills, with extensive experience writing complex queries, optimizing performance, and modeling data in BigQuery.
- Core GCP Services: Hands-on, in-depth experience with core GCP data services, including BigQuery, Cloud Storage, Cloud Dataflow, and Pub/Sub.
- Data Architecture: Solid understanding of modern data warehousing concepts (e.g., Kimball, Inmon), data lake architectures, distributed systems, and data governance best practices within a cloud context.
- Data Integration: Proficiency in various data integration techniques, including batch processing, real-time streaming, and API-based data ingestion on GCP.
- Orchestration: Experience with workflow orchestration tools, preferably Cloud Composer (Managed Apache Airflow).
- Infrastructure as Code (IaC): Familiarity with managing GCP resources using IaC tools like Terraform.
- Database Knowledge: Experience with relational databases like Cloud SQL (MySQL, PostgreSQL) and NoSQL databases.
- Machine Learning: Familiarity with ML concepts and experience with GCP’s ML services, such as Vertex AI or BigQuery ML.
- Data Visualization: Experience building reports and dashboards using Looker or Looker Studio.
- Security & Compliance: Knowledge of GCP security best practices, IAM roles, and data compliance standards (e.g., GDPR, POPIA).
- Software Engineering Practices: Understanding of CI/CD principles and experience using tools like Cloud Build and version control systems like Git.
- Problem-Solving: Excellent analytical and problem-solving skills, with the ability to work effectively in a fast-paced, agile environment.
- Communication: Strong verbal and written communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
Click Go Apply to apply online!
Get Similar Job Alerts
Apply directly for this position. Please read all instructions carefully.
We do not process job applications; we simply aggregate and display job listings.
More related positions
Cape Town: Overhead Crane Technician Assistant – Urgently needed posted by BB Cranes PTY Ltd
Carries out verbal instructions of the crane technicianbranch manager, on a daily basis, in terms of the production program and output required.Collects and...
View Job
Overhead Crane Technician Assistant – Urgently needed
Cape Town: Motorcycle Drivers for Merchandising Project posted by UFC
I am looking to recruit 20 preferably female motorcycle drivers to assist with merchandising bread and confectionery across various retail stores.
View Job
Motorcycle Drivers for Merchandising Project
Cape Town: Key Account Manager – Capital Equipment posted by The Recruitment Guy
An experienced KEY ACCOUNT MANAGER FOR CAPITAL EQUIPMENT is required for permanent employment based in Cape Town.To maintain and expand relationships with...
View Job
Key Account Manager – Capital Equipment
Cape Town: Legal Advisor posted by SEESA pty Ltd
VACANCY ANNOUNCEMENT LABOUR LEGAL ADVISOR CAPE TOWNA new vacancy exists for a Labour Legal Advisor in Cape Town.MINIMUM REQUIREMENTSApplicants must be in...
View Job
Legal Advisor
Cape Town: Service Delivery Coordinator posted by Stipenda
Answering high volume of customer and interpreter callsTo allocate Face to Face Interpreting assignments to freelance interpreters according to customer...
View Job
Service Delivery Coordinator
Cape Town: Mobile Technician posted by BCE Foodservice Equipment
Overall purpose of the job To produce the required labour hours through maintenance and repair work, in accordance with the Company quality and efficiency...
View Job
Mobile Technician
Cape Town: STOREMAN posted by METAPIPE (PTY) LTD
Issuing material to running projects.Issuing Equipment to artisans.Receiving material from suppliers for various projects.Return material for Credit.Keep...
View Job
STOREMAN
Cape Town: External Sales Representative – Electronics & Guarding Services (JB5317) posted by Kontak Recruitment
External Sales Representative Electronics Guarding Services JB5317Cape Town, Western Cape R14 000 R20 000 basic plus commissionPermanentSecurity solutions...
View Job
External Sales Representative – Electronics & Guarding Services (JB5317)
Cape Town: Junior Infrastructure Engineer (Retail) (JB5316) posted by Kontak Recruitment
Junior Infrastructure Engineer Retail JB5316 Airport Industria, Cape Town R180 000 to R220 000 Annually CTCPermanentA Junior Infrastructure Engineer role is...
View Job
Junior Infrastructure Engineer (Retail) (JB5316)
Cape Town: New Business Development posted by Kontak Recruitment
New Business Development - Plastic and Packaging JB5332 Cape Town, Western Cape R25 000 - R40 000 CTC per month plus commission PermanentFully remote Join a...
View Job
New Business Development
Cape Town: Senior Project Manager (Electrical) posted by Construct Executive Search
Applicants must have the following requirementsMust have a Bachelors Degree or Diploma in Electrical Engineering or Construction.Must have 8-10 years of...
View Job
Senior Project Manager (Electrical)
Cape Town: Senior Site Agent posted by Construct Executive Search
Applicants must have the following requirementsMust have a Bachelors Degree or Diploma in Electrical Engineering or Construction.Must have a Trade Test...
View Job
Senior Site Agent
Cape Town: Professional Bridge Design Engineer posted by Construct Executive Search
Applicants must have the following requirementsMust have a Bachelor of Science Degree in Civil Engineering.Must be registered with ECSA as a Professional...
View Job
Professional Bridge Design Engineer
Cape Town: Professional Engineer (Roads & Municipal Services) posted by Construct Executive Search
Applicants must have the following requirementsMust have a Bachelor of Science Degree or a Bachelor of Engineering Degree in Civil Engineering.Must be...
View Job
Professional Engineer (Roads & Municipal Services)
Cape Town: Professional Transportation Engineer posted by Construct Executive Search
Applicants must have the following requirementsMust have a Bachelor of Science Degree or a Bachelor of Technology Degree in EngineeringMust be registered...
View Job
Professional Transportation Engineer
Cape Town: Warehouse Checker posted by ROSS Verification Services
Warehouse checkers need for the Cape Town area.Must have a minimum of 2 years warehousing experience. Must have matric with Maths.
View Job
Warehouse Checker
Cape Town City Centre: ALP
Are you an achiever with a track record of overcoming challenges and excelling despite obstacles? Do you have a strong work ethic and a relentless drive to...
View Job
ALP
Cape Town: Personal Assistant (PA) – Urgently Needed posted by Epiglo PTY LTD
Medicore situated in Constantia is seeking an exceptional Personal Assistant to provide comprehensive support to a dynamic professional businessman active...
View Job
Personal Assistant (PA) – Urgently Needed
Cape Town: Office Administrator posted by Private
Office AdministratorJob SummaryWe seek a detail-oriented and organized Office Administrator to join our team. This permanent position is based in...
View Job
Office Administrator
Western Cape: Autos Press & Plating Team Leader – Maitland, Cape town posted by Paqueta Personnel
My client, a leading Manufacturing Company based in Maitland, Cape Town requires a competent and experienced Autos Press and Plating Team Leader to join...
View Job
Autos Press & Plating Team Leader – Maitland, Cape town
Cape Town: External Sales Rep – Urgently needed posted by Africote
External Sales Rep RequiredArea Paarden Eiland Cape TownSalary To be discussed Permanent position We are a well-established Paint, Plaster and screed...
View Job
External Sales Rep – Urgently needed
Cape Town: Regional Technical Support Specialist – Capital Equipment posted by The Recruitment Guy
An experienced REGIONAL TECHNICAL SUPPORT SPECIALIST is required for permanent employment based in Cape Town Western Cape region.To be the technical...
View Job
Regional Technical Support Specialist – Capital Equipment
Cape Town City Centre: Designer
Duties and Responsibilities Constant Trend research Compile OVS instore images from merchandisers buying trip. Trend presentation preparation presenting to...
View Job
Designer
Cape Town: Junior Administrator – Urgently Needed posted by Nu-Line Elevator Maintenance
Nu-Line Elevator Maintenance is looking for a Junior Administrator to join our team in Killarney Gardens, Cape Town. If youre passionate about learning and...
View Job
Junior Administrator – Urgently Needed
Cape Town: Accountant – Urgently Needed posted by Epiglo PTY LTD
A dynamic pharmaceutical company in Constantia, Cape Town is seeking a highly organised and detail-oriented Accountant to join our growing team. The ideal...
View Job
Accountant – Urgently Needed