Cape Town: Data Engineer (CPT Onsite) (3-month Contract to go perm) posted by Datafin
Posted on 2025-07-04 00:00:00
| Employer | Datafin |
|---|---|
| Category | Other IT/Computer |
| Location | Western Cape / Cape Town |
Job Summary
Cape Town – Western Cape – South Africa
- Design & Build Data Pipelines: Design, develop, and deploy scalable, production-grade data pipelines on Google Cloud Platform using services like Cloud Dataflow, Cloud Dataproc, and Cloud Functions to ingest, process, and transform large volumes of structured and unstructured data.
- Architect Data Warehouses & Lakes: Develop and maintain their core data warehouse architecture in BigQuery and their data lake on Google Cloud Storage, ensuring data integrity, security, performance, and cost-optimization.
- Data Modeling & ETL/ELT: Design and implement robust data models and schemas within BigQuery. Build and manage complex ETL/ELT processes using SQL, Dataflow, and other GCP-native tools to support analytics and reporting.
- Custom API Integration: Design, build, and maintain custom RESTful APIs using Python (e.g., Flask, FastAPI) and PHP to handle data ingress from and egress to various internal systems, bot platforms, and third-party services.
- Integration & Streaming: Develop and maintain data integration solutions to connect various internal and external data sources using Pub/Sub for real-time streaming and Cloud Data Fusion for batch integration.
- Monitoring & Optimization: Proactively monitor, troubleshoot, and optimize data pipelines and BigQuery performance. Utilize Cloud Monitoring and Cloud Logging to ensure reliability, identify bottlenecks, and resolve issues.
- Data Governance: Implement and manage data quality and governance frameworks within the GCP ecosystem, leveraging tools like Dataplex for data discovery, metadata management, and policy enforcement.
- Collaboration: Work closely with data scientists, analysts, and software engineers to understand data requirements, translate them into technical specifications, and integrate data solutions into production applications.
- Innovation & Best Practices: Stay current with the latest GCP data services and industry best practices. Champion and implement innovative solutions to continuously improve their data platform.
- Documentation: Create and maintain comprehensive documentation for data pipelines, architectures, and processes to ensure knowledge sharing and long-term maintainability.
- Proven Experience: Demonstrated experience as a Data Engineer, with a strong portfolio of designing and building data solutions specifically on Google Cloud Platform.
- Programming Proficiency: Strong proficiency in backend programming with Python and PHP for data processing, API development, and infrastructure automation. Ability to write clean, efficient, and maintainable code.
- SQL Mastery: Expert-level SQL skills, with extensive experience writing complex queries, optimizing performance, and modeling data in BigQuery.
- Core GCP Services: Hands-on, in-depth experience with core GCP data services, including BigQuery, Cloud Storage, Cloud Dataflow, and Pub/Sub.
- Data Architecture: Solid understanding of modern data warehousing concepts (e.g., Kimball, Inmon), data lake architectures, distributed systems, and data governance best practices within a cloud context.
- Data Integration: Proficiency in various data integration techniques, including batch processing, real-time streaming, and API-based data ingestion on GCP.
- Orchestration: Experience with workflow orchestration tools, preferably Cloud Composer (Managed Apache Airflow).
- Infrastructure as Code (IaC): Familiarity with managing GCP resources using IaC tools like Terraform.
- Database Knowledge: Experience with relational databases like Cloud SQL (MySQL, PostgreSQL) and NoSQL databases.
- Machine Learning: Familiarity with ML concepts and experience with GCP’s ML services, such as Vertex AI or BigQuery ML.
- Data Visualization: Experience building reports and dashboards using Looker or Looker Studio.
- Security & Compliance: Knowledge of GCP security best practices, IAM roles, and data compliance standards (e.g., GDPR, POPIA).
- Software Engineering Practices: Understanding of CI/CD principles and experience using tools like Cloud Build and version control systems like Git.
- Problem-Solving: Excellent analytical and problem-solving skills, with the ability to work effectively in a fast-paced, agile environment.
- Communication: Strong verbal and written communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
Click Go Apply to apply online!
Get Similar Job Alerts
Apply directly for this position. Please read all instructions carefully.
We do not process job applications; we simply aggregate and display job listings.
More related positions
Cape Town: Foreman – Urgently Needed posted by Batt Master Roofing and Guttering
MUST HAVEDrivers LicenseAttention to detailBe able to work under pressureExperience in roof sheeting, roof tiling, waterproofing and paintingRun...
View Job
Foreman – Urgently Needed
Cape Town: Business Development Representative (Remote) (JB5287) posted by Kontak Recruitment
Business Development Representative Remote JB5287 Remote, for applicants residing in Cape TownR25 000 - R30 000 Basic plus commission, cellphone and...
View Job
Business Development Representative (Remote) (JB5287)
Cape Town: Creative Marketing Lead (JB5294) posted by Kontak Recruitment
Creative Marketing Lead JB5294Cape Town, Western CapeR25 000 R35 000 CTC per monthIn-office culture, creative freedom, founder collaborationPermanentPremium...
View Job
Creative Marketing Lead (JB5294)
Cape Town: Customs Entry Clerk posted by Zedcore PTY LTD 4780165421
Customs Entry Clerk must have experiences between 2 - 5 years, is responsible for accurately preparing and submitting customs documentation, ensuring...
View Job
Customs Entry Clerk
Cape Town: Overhead Crane Technician Assistant – Urgently needed posted by BB Cranes PTY Ltd
Carries out verbal instructions of the crane technicianbranch manager, on a daily basis, in terms of the production program and output required.Collects and...
View Job
Overhead Crane Technician Assistant – Urgently needed
Cape Town: Motorcycle Drivers for Merchandising Project posted by UFC
I am looking to recruit 20 preferably female motorcycle drivers to assist with merchandising bread and confectionery across various retail stores.
View Job
Motorcycle Drivers for Merchandising Project
Cape Town: Key Account Manager – Capital Equipment posted by The Recruitment Guy
An experienced KEY ACCOUNT MANAGER FOR CAPITAL EQUIPMENT is required for permanent employment based in Cape Town.To maintain and expand relationships with...
View Job
Key Account Manager – Capital Equipment
Cape Town: Legal Advisor posted by SEESA pty Ltd
VACANCY ANNOUNCEMENT LABOUR LEGAL ADVISOR CAPE TOWNA new vacancy exists for a Labour Legal Advisor in Cape Town.MINIMUM REQUIREMENTSApplicants must be in...
View Job
Legal Advisor
Cape Town: Service Delivery Coordinator posted by Stipenda
Answering high volume of customer and interpreter callsTo allocate Face to Face Interpreting assignments to freelance interpreters according to customer...
View Job
Service Delivery Coordinator
Cape Town: STOREMAN posted by METAPIPE (PTY) LTD
Issuing material to running projects.Issuing Equipment to artisans.Receiving material from suppliers for various projects.Return material for Credit.Keep...
View Job
STOREMAN
Cape Town: Mobile Technician posted by BCE Foodservice Equipment
Overall purpose of the job To produce the required labour hours through maintenance and repair work, in accordance with the Company quality and efficiency...
View Job
Mobile Technician
Cape Town: Junior Infrastructure Engineer (Retail) (JB5316) posted by Kontak Recruitment
Junior Infrastructure Engineer Retail JB5316 Airport Industria, Cape Town R180 000 to R220 000 Annually CTCPermanentA Junior Infrastructure Engineer role is...
View Job
Junior Infrastructure Engineer (Retail) (JB5316)
Cape Town: External Sales Representative – Electronics & Guarding Services (JB5317) posted by Kontak Recruitment
External Sales Representative Electronics Guarding Services JB5317Cape Town, Western Cape R14 000 R20 000 basic plus commissionPermanentSecurity solutions...
View Job
External Sales Representative – Electronics & Guarding Services (JB5317)
Cape Town City Centre: Warehouse & Material Flow Manager – Northern Suburbs, Cape Town
Position Summary As the Warehouse Material Flow Manager , you will be responsible for overseeing all inbound, internal, and outbound material flow within a...
View Job
Warehouse & Material Flow Manager – Northern Suburbs, Cape Town
Cape Town City Centre: Intermediate .NET Developer
Full Ownership - Design, implement, document, and maintain the applications you build. - Monitor, troubleshoot, and fix bugs across the entire stack UI,...
View Job
Intermediate .NET Developer
Cape Town City Centre: Pre-Sales Engineer
Pre-sales engineers play a crucial role in the sales process by providing technical expertise and support to potential customers before a sale is finalised....
View Job
Pre-Sales Engineer
Cape Town: New Business Development posted by Kontak Recruitment
New Business Development - Plastic and Packaging JB5332 Cape Town, Western Cape R25 000 - R40 000 CTC per month plus commission PermanentFully remote Join a...
View Job
New Business Development
Cape Town City Centre: SAIPA Trainee Accountant
Main Responsibilities include but are not limited to Completion of all tasks necessary to gain sufficient exposure to the SAIPA Training Program...
View Job
SAIPA Trainee Accountant
Cape Town City Centre: Project Leader â Registered Architectural Technologist (Pr.Arch Tech) x2
Minimum requirements Professional Registration - Must be a Registered Architectural Technologist Pr.Arch Tech Candidate registrations are NOT considered...
View Job
Project Leader â Registered Architectural Technologist (Pr.Arch Tech) x2
Cape Town City Centre: Administrative & Logistics Assistant
Key Responsibilities Provide day-to-day administrative support across departments Work extensively with Microsoft Excel to manage, interpret, and analyze...
View Job
Administrative & Logistics Assistant
Cape Town City Centre: Plant and Operations Manager
The Plant Operations Manager is a pivotal leadership role responsible for driving operational excellence across manufacturing functions. This position...
View Job
Plant and Operations Manager
Cape Town City Centre: Maintenance Supervisor
If you have proven experience as a Maintenance Supervisor in the hospitality industry, strong attention to detail, and presentation with 2 to 5 years...
View Job
Maintenance Supervisor
Cape Town City Centre: Diesel Technician
Duties include but not limited to Maintenance and repairs on diesel vehicles and trucks Diagnostic testing Test driving Maintain detailed records Servicing...
View Job
Diesel Technician
Cape Town City Centre: Data Engineer (Azure Data Factory & Databricks)
PBT Group is seeking a skilled Data Engineer with strong expertise in Azure Data Factory ADF and Databricks to support and scale our data infrastructure....
View Job
Data Engineer (Azure Data Factory & Databricks)
Cape Town City Centre: F&B Sales Manager
Are you passionate about creating unforgettable guest experiences? Do you thrive in a fast-paced luxury environment where excellence and hospitality go hand...
View Job
F&B Sales Manager