Cape Town: Data Engineer (CPT Onsite) (3-month Contract to go perm) posted by Datafin
Posted on 2025-06-17 00:00:00
| Employer | Datafin |
|---|---|
| Category | Other IT/Computer |
| Location | Western Cape / Cape Town |
Job Summary
Cape Town – Western Cape – South Africa
- Design & Build Data Pipelines: Design, develop, and deploy scalable, production-grade data pipelines on Google Cloud Platform using services like Cloud Dataflow, Cloud Dataproc, and Cloud Functions to ingest, process, and transform large volumes of structured and unstructured data.
- Architect Data Warehouses & Lakes: Develop and maintain their core data warehouse architecture in BigQuery and their data lake on Google Cloud Storage, ensuring data integrity, security, performance, and cost-optimization.
- Data Modeling & ETL/ELT: Design and implement robust data models and schemas within BigQuery. Build and manage complex ETL/ELT processes using SQL, Dataflow, and other GCP-native tools to support analytics and reporting.
- Custom API Integration: Design, build, and maintain custom RESTful APIs using Python (e.g., Flask, FastAPI) and PHP to handle data ingress from and egress to various internal systems, bot platforms, and third-party services.
- Integration & Streaming: Develop and maintain data integration solutions to connect various internal and external data sources using Pub/Sub for real-time streaming and Cloud Data Fusion for batch integration.
- Monitoring & Optimization: Proactively monitor, troubleshoot, and optimize data pipelines and BigQuery performance. Utilize Cloud Monitoring and Cloud Logging to ensure reliability, identify bottlenecks, and resolve issues.
- Data Governance: Implement and manage data quality and governance frameworks within the GCP ecosystem, leveraging tools like Dataplex for data discovery, metadata management, and policy enforcement.
- Collaboration: Work closely with data scientists, analysts, and software engineers to understand data requirements, translate them into technical specifications, and integrate data solutions into production applications.
- Innovation & Best Practices: Stay current with the latest GCP data services and industry best practices. Champion and implement innovative solutions to continuously improve their data platform.
- Documentation: Create and maintain comprehensive documentation for data pipelines, architectures, and processes to ensure knowledge sharing and long-term maintainability.
- Proven Experience: Demonstrated experience as a Data Engineer, with a strong portfolio of designing and building data solutions specifically on Google Cloud Platform.
- Programming Proficiency: Strong proficiency in backend programming with Python and PHP for data processing, API development, and infrastructure automation. Ability to write clean, efficient, and maintainable code.
- SQL Mastery: Expert-level SQL skills, with extensive experience writing complex queries, optimizing performance, and modeling data in BigQuery.
- Core GCP Services: Hands-on, in-depth experience with core GCP data services, including BigQuery, Cloud Storage, Cloud Dataflow, and Pub/Sub.
- Data Architecture: Solid understanding of modern data warehousing concepts (e.g., Kimball, Inmon), data lake architectures, distributed systems, and data governance best practices within a cloud context.
- Data Integration: Proficiency in various data integration techniques, including batch processing, real-time streaming, and API-based data ingestion on GCP.
- Orchestration: Experience with workflow orchestration tools, preferably Cloud Composer (Managed Apache Airflow).
- Infrastructure as Code (IaC): Familiarity with managing GCP resources using IaC tools like Terraform.
- Database Knowledge: Experience with relational databases like Cloud SQL (MySQL, PostgreSQL) and NoSQL databases.
- Machine Learning: Familiarity with ML concepts and experience with GCP’s ML services, such as Vertex AI or BigQuery ML.
- Data Visualization: Experience building reports and dashboards using Looker or Looker Studio.
- Security & Compliance: Knowledge of GCP security best practices, IAM roles, and data compliance standards (e.g., GDPR, POPIA).
- Software Engineering Practices: Understanding of CI/CD principles and experience using tools like Cloud Build and version control systems like Git.
- Problem-Solving: Excellent analytical and problem-solving skills, with the ability to work effectively in a fast-paced, agile environment.
- Communication: Strong verbal and written communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
Click Go Apply to apply online!
Get Similar Job Alerts
Apply directly for this position. Please read all instructions carefully.
We do not process job applications; we simply aggregate and display job listings.
More related positions
Cape Town: Regional Building & infrastructure Executive posted by Construct Executive Search
Applicants must have the following requirementsMust have a Masters Degree or a Bachelors Degree in Engineering.Must be registered as a Professional Engineer...
View Job
Regional Building & infrastructure Executive
Cape Town: Tax Administrator (JB5261) posted by Kontak Recruitment
Tax Administrator JB5261Northern Suburbs, Cape Town, Western Cape Compensation R18 000 - R25 000 per month Based on experience and qualificationsCategory...
View Job
Tax Administrator (JB5261)
Cape Town: Medical Sales Specialist – Devices (JB5278) posted by Kontak Recruitment
Medical Sales Specialist - Devices JB5278Dieprivier, Cape Town R31 500 Basic Salary, plus commission, fuel and cell phone allowance Duration Permanent A...
View Job
Medical Sales Specialist – Devices (JB5278)
Cape Town: Accounts Clerk posted by Executive Placings
Travel Management Company is looking for an Accounts Clerk to perform all relevant duties of debtor and creditor accounting procedures and controls, plus...
View Job
Accounts Clerk
Cape Town: Corporate Receptionist (Cape Town) posted by Corporate Placements
Do you enjoy the flexibility and available immediately then register with us today for Temporary Receptionist assignments.Must haveGrade 12Min. 2 years...
View Job
Corporate Receptionist (Cape Town)
Cape Town: Reservations Manager posted by Executive Placings
Leading Destination Management Company is looking for an experienced Reservations Manager to serve as a sales driver and main point for key accounts while...
View Job
Reservations Manager
Cape Town: Product Manager posted by Executive Placings
Leading Destination Management Company is looking for an experienced Product Manager to maintain, develop and discover Product offerings for FIT, Groups and...
View Job
Product Manager
Cape Town: Head of Wholesale (Exco) posted by Corporate Placements
Our client a luxury retail business is seeking a Head of Wholesale to join their team. You will be responsible for strategic development and implementation....
View Job
Head of Wholesale (Exco)
Cape Town: Strategic Projects & Partnership Lead posted by Corporate Placements
Our client a non-profit organisation is seeking a Strategic Projects and Partnerships Lead to join their team. You will be responsible for leading the...
View Job
Strategic Projects & Partnership Lead
Cape Town: Foreman – Urgently Needed posted by Batt Master Roofing and Guttering
MUST HAVEDrivers LicenseAttention to detailBe able to work under pressureExperience in roof sheeting, roof tiling, waterproofing and paintingRun...
View Job
Foreman – Urgently Needed
Cape Town: Business Development Representative (Remote) (JB5287) posted by Kontak Recruitment
Business Development Representative Remote JB5287 Remote, for applicants residing in Cape TownR25 000 - R30 000 Basic plus commission, cellphone and...
View Job
Business Development Representative (Remote) (JB5287)
Cape Town: Creative Marketing Lead (JB5294) posted by Kontak Recruitment
Creative Marketing Lead JB5294Cape Town, Western CapeR25 000 R35 000 CTC per monthIn-office culture, creative freedom, founder collaborationPermanentPremium...
View Job
Creative Marketing Lead (JB5294)
Cape Town: Customs Entry Clerk posted by Zedcore PTY LTD 4780165421
Customs Entry Clerk must have experiences between 2 - 5 years, is responsible for accurately preparing and submitting customs documentation, ensuring...
View Job
Customs Entry Clerk
Cape Town: Overhead Crane Technician Assistant – Urgently needed posted by BB Cranes PTY Ltd
Carries out verbal instructions of the crane technicianbranch manager, on a daily basis, in terms of the production program and output required.Collects and...
View Job
Overhead Crane Technician Assistant – Urgently needed
Cape Town: Motorcycle Drivers for Merchandising Project posted by UFC
I am looking to recruit 20 preferably female motorcycle drivers to assist with merchandising bread and confectionery across various retail stores.
View Job
Motorcycle Drivers for Merchandising Project
Cape Town City Centre: . Net Developer (30 Month Contract)
Key Responsibilities Participate in planning, requirements gathering, and deployment of enterprise-wide systems. Develop scalable, full-stack solutions...
View Job
. Net Developer (30 Month Contract)
Cape Town: Key Account Manager – Capital Equipment posted by The Recruitment Guy
An experienced KEY ACCOUNT MANAGER FOR CAPITAL EQUIPMENT is required for permanent employment based in Cape Town.To maintain and expand relationships with...
View Job
Key Account Manager – Capital Equipment
Cape Town City Centre: Executive Assistant to COO
A highly organised, professional and well presented Executive Assistant is sought for the COO of this hospitality group, based in Cape Town. Requirements...
View Job
Executive Assistant to COO
Cape Town City Centre: FRONT OFFICE ATTENDANT
Key Responsibilities Transport guests safely and comfortably to and from the airport, local destinations, and hotel events Maintain cleanliness and upkeep...
View Job
FRONT OFFICE ATTENDANT
Cape Town City Centre: Service Technician
Job Purpose The Services Technician is responsible for providing support to end-users including 2nd Level Application and Technical Support to the companys...
View Job
Service Technician
Cape Town City Centre: SENIOR TECHNICIAN â REFRIGERATION (CAPE TOWN)
Minimum Requirements Grade 12 Matric. N6 Technical qualification. Trade-tested refrigeration qualification non-negotiable. 2-3 years of experience in...
View Job
SENIOR TECHNICIAN â REFRIGERATION (CAPE TOWN)
Cape Town City Centre: Project Manager (IT Infrastructure Domain)
Job Description Our client is seeking a highly skilled and experienced IT Infrastructure Project Manager to lead and coordinate the planning, execution, and...
View Job
Project Manager (IT Infrastructure Domain)
Cape Town City Centre: Temporary Painter
Were on the lookout for a temporary Painter to join the team at a leading site in Newlands. If you take pride in delivering neat, quality finishes and enjoy...
View Job
Temporary Painter
Cape Town City Centre: Events and Logistics Coordinator
Events and Logistics Coordinator Location Cape Town Employment Type Permanent Are you a dynamic, detail-oriented individual with a passion for events and...
View Job
Events and Logistics Coordinator
Cape Town City Centre: Financial Analyst
My client, a leader in the FMCG sector, is seeking an ambitious Financial Analyst who is eager to gain international exposure and will support the Africa...
View Job
Financial Analyst