Cape Town: Data Engineer (CPT Onsite) (3-month Contract to go perm) posted by Datafin
Posted on 2025-07-15 00:00:00
| Employer | Datafin |
|---|---|
| Category | Other IT/Computer |
| Location | Western Cape / Cape Town |
Job Summary
Cape Town – Western Cape – South Africa
- Design & Build Data Pipelines: Design, develop, and deploy scalable, production-grade data pipelines on Google Cloud Platform using services like Cloud Dataflow, Cloud Dataproc, and Cloud Functions to ingest, process, and transform large volumes of structured and unstructured data.
- Architect Data Warehouses & Lakes: Develop and maintain their core data warehouse architecture in BigQuery and their data lake on Google Cloud Storage, ensuring data integrity, security, performance, and cost-optimization.
- Data Modeling & ETL/ELT: Design and implement robust data models and schemas within BigQuery. Build and manage complex ETL/ELT processes using SQL, Dataflow, and other GCP-native tools to support analytics and reporting.
- Custom API Integration: Design, build, and maintain custom RESTful APIs using Python (e.g., Flask, FastAPI) and PHP to handle data ingress from and egress to various internal systems, bot platforms, and third-party services.
- Integration & Streaming: Develop and maintain data integration solutions to connect various internal and external data sources using Pub/Sub for real-time streaming and Cloud Data Fusion for batch integration.
- Monitoring & Optimization: Proactively monitor, troubleshoot, and optimize data pipelines and BigQuery performance. Utilize Cloud Monitoring and Cloud Logging to ensure reliability, identify bottlenecks, and resolve issues.
- Data Governance: Implement and manage data quality and governance frameworks within the GCP ecosystem, leveraging tools like Dataplex for data discovery, metadata management, and policy enforcement.
- Collaboration: Work closely with data scientists, analysts, and software engineers to understand data requirements, translate them into technical specifications, and integrate data solutions into production applications.
- Innovation & Best Practices: Stay current with the latest GCP data services and industry best practices. Champion and implement innovative solutions to continuously improve their data platform.
- Documentation: Create and maintain comprehensive documentation for data pipelines, architectures, and processes to ensure knowledge sharing and long-term maintainability.
- Proven Experience: Demonstrated experience as a Data Engineer, with a strong portfolio of designing and building data solutions specifically on Google Cloud Platform.
- Programming Proficiency: Strong proficiency in backend programming with Python and PHP for data processing, API development, and infrastructure automation. Ability to write clean, efficient, and maintainable code.
- SQL Mastery: Expert-level SQL skills, with extensive experience writing complex queries, optimizing performance, and modeling data in BigQuery.
- Core GCP Services: Hands-on, in-depth experience with core GCP data services, including BigQuery, Cloud Storage, Cloud Dataflow, and Pub/Sub.
- Data Architecture: Solid understanding of modern data warehousing concepts (e.g., Kimball, Inmon), data lake architectures, distributed systems, and data governance best practices within a cloud context.
- Data Integration: Proficiency in various data integration techniques, including batch processing, real-time streaming, and API-based data ingestion on GCP.
- Orchestration: Experience with workflow orchestration tools, preferably Cloud Composer (Managed Apache Airflow).
- Infrastructure as Code (IaC): Familiarity with managing GCP resources using IaC tools like Terraform.
- Database Knowledge: Experience with relational databases like Cloud SQL (MySQL, PostgreSQL) and NoSQL databases.
- Machine Learning: Familiarity with ML concepts and experience with GCP’s ML services, such as Vertex AI or BigQuery ML.
- Data Visualization: Experience building reports and dashboards using Looker or Looker Studio.
- Security & Compliance: Knowledge of GCP security best practices, IAM roles, and data compliance standards (e.g., GDPR, POPIA).
- Software Engineering Practices: Understanding of CI/CD principles and experience using tools like Cloud Build and version control systems like Git.
- Problem-Solving: Excellent analytical and problem-solving skills, with the ability to work effectively in a fast-paced, agile environment.
- Communication: Strong verbal and written communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
Click Go Apply to apply online!
Get Similar Job Alerts
Apply directly for this position. Please read all instructions carefully.
We do not process job applications; we simply aggregate and display job listings.
More related positions
Cape Town: Intermediate Developer posted by Fempower Personnel
Job Title Intermediate Developer Location Hybrid Based in StellenboschA leading payment aggregator and provider of secure payment solutions is looking for...
View Job
Intermediate Developer
Cape Town: Technical Buyer Ref: 3884 posted by NDC Personnel
Yoursquoll need Matric engineering qualifications are advantageous, as well as 3 ndash 5 yearsrsquo experience in the FMCG Transportation sectors in a...
View Job
Technical Buyer Ref
Cape Town: Technical Sales Representative (Granular Fertilizers & Biostimulants) – Western Cape posted by Mirna Butler Recruitment
Are you passionate about agriculture and eager to help farmers succeed? Join my clients dynamic team leading the way in granular fertilizers and...
View Job
Technical Sales Representative (Granular Fertilizers & Biostimulants) – Western Cape
Cape Town: Back Office Employee posted by abc worldwide
Back Office Employee mfd for Consulting Firm Location Remote Home Office Start Date ImmediatelyWorking Hours Full-time Our goal is to guide doctors through...
View Job
Back Office Employee
Cape Town: Safer Gambling Analyst – Remote posted by Hire Resolve
Hire Resolves client is urgently seeking the expertise of a Safer Gambling Analyst for a fully remote position.ResponsibilitiesCommunicating with customers...
View Job
Safer Gambling Analyst – Remote
Cape Town: Site Engineer posted by Hire Resolve
Hire Resolves client, who offers the highest quality construction, project management and specialist subcontracting services, is searching for a Site...
View Job
Site Engineer
Cape Town: Project Manager posted by Hire Resolve
Hire Resolves client is a leading player in the engineering industry, renowned for delivering high-quality infrastructure and industrial projects across...
View Job
Project Manager
Cape Town: Projects Contracts Manager posted by Hire Resolve
Hire Resolves client is looking for a ProjectsContracts Manager to join their team in Cape Town. Theynbspare a construction industry and mainly serve...
View Job
Projects Contracts Manager
Cape Town: Operations Administrator posted by AJ Roodt Consulting (Pty) Ltd
Position available for an experienced Administrator to handle daily admin and operations functions.Candidates must have previous working experience in the...
View Job
Operations Administrator
Cape Town: Junior Administrator – Urgently Needed posted by Nu-Line Elevator Maintenance
Nu-Line Elevator Maintenance is looking for a Junior Administrator to join our team in Killarney Gardens, Cape Town. If youre passionate about learning and...
View Job
Junior Administrator – Urgently Needed
Cape Town: Facility / Office Manager (6 month fixed term contract) posted by O’Brien Recruitment
Are you a facilities mastermind with a passion for creating smooth, well-oiled office environments? Do you thrive on managing multiple sites, juggling...
View Job
Facility / Office Manager (6 month fixed term contract)
Western Cape: DIESEL TECHNICIAN – COMMERCIAL TRUCKS (CAPE TOWN) posted by MPRTC Recruitment
Minimum RequirementsMatricDiesel Trade Test completed2ndash3 years relevant experienceValid Code 14 Driverrsquos License with PDPSolid technical knowledge...
View Job
DIESEL TECHNICIAN – COMMERCIAL TRUCKS (CAPE TOWN)
Cape Town: Brand Training Specialist posted by O’Brien Recruitment
Were Hiring Brand Training Specialist Cape Town ParowOffice-based Employment Equity Applicable All Genders and Races WelcomeWere seeking a dynamic Brand...
View Job
Brand Training Specialist
Cape Town: Engineering Manager Ref: 3974 posted by NDC Personnel
With 5 years of experience and a strong skill-set in project management, planning, and organisation, yoursquoll be a great fit! Effective communication is...
View Job
Engineering Manager Ref
Cape Town: Senior Quantity Surveyor (Building Construction) – CPT posted by Hire Resolve
Hire Resolves client, a well-established and dynamic construction company known for delivering high-quality residential and commercial building projects, is...
View Job
Senior Quantity Surveyor (Building Construction) – CPT
Cape Town: Senior Foreman posted by Hire Resolve
Hire Resolves client is looking for a Senior Foreman to join their team in Cape Town. The company always ensures the best results for their clients in the...
View Job
Senior Foreman
Cape Town: Project Manager posted by Sharon Nurock Recruitment cc
Main Purpose of the RoleOversee the daily workings of the organisations operations.Responsible to maintain a high-quality standard of the organisations...
View Job
Project Manager
Cape Town: General Worker posted by Cre8work!
MATRIC, ID, UI19, CONTACTABLE REFERENCES, CLEAR CRIMINAL RECORD SA CITIZEN SHIFT WORK, HOURS ARE FROM 07h00 TO 19h00 OR 19h00 TO 07h00 HARD LABOUR STRONG...
View Job
General Worker
Cape Town: T/A Merchandiser – Retail posted by Recruitment Matters
Our client is looking for a dynamic and proactive TA Merchandiser to support the retail sales team through effective merchandising, store execution, and...
View Job
T/A Merchandiser – Retail
Cape Town: Technical Sales Representative posted by Time Personnel
REQUIREMENTSMatric,Valid drivers license and own transportation3 4 years experience in a similar industryExceptional customers communication and...
View Job
Technical Sales Representative
Cape Town: Reach Truck Drivers posted by Tailormade Solution
We are Hiring Reach Truck Drivers The Reach Truck Operator is responsible for operating a reach truck to move, locate, relocate, and stack merchandise in...
View Job
Reach Truck Drivers
Cape Town: Player Success Agent posted by Hire Resolve
Job Title Player Success AgentThe Player Success Agent is responsible for managing daily operational assessments, customer interactions, and reviews to...
View Job
Player Success Agent
Cape Town: Senior Foreman posted by Hire Resolve
Hire Resolves client whonbspoperates out ofnbsp Wetton, Cape Town and offers the highest quality construction, project management and specialist...
View Job
Senior Foreman
Cape Town: Mechanical Design Technologist/Engineer posted by Hire Resolve
Hire Resolves client is looking for a Mechanical Design TechnologistEngineer to join their team in Cape Town. This client is a multi-disciplinary firm...
View Job
Mechanical Design Technologist/Engineer
Cape Town: Mechanical Design Technologist/Engineer (Consulting) – CPT posted by Hire Resolve
Hire Resolves client, a reputable and forward-thinking consulting engineering firm known for its commitment to innovation and excellence in building...
View Job
Mechanical Design Technologist/Engineer (Consulting) – CPT