Menu Close

Cape Town: Data Engineer : Ingestions Squad (Group Data & ML Platform Tribe) posted by Progressive IT Resourcing

Data Engineer : Ingestions Squad (Group Data & ML Platform Tribe)

Posted on 2025-04-27 13:18:52

Company Progressive IT Resourcing
Salary 0
Category Other IT/Computer
Location South Africa  /  Western Cape  /  Cape Town

Job Summary

Job Details

Who are we?

Company Group Technology is responsible for the provision of a digitally enabled technology service as a group COE, drive business and transformation and provide group-wide digital and data architecture. We operate the various technology platforms and shared services, ensure Cyber and Information Security resilience, and act as technology governance and risk orchestrator for technology across Company

.What will you do?

Were a newly created engineering organisation on a mission to empower and accelerate autonomous value creation through insights and ML across the group. Our main users are the data scientists and analysts who sit inside the business units and support them in developing and re-developing their financial and digital products to better meet the needs of our clients and our shareholders. Our other users are the data management and data product teams who produce the data sets analysts and scientists need to launch straight into their work.

The tribe is made up of 3 different engineering squads covering everything from how we ingest and store data, to how we develop and share ML artefacts and insights. We have one for ingestions, one for data storage and processing and one for insights & ML tooling. We have one data engineering talent pipeline for the whole tribe, where placements into specific squads are explored later in the interview process based on squad needs and candidate abilities. Data engineers report to the tech leads of the squad they are embedded into.

What will make you successful in this role?

What success will look like for you 12 months into the role.

  • Built configurable data ingestion platforms to collect and protect data from any source within two weeks or less. The team has bought and built to assemble a tool suite of persistent and secure ingestion tools, optimized for cost efficiency, with reliability as a non-negotiable standard. Success will be measured by the consistent achievement of the two-week ingestion target while maintaining high reliability and security.
  • Engineered the platform for observability and performance. Made your systems observable from the outside, taking ownership of their performance. Facilitated real-time tracking of the systems internal state through logs, metrics, and traces. Defined clear and measurable Service Level Indicators (SLIs) and Service Level Objectives (SLOs), and integrate these into team OKRs to drive excellence. Success will be measured by meeting or exceeding defined performance and reliability benchmarks, and the quick identification and resolution of issues.
  • Engineer the platform with privacy and security by design. Secured the platform through compliance engineering initiatives that reduced the time and cognitive load related to compliance. Implemented our Data Governance and Privacy policies effectively, leveraging encryption, firewalls, VPCs, Role Based Access Control and IAM integrations such as Active Directory. Success will be measured by the platforms adherence to security and privacy standards, with minimal compliance overhead for users.
  • Leveraged containerization and infrastructure as code. Implemented containerization strategies and infrastructure as code (IaC) practices to ensure scalable, repeatable, and efficient deployment of data services. Utilized tools like Docker and Terraform to automate infrastructure management and streamline the development pipeline. Success will be measured by the consistency and reliability of deployments, reduced deployment times, and increased system scalability.
  • Contributed to technology decisions. Engaged in technology spikes to clarify needs and articulate solution criteria. Evaluated different options through hands-on experimentation, leading to documented evaluations and recommendations that were understood by technology leaders or users across the company. Success will be measured by the quality and clarity of your contributions to technology decisions and the adoption of your recommendations.
  • Invested in your own development. Executed on a culture of excellence and continuous improvement for both your part of the platform and your own abilities. Lived our culture of ownership, curiosity, and continuous learning. Participated in pre and post-mortems, sharing lessons learned across the organization along with how you are improving based on those lessons. Success is measured by your active participation in continuous learning initiatives and the tangible improvements in platform performance and personal growth.
View Job  Randburg: Full Stack AWS Developer posted by Hire Resolve

Qualification

  • Matric
  • National Diploma in an Information Technology related discipline or a bachelor’s degree in computer science, Statistics, Informatics, Information Systems, or a quantitative field will be recommended.

Experience

  • 5+ years’ experience as a data engineer in data platform or service engineering team with at least 2 in a modern context with cloud technologies
  • Track record of high impact and investing in your own development.

Skills and Thinking Preference

  • You look for ways to do, instead of reasons not to. You are high agency, high ownership in everything you do. You know we own our future, only we can make it happen. If you find something that keeps us from our mission, you own it or work collaboratively with others until you find the right owner of it.
  • Where some see impossible, you see a way. You have the strong conviction in our ability to innovate and bring financial services, and financial confidence to everyone across the continent. You turn obstacles into opportunity, chance into change.
  • You move fast and fix things. You publish early and often, when you fail you learn and go again. You know that speed gives you the licence to fail, because youre moving fast enough to correct course.
  • You love creating more with less. Knowing that the biggest impact is born of the smallest acts, scale is built into your thinking from day 1. This is not about meeting the needs of thousands; its about exceeding the needs of millions. And working through small and mighty teams is way well do it.
  • You seek out and value different perspectives. Believing that when we change how we look at things, how things look for us also changes. Making tomorrow not just different, but better.

Personal Attributes

  • Plans and aligns – Contributing through others
  • Optimises work processes – Contributing through others
  • Collaborates – Contributing strategically
  • Resourcefulness – Contributing strategically
  • Tech savvy – Contributing strategically
Job Seeker Tip

Consider temporary or contract work to gain experience and potentially convert to permanent roles.

Click Go Apply to apply online!

Apply directly for this position. Please read all instructions carefully.

We do not process job applications; we simply aggregate and display job listings.

More related positions


Midrand: Senior Manager Data Engineer posted by Deka Minas (Pty) Ltd

RESPONSIBILITIES

  • Create and execute a long-term strategy for leveraging data and analytics as a key business enabler and competitive differentiator across the business.
  • Take responsibili


    View Job
    Senior Manager Data Engineer

Pretoria: Data Engineer posted by IT Ridge Technologies

Role overview:

We are looking for an experienced Data Engineer to design and manage a resilient data pipeline that effectively processes and integrates both structured and unstructured data into data lakes, warehouses, databases,


View Job
Data Engineer

Johannesburg: Data Engineer: 12 Month FTC posted by Isilumko Staffing

Description

The Group Data and Analytics Office is on the lookout for a skilled Data Engineer. The ideal candidate will have extensive experience in data engineering, particularly with SAS, SQL and Oracle, and will be responsible


View Job
Data Engineer

Johannesburg: Ab Initio & Oracle Data Engineer posted by PBT Group

We are seeking a skilled Data Engineer with expertise in Oracle PL/SQL, Ab Initio, and Power BI to join our team. The ideal candidate will be responsible for designing, developing, and maintaining robust data solutions that support business intelligenc


View Job
Ab Initio & Oracle Data Engineer

Johannesburg: Senior Data Engineer posted by Deka Minas (Pty) Ltd

Senior Data Enbgineer required in Sandton for a contract. This role is pivotal in driving our data strategy and ensuring the efficient and effective management of our data infrastructure. We are looking for a proactive, innovative, and solution-oriente


View Job
Senior Data Engineer

Cape Town: Data Engineering & Streaming Lead posted by PBT Group

The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a se


View Job
Data Engineering & Streaming Lead

Cape Town: AWS Data Engineer posted by PBT Group

AWS Data Engineer Opportunity at PBT Group

Are you a skilled Data Engineer with AWS expertise looking for an opportunity to grow your career in a dynamic, consultancy-driven environment? PBT Gr


View Job
AWS Data Engineer

Pretoria North: AWS Data Engineer posted by CodeConnect Staffing (Pty) Ltd

Overview

An exciting opportunity has become available for an experienced AWS Data Engineer. In this role, you'll be responsible for building and managing scalable data systems, from setting up data sources to integrating analytic


View Job
AWS Data Engineer

Johannesburg North: Data Engineer posted by Network Finance

We are seeking a highly skilled and experienced Data Engineer to join our team. In this role, you will be responsible for developing and maintaining scalable, secure, and repeatable data pipelines. You will manage data platforms, ensuring the accuracy and


View Job
Data Engineer

Centurion: Data Engineer posted by Communicate Finance


Job Description:
Design, build, and maintain robust data pipelines and ETL processes
Work with AWS services (S3, Glue, Redshift, Lambda, etc.) to manage and optimize data infrastructure
Write efficient, product


View Job
Data Engineer

Johannesburg North: AWS Data Engineer posted by Network Finance

Minimum Experience:

  • 5 years of experience in Python/C# development
  • View Job
    AWS Data Engineer

Johannesburg North: Data Engineer posted by Network Finance

FinTech | Data Engineer | South Africa (Remote/Hybrid) | Python, AWS & PySpark Experience a Must
An innovative FinTech firm is hiring a Data Engineer to drive data infrastructure initiatives and streamline data processes. If you


View Job
Data Engineer

Western Cape: Data Engineer posted by Network Finance

FinTech | Data Engineer | South Africa (Remote/Hybrid) | Python, AWS & PySpark Experience a Must
An innovative FinTech firm is hiring a Data Engineer to drive data infrastructure initiatives and streamline data processes. If you


View Job
Data Engineer

Durban North: Data Engineer posted by Network Finance

FinTech | Data Engineer | South Africa (Remote/Hybrid) | Python, AWS & PySpark Experience a Must
An innovative FinTech firm is hiring a Data Engineer to drive data infrastructure initiatives and streamline data processes. If you


View Job
Data Engineer

Centurion: Data Engineer posted by Network Finance

Responsibilities:

  • Designing, building and maintaining scalable data pipelines to process large volumes of data from various sources 
  • Develop and Maintain data models databases and data warehouses to support the analytics teams re


    View Job
    Data Engineer

Centurion: Data Engineer posted by Network Finance

Why Join Us?

  • Top Employer South Africa since 2022
  • Inclusive and vibrant company culture
  • Rewards and recognition programs
  • Comprehensive employee benefits (Medical Aid, Pension Fund, Life & Disabilit


    View Job
    Data Engineer

Gauteng: Data Engineer posted by Network Finance

Company & Role Description:
My client is part of South Africa's top insurance group, the business unit offers  wealth-building investment and retirement solutions. As a Data Engineer, you'll manage an


View Job
Data Engineer

Centurion: Data Engineer posted by Communicate Finance


In this role, you’ll play a key part in developing robust data pipelines, integrating diverse data sources, and supporting a modern data platform that’s ready for BI, analytics, and machine learning.

Skills & Ex


View Job
Data Engineer

Centurion: Data Engineer posted by Communicate Finance


Designing and implementing robust, scalable data pipelines and architectures
Managing and optimizing data storage using AWS S3
Creating and tuning queries using AWS Athena
Working closely with analysts, engineers, and stakeholders to


View Job
Data Engineer

Centurion: 2063 AWS Data Engineer posted by Jordan Human Resources


Product / Feature Team Information (if applicable)
Our Domain, Enterprise Data Services, Platforms, comprises of a number of Data Business Objects, namely, Quality, Motorbike, Purchasing & Supplier, Marketing & Research, Finance, Sales,


View Job
2063 AWS Data Engineer

Johannesburg: Data Engineer posted by Goldman Tech

Job Description:
We have an amazing opportunity for a Data Engineer to work Remotely!

We require a candidate with experience in:
• A cloud environment (AWS, Azure)
• Python, R, Scala
• SQL (MS SQL Server, Postgres


View Job
Data Engineer

Cape Town: Data Streaming Engineer posted by PBT Group

Are you a skilled Data Streaming Engineer passionate about real-time data processing and cutting-edge technology? Join our PBT team and play a pivotal role in shaping data streaming architecture. This position offers the opportunity to work with indust


View Job
Data Streaming Engineer

Cape Town: 3rd Line Technical Support Engineer (CPT Hybrid) posted by Datafin

3rd Line Technical Support Engineer (CPT Hybrid)IT - Support
Cape Town - Western Cape - South Africa
ENVIRONMENT:
OUR client is a thriving Managed Service Provider company dedicated to


View Job
3rd Line Technical Support Engineer (CPT Hybrid)

Johannesburg: Intermediate Java Engineer (JHB Onsite) posted by Datafin

Intermediate Java Engineer (JHB Onsite)IT - Software Development
Johannesburg - Gauteng - South Africa
ENVIRONMENT:
A dynamic Software Specialist in Joburg seeks the coding talents of


View Job
Intermediate Java Engineer (JHB Onsite)

Cape Town: Senior JavaScript Software Engineer posted by Datafin

Senior JavaScript Software EngineerEngineering/Technical
Cape Town - Western Cape - South Africa
ENVIRONMENT:
Join a dynamic, multidisciplinary environment that fosters versatility and


View Job
Senior JavaScript Software Engineer

Error making API request: cURL error 6: Could not resolve host: publisher.resgen.us
Share this to someone who needs a job:
Posted in Jobs in Cape Town, Jobs in South Africa, Jobs in Western Cape

More Jobs in Your Area