Menu Close

Cape Town City Centre: Data Engineering & Streaming Lead

Data Engineering & Streaming Lead

Posted on 2025-02-04

Category It Computing Software

Location

Job Summary

The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank`s continued growth through efficient data handling. The role also includes collaborating with multiple teams to understand data needs and translate them into scalable platforms and services. Duties & Responsibilities Operational Delivery Build, enhance, and maintain real-time data pipelines. Work with infrastructure and operations teams to maintain data infrastructure. Identify, document, and resolve feature gaps while implementing solutions. Modernize and maintain services and tooling to ensure resilience, fix data discrepancies, and enhance customer experience. Monitor daily execution, diagnose and log issues, and fix pipelines to ensure SLAs are met. Mentor junior engineers through code reviews, best practices, and knowledge sharing. Technical Leadership Participate in the engineering community of practice. Share AWS knowledge and practical experience within the community. Challenge and contribute to the development of architectural principles and patterns. Compliance Ensure solutions adhere to Olympus patterns, guidelines, and standards. Operate within project environments and participate in continuous improvement efforts. Delivery Management Follow and participate in Agile methodologies such as sprint planning, backlog grooming, retrospectives, demos, and PI planning. Skills & Experience Required Essential: Strong AWS Data Engineering experience. Hands-on experience with Kafka and Confluent. Strong expertise in real-time data streaming tools such as Kafka & Flink. Proficiency in: Apache Spark Apache Storm Cloudera Data Flow Databricks Hadoop AWS Kinesis Streaming Experience in developing cloud-based solutions. 2-3 years of experience designing and developing streaming data pipelines for ingestion or transformation using AWS technologies. Experience with distributed log systems such as Kafka and AWS Kinesis. Qualifications & Certifications A degree in Computer Science, Information Systems, Business Administration, Commerce, or equivalent. Bachelor’s Degree in Computer Science, Big Data, or similar fields. AWS Data Engineer Certification (Advantageous). Relevant technical certifications (Preferred). In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent If you have not received any feedback after 2 weeks, please consider you application as unsuccessful.

View Job  Cape Town City Centre: Accountant

Data Engineering & Streaming Lead position available in Cape Town Region, Cape Town City Centre. This job position was posted by . The job has been posted on 2025-02-04 in the It Computing Software category

Click Go Apply to apply online!

You might also like these jobs in the same area.

More jobs in Cape Town City Centre

More jobs in Cape Town Region

Browse It Computing Software Jobs

Apply directly for this position. Please read all instructions carefully.

We do not process job applications; we simply aggregate and display job listings.

More related positions

Johannesburg: Data Engineering Specialist

Experience using Kimball Experience working on Microsoft Dynamics 365 Experience working with Azure cloud-based data platform and Azure Databricks (data bricks is advantageous) Inmon would be advantageous but not essential Data Engineering Specialist posit


View Job
Data Engineering Specialist

Johannesburg: Data Engineering Specialist

Requirements: A Masters Degree in Computer Sciences or Data Sciences Certifications in cloud platforms (e.g., Azure) and Microsoft products (e.g., Power BI, Dynamics365) Responsibilities: Responsible to design and build data pipelines and efficient data fl


View Job
Data Engineering Specialist

Johannesburg: Data Engineering Specialist posted by The Skills Mine

Requirements:A Masters Degree in Computer Sciences or Data SciencesCertifications in cloud platforms (e.g., Azure) and Microsoft products (e.g., Power BI, Dynamics365)Responsibilities: Responsible to design and build data pipelines and efficient data flow


View Job
Data Engineering Specialist

South Africa: Engineering Data Controller (Permanent) posted by Profile Personnel

Brief Role DescriptionTo ensure that all activities set out by Supervision are undertaken and completed by the Engineering Data Controller to ensure all data bases are completed on time and to provide product support regarding technical information company

View Job  Cape Town City Centre: New Business Development Consultant

View Job
Engineering Data Controller (Permanent)

Cape Town: Project Engineering Assistant Cpt Onsite posted by Datafin

Project Engineering Assistant - CPT - OnsiteEngineering/TechnicalCape Town - Western CapeENVIRONMENT: A company committed to understanding, supporting, and empowering small business owners and entrepreneurs in the food, beverage, pharmaceutical, and allied


View Job
Project Engineering Assistant Cpt Onsite

Amsterdam: Engineering Project Manager (Ship Building) (Netherlands) posted by Datafin

Engineering Project Manager (Ship Building) (Netherlands)Engineering/Technical ~ Product/Project managementAmsterdam - NetherlandsENVIRONMENT: If you have Shipbuilding experience and want to work in the Netherlands, then this is the role for you!A dynamic


View Job
Engineering Project Manager (Ship Building) (Netherlands)

Johannesburg: Intermediate Electronics Software Engineer (C++ And C#) (Engineering Degree) posted by Datafin

Intermediate Electronics Software Engineer (C++ and C#) (Engineering Degree)IT - Software DevelopmentJohannesburg - GautengENVIRONMENT: DEFINE technical proposal content, ensure teams have appropriate product & technical specs and the flawless execution of


View Job
Intermediate Electronics Software Engineer (C++ And C#) (Engineering Degree)

Johannesburg: Junior Electronics Software Developer (C++ And C#) (Engineering Degree) posted by Datafin

Junior Electronics Software Developer (C++ and C#) (Engineering Degree)IT - Software DevelopmentJohannesburg - GautengENVIRONMENT: DEFINE technical proposal content, ensure teams have appropriate product & technical specs and the flawless execution of proj


View Job
Junior Electronics Software Developer (C++ And C#) (Engineering Degree)

Cape Town: Software Engineering Team Lead (C#/Java/Python) posted by Datafin

Software Engineering Team Lead (C#/Java/Python)IT - Software DevelopmentCape Town - Western CapeENVIRONMENT: THE coding expertise of a hands-on, passionate and self-driven Software Engineering Team Lead is sought to lead the Front Office Research Dev team


View Job
Software Engineering Team Lead (C#/Java/Python)

Cape Town City Centre: Data Engineering & Streaming Lead

The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior eng

View Job  Cape Town City Centre: Senior Quantity Surveyor - Prqs Registered (Cape Town)

View Job
Data Engineering & Streaming Lead

Cape Town: Data Engineering & Streaming Lead posted by PBT Group

The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior


View Job
Data Engineering & Streaming Lead

Cape Town: Director Of Engineering (Automotive) posted by Datafin

Director of Engineering (Automotive)Engineering/TechnicalCape Town - Western CapeENVIRONMENT: A dynamic Engineering Service Provider in Durbanville is seeking a Director of Engineering to join their team and align with the company`s culture, which thrives


View Job
Director Of Engineering (Automotive)

Johannesburg: Senior Data Engineering Specialist

Roles and Responsibilities Technical Leadership: Cloud Architecture & Data Engineering Cloud Solution Design: Architect scalable, secure, and cost-effective cloud solutions using Azure technologies, addressing multi-domain requirements. Key services includ


View Job
Senior Data Engineering Specialist

Gauteng: Data Analyst (Engineering/Construction) posted by Lotus HR and Recruitment

KEY REQUIREMENTS TO MEET FOR CONSIDERATION: You will have a tertiary qualification in Information Technology or Computer Science, coupled with 3-5 years experience in data analysis of project data.Proven expertise in statistical software and programming la


View Job
Data Analyst (Engineering/Construction)

Johannesburg: Data Analyst (Engineering/Construction)

KEY REQUIREMENTS TO MEET FOR CONSIDERATION: You will have a tertiary qualification in Information Technology or Computer Science, coupled with 3-5 years experience in data analysis of project data. Proven expertise in statistical software and programming l


View Job
Data Analyst (Engineering/Construction)

Error making API request.
Share this to someone who needs a job:
Posted in Jobs in Cape Town City Centre, Jobs in Cape Town Region

More Jobs in Your Area