Cape Town: Principal AWS Data Engineer Lead posted by PBT Group
Posted on 2025-05-21 00:00:00
Employer | PBT Group |
---|---|
Category | Other Finance/Accounting |
Location | Western Cape / Cape Town |
Job Summary
PBT Group has an opportunity for a Principal AWS Data Engineer Lead, must have experience with AWS, AWS Native, Glue ETL, Redshift, Lambda functions.
Duties:
- Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.
- Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
- Design and build production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala.
- Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
- Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
- Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
- Creation and support of real-time data pipelines built on AWS technologies including Glue, Redshift/Spectrum, Kinesis, EMR and Athena
- Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency
- Working closely with team members to drive real-time model implementations for monitoring and alerting of risk systems.
- Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning
- Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers
Experience:
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
- Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
- Experience working with distributed systems as it pertains to data storage and computing
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation, data structures, meta data, dependency and workload management.
- A successful history of manipulating, processing and extracting value from large disconnected data sets.
- Working knowledge of message queuing, stream processing, and highly scalable Big Data, data stores.
- Strong project management and organizational skills.
- Experience supporting and working with cross-functional teams in a dynamic environment.
- Experience in a Data Engineer or similar roles
- Experience with big data tools is a must: Hadoop, Spark, Kafka, etc.
- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
- Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Experience with AWS cloud services: EC2, EMR, RDS, Redshift
- Experience with stream-processing systems: Storm, Spark-Streaming, etc.
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc
Qualifications/ Certification:
- Bachelor’s Degree in Computer Science, Information Technology or other relevant fields
- Has experience in any of the following AWS Athena and Glue Pyspark, EMR, DynamoDB, Redshift, Kinesis, Lambda, Snowflake
- Proficient in AWS Redshift, S3, Glue, Athena, DynamoDB, EMR
- Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations
Click Go Apply to apply online!
Get Similar Job Alerts
More jobs in Cape Town
More jobs in Western Cape
Browse Other Finance/Accounting Jobs
Browse PBT Group Jobs
Apply directly for this position. Please read all instructions carefully.
We do not process job applications; we simply aggregate and display job listings.
More related positions
Cape Town: National Key Accounts Sales Manager posted by The Individual Touch CC
Generate new additional businessGrow the national customer baseManaging analyzing the national customer baseAnalyzing price decreases suspended business...
View Job
National Key Accounts Sales Manager
Cape Town: Junior Accountant posted by Africa Thinking Management Solutions
Job Advertisement Junior AccountantLocation Cape Town -Observatory Company Africa Thinking Management SolutionsAfrica Thinking Management Solutions is...
View Job
Junior Accountant
Cape Town: Intermediate Financial Administrator posted by Construct Executive Search
Applicants must have the following requirementsMust have a Diploma in Accounting or Bookkeeping.Must have 3-5 years of experience as a Financial...
View Job
Intermediate Financial Administrator
Eastern Cape: Town Planner (JB5167) posted by Kontak Recruitment
Town Planner JB5167 Chintsa, Eastern CapeCompensation Range R40 45 000 CTC per month Negotiable Category EngineeringLead zoning, rezoning, and land use...
View Job
Town Planner (JB5167)
Cape Town: Roof sheeter – Urgently Needed posted by Batt Master Roofing and Guttering
Roof sheeter needed Must be able to remove and install roof sheets etc and waterproofing for maintenance company Work in fascias as wellWork experience a...
View Job
Roof sheeter – Urgently Needed
Cape Town: Litigation Secretary posted by Marais Muller Hendricks Attorneys
Marais Muller Hendricks Inc., a leading law firm in Cape Town, is seeking to appoint an experienced and highly organised Litigation Secretary to join our...
View Job
Litigation Secretary
Cape Town: Bookkeeper – Restaurant Industry (Salt River) posted by The Vocation Station
About the RoleWere looking for a detail-oriented Bookkeeper to join our clients team and support the financial operations of four businesses under one roof....
View Job
Bookkeeper – Restaurant Industry (Salt River)
Cape Town: Litigation Secretary posted by Cora O’Neil Recruitment Agency
Cape Town City Centre LITIGATION SECRETARY Minimum Requirements-Well presentable and professional preferably female-Excellent verbal communication skills-2...
View Job
Litigation Secretary
Cape Town: Electrical Engineer – Building Services posted by Business Recruitment Services
This role requires collaboration with multidisciplinary teams and clients to deliver fit-for-purpose solutions that surpass industry best practices and...
View Job
Electrical Engineer – Building Services
Cape Town: Intermediate SHERQ Officer posted by Construct Executive Search
Applicants must have the following requirementsMust have a Bachelors Degree or National Diploma in Safety Management or Occupational Health and Safety. Must...
View Job
Intermediate SHERQ Officer
Cape Town: Financial Accountant posted by Executive Placings
Travel Management Company is looking for an Accountant to be based in Cape Town. This is a hybrid role.The main purpose of this role is to ensure accuracy...
View Job
Financial Accountant
Cape Town: Regional Manager in Built-in Furniture Manufacturing Industry posted by Home Concept
Home Concept is looking for an experienced Regional Manager to join our team. In this role, youll lead sales, manage operations, and support teams to ensure...
View Job
Regional Manager in Built-in Furniture Manufacturing Industry
Cape Town: New Business Consultant posted by Intercon Recruitment
Company based in Cape Town is looking for a Senior New Business Consultant Sales Increase and drive new business with a focus on quality and...
View Job
New Business Consultant
Cape Town: Car Rental Agent – Urgent posted by Vogue Personnel
Join Our Team as a Car Rental Agent! Were looking for a dedicated and organized Car Rental Agent to join our dynamic team and play a key role in ensuring...
View Job
Car Rental Agent – Urgent
Cape Town: New Vehicle Sales Executive posted by Intercon Recruitment
Dealership in Cape Town is looking for an experienced New Vehicle Sales Executive with previous Motor Dealership New Vehicle Sales Experience. Salary Basic...
View Job
New Vehicle Sales Executive
Cape Town: Tour Operator – Urgent posted by Vogue Personnel
Exciting Career Opportunity in Southern Africa Travel!Are you passionate about travel and adventure? Do you have the skills and drive to make unforgettable...
View Job
Tour Operator – Urgent
Cape Town: Senior Construction Contracts Manager posted by Construct Executive Search
Applicants must have the following requirementsMust have a Bachelors Degree or Diploma in Building or Civil Engineering.Must be registered with SACPCMP as a...
View Job
Senior Construction Contracts Manager
Cape Town: Business Development Manager – Urgently Needed posted by One Source ICT
One Source ICT is an innovative technology company specializing in Information and Communication Technology ICT solutions. With a focus on excellence and...
View Job
Business Development Manager – Urgently Needed
Cape Town: Sales Representative | New Business Consultant posted by Intercon Recruitment
Company based in Cape Town is looking for a Sales Representative New Business Consultant to Increase Sales and drive new business with a focus on quality...
View Job
Sales Representative | New Business Consultant
Cape Town: SALES MANAGER posted by The Recruitment Guy
Title SALES MANAGERArea Cape Town PE HubsIndustry Rental Hire Working of height machinery and equipment Trade Desk Ref No. TRG 2303Salary TCT R71...
View Job
SALES MANAGER
Cape Town: IT Manager – Urgently needed posted by Edelson & Associates
IT Manager Cape Town R45000 NEG Working Knowledge of Microsoft Admin Centres and Licensing. Knowledge of SQL database Management. Also on site...
View Job
IT Manager – Urgently needed
Cape Town: SITE AGENT posted by Furipoint pty,ltd
Furipoint is excited to announce that we are currently expanding our team and are looking for highly motivated and skilled professionals to fill the Site...
View Job
SITE AGENT
Cape Town: QUANTITY SURVEYORS posted by Furipoint pty,ltd
Furipoint is excited to announce that we are currently expanding our team and are looking for highly motivated and skilled professionals to fill the...
View Job
QUANTITY SURVEYORS
Cape Town: Quality control inspector – PLEASE READ BEFORE APPLYING posted by Prokon
Position Quality Control Inspector Fresh Fruits Vegetables? Location Epping Private Market, Cape Town You need to reside in the Western Cape? Starting Date...
View Job
Quality control inspector – PLEASE READ BEFORE APPLYING
Cape Town City Centre: Accounts Payable Specialist
Key Responsibilities Process and verify invoices, expense reports, and payment requests Reconcile vendor statements and resolve discrepancies Maintain...
View Job
Accounts Payable Specialist