Midrand: Apache Airflow Specialist posted by Sabenza IT & Recruitment
Posted on 2025-07-31 00:00:00
Employer | Sabenza IT & Recruitment |
---|---|
Category | Other IT/Computer |
Location | Gauteng / Midrand |
Job Summary
This is a remote position.
Join Our Data Engineering Crew as an Apache Airflow Optimization Specialist!
As our Airflow Optimization Specialist, youll be the maestro of our data orchestration, fine-tuning our Apache Airflow environment to deliver high-performance, reliable, and scalable solutions. Youll collaborate with our data engineering and platform teams, wield Azures powerful tools, and craft CI/CD pipelines that make data flow smoother than a summer breeze. Ready to make an impact?
Heres what youll be doing:
- Analyse and optimize the current Apache air flow environment, identify performance bottlenecks and implement best practices for orchestration and scheduling
- Design and implement scalable modular and reusable DHG’s to support complex data workflows
- Collaborates with data engineers and platform teams to integrate airflow with Azure data factory Azure data bricks and other Azure-native services
- Develop and maintain CI CD pipelines using Azure DevOps for air flow DAG deployment testing and version control.
- Establish monitoring alerting and logging standards for air flow jobs to ensure operational excellence and rapid incident response
- Provide architectural guidance and hands-on support for new data pipeline development using air flow and Azure servicesDocuments air flow configurations deployment processes and operational runbooks for E internal teams
- Mentor engineers and contribute to knowledge sharing sessions on orchestration and workflow management.
Required skills and qualifications
- Proven experience as an Apache air flow SME or lead developer in a production-grade environment with a strong understanding of air flow internals includin
- scheduler executor types (Celery Kubernetes) and plug-in developments.
- Experience with workload orchestration and auto scaling using K EDA based event driven auto scaler and familiarity with celery to distributed task execution and background job processing particularly in data pipeline or micro services environments
- Hands-on experience with Azure cloud services, especially as your data factory as your data bricks, as your storage and as your synapse
- Proficiency in designing and deploying CI CD pipelines using Azure DevOps (YAML pipelines, release management, artifact handling)
- Solid programming skills in Python with experience in writing modular testable and reusable code.
- With containerization (Docker) and orchestration (Kubernetes) as it relates to airflow deployment
If youre ready to optimize, orchestrate, and dominate the data pipeline game, we want YOU! Apply now and lets make data magic together.
Click Go Apply to apply online!
Get Similar Job Alerts
More jobs in Midrand
More jobs in Gauteng
Browse Other IT/Computer Jobs
Browse Sabenza IT & Recruitment Jobs
Apply directly for this position. Please read all instructions carefully.
We do not process job applications; we simply aggregate and display job listings.
More related positions
Midrand: Security Operational Manager GRADE A&B ONLY – Urgently needed posted by Spartak Investments
Spartak Security seeks experienced Operational Manager. Min 5 yrs security management, PSIRA Grade A B.IF YOU DONT HAVE ALL OF THE DOCUMENTS LISTED BELOW...
View Job
Security Operational Manager GRADE A&B ONLY – Urgently needed
Midrand: Podcast Host posted by Exceed Education
Host a weekly podcast episode Prepare notes and discussion pointsInterview guests and moderate discussionsCollaborate on content marketing and promotion...
View Job
Podcast Host
Midrand: Cybersecurity Analyst / Engineer posted by Recruitpro Solutions
Duties amp ResponsibilitiesMonitor and analyse network traffic and security eventsConduct vulnerability assessments and risk auditsImplement firewalls,...
View Job
Cybersecurity Analyst / Engineer
Midrand: Data Engineer posted by Recruitpro Solutions
Duties amp ResponsibilitiesDevelop and maintain robust data pipelinesDesign scalable architectures for structured and unstructured dataEnsure data quality,...
View Job
Data Engineer
Midrand: DevOps Engineer posted by Recruitpro Solutions
Duties amp ResponsibilitiesImplement and manage CICD pipelinesAutomate infrastructure provisioning and monitoringMaintain Kubernetes clusters and Dockerised...
View Job
DevOps Engineer
Midrand: Cloud Architect / Cloud Engineer posted by Recruitpro Solutions
Duties amp ResponsibilitiesDesign and implement cloud-native architecture solutionsMigrate on-prem systems to the cloudBuild and manage...
View Job
Cloud Architect / Cloud Engineer
Midrand: AI / Machine Learning Engineer posted by Recruitpro Solutions
Duties amp ResponsibilitiesDesign, build, and deploy machine learning models in production environmentsWork with data scientists and software engineers to...
View Job
AI / Machine Learning Engineer
Midrand: Operations Fleet Controller – Abnormal Load posted by Objective Personnel
RequirementsMinimum 5 yearsrsquo experience of fleet control within the abnormal load transport environment multi-axleProficient in managing modular trailer...
View Job
Operations Fleet Controller – Abnormal Load
Centurion: Point-Of-Sales Menu Administrator posted by Express Employment Professionals Midrand
Duties amp ResponsibilitiesLoad items and update menu, pricing and recipes on the POS SystemDeploy menu changes across all stores efficientlyEnsure accuracy...
View Job
Point-Of-Sales Menu Administrator
Midrand: Field Service Technician posted by Sharon Nurock Recruitment cc
Minimum Essential Job RequirementsQualified Electrician or Technician.N3 or National Diploma Electrical Engineering.Minimum 3 yearsrsquo experience in...
View Job
Field Service Technician
Midrand: SAVOURY FLAVOURIST posted by PAR EXCELLANCE
NDBTech.BSc.Food ScChem.Flavour CreationFoodstuffs, Cosmetics, Disinfectants Act GC MS OperationAnalysis. 5 Five Plus years exp of SAVOURY flavour...
View Job
SAVOURY FLAVOURIST
Midrand: Electrical Sales Engineer – Midrand posted by Network Finance
Are you a qualified electrical professional with a passion for client management and solution selling? A company specialising in electrical protection...
View Job
Electrical Sales Engineer – Midrand
Gauteng: SALES REPRESENTATIVE – FOOD SERVICES (MIDRAND) posted by MPRTC Recruitment
Candidate ProfileMust currently reside in the areaMust have own reliable transport tracker will be installed at company expenseMinimum RequirementsMatric...
View Job
SALES REPRESENTATIVE – FOOD SERVICES (MIDRAND)
Midrand: Key Accounts Representative posted by Network Finance
Job amp Company DescriptionJoin a leading organisation in the electrical sector that prides itself on innovation, customer-centric solutions, and national...
View Job
Key Accounts Representative
South Africa: Head – Tooling & Design posted by Express Employment Professionals Midrand
We are currently on a search for a Head ndash Tooling amp Design to join the largest manufacturer and distributor of major appliances company in Southern...
View Job
Head – Tooling & Design
Centurion: Electrical Technician posted by Express Employment Professionals Midrand
????????????Matric Grade 12Electrical qualification OR proven minimum 4-6 yearsrsquo experience in practical electrical installationsHands-on experience...
View Job
Electrical Technician
Midrand: Senior Mechanical Engineer posted by Network Finance
Job amp Company descriptionJoin a global engineering firm at the forefront of metals and mining innovation. They design world-class technologies that reduce...
View Job
Senior Mechanical Engineer
Midrand: Website and Ecommerce Specialist (Umbraco exp) posted by Top Vitae
You will be responsible for the creation, management, and optimisation of websites and digital platforms across multiple brands. This includes maintaining...
View Job
Website and Ecommerce Specialist (Umbraco exp)
Centurion: Mechanical Service Technician posted by Express Employment Professionals Midrand
????????????Matric Grade 12Minimum 4-6 years of mechanical experience, preferably in trailers, caravans, or related vehiclesKnowledge of wheel bearings,...
View Job
Mechanical Service Technician
Midrand: Maintenance Fitter and Turner posted by Network Finance
Job amp Company descriptionThis leading name in South Africarsquos food manufacturing industry offers stability, scale, and a fast-paced environment. With...
View Job
Maintenance Fitter and Turner
Midrand: Printer Technician posted by Fusion Personnel
Minimum RequirementsMatricValid Driverrsquos License amp Own Vehicle3 ndash 5 yearsrsquo experience as a Printer TechnicianExperience working on the...
View Job
Printer Technician
Midrand: Onsite Software Specialist (Equitrac) posted by Daisy JHB (Pty) Ltd
Job Description Remote SupportAssist clients with workstation and printer issues remotely.SoftwareApplication SupportManaging the printing...
View Job
Onsite Software Specialist (Equitrac)
Gauteng: Industrial Engineer (Special Projects) posted by Express Employment Professionals Midrand
Qualification, knowledge, and skills requirementsBEng Industrial Engineering DegreeOwn Vehicle with Valid Driverrsquos LicenseAdvanced MS Excel and...
View Job
Industrial Engineer (Special Projects)
Midrand: Procurement Manager – Construction Materials posted by Hire Resolve
Our client is looking for a Procurement Manager from the Manufacturing industry to join their team in Midrand. The role involves leading the procurement and...
View Job
Procurement Manager – Construction Materials
Johannesburg: DENTAL OFFICE ADMINISTRATOR/TELEMARKETER – DENTAL EQUIPMENT COMPANY | MIDRAND, JHB posted by MedE Recruit
A new and growing dental equipment company based in Midrand, Johannesburg, is looking for a reliable and driven Office AdministratorTelemarketer to join...
View Job
DENTAL OFFICE ADMINISTRATOR/TELEMARKETER – DENTAL EQUIPMENT COMPANY | MIDRAND, JHB