Job Description
Responsibilities:
-
Spark Optimization: Act as the internal SME for Spark internals; manage memory, shuffle tuning, and partitioning for cost-effective performance.
-
Cloud-Agnostic Development: Build pipelines using Python and Delta Lake, decoupling code from specific cloud providers and reducing reliance on GUI tools (e.g., ADF).
-
Refactoring & Modernization: Migrate complex SQL-based ETL into modular, testable, and maintainable Python libraries.
-
Lakehouse Engineering: Manage Medallion Architecture (Bronze/Silver/Gold) using Delta Lake, ensuring storage performance via Z-Ordering and Vacuuming.
-
Code-First Orchestration: Support the transition to code-centric patterns (Airflow, Dagster) to prioritize portability.
-
Technical Excellence: Lead code reviews, mentor junior engineers, and implement automated testing frameworks (Pytest).
Minimum Requirements:
- Education: Bachelors degree in Computer Science, Information Systems, Engineering, or a related field.
-
Spark Mastery: 6+ years of Spark/PySpark experience; expert ability to diagnose bottlenecks via Spark UI and optimize complex DAGs.
-
Advanced Python: Proficiency in production-grade Python, including building reusable libraries and automated testing.
-
Azure Ecosystem: Strong experience with Azure Synapse, Dedicated SQL Pools, and Data Factory.
-
Modern Data Stack: Hands-on experience with Delta Lake, Parquet, and containerization (Docker).
-
Migration Skills: Solid T-SQL skills to interpret and migrate legacy logic into Python-centric environments.
-
Security & Governance: Proven ability to implement high levels of security and compliance across data processes.
Benefits:
- Competitive salary based on experience (salary can potentially be more based on experience/skills)
IF you meet the above requirements and want to make a career-changing move, apply today by emailing your CV to .za
How to Apply
Click the green “Go Apply” button below to apply directly online with the employer.
About Other IT/Computer Jobs in Gauteng
The IT and computer industry is one of the most dynamic and growing sectors in Gauteng, with a thriving job market that offers a wide range of opportunities for career advancement.
In terms of salary ranges, IT professionals in Gauteng can expect to earn anywhere from R800 000 per annum for junior positions to over R2 million for senior executives. The average salary for software engineers is around R1.5 million, while data analysts can earn between R900 000 and R1.2 million. With experience and skills, IT professionals in Gauteng can expect significant salary increases, with some roles offering bonuses and benefits that can increase their overall remuneration.
To succeed in the IT industry in Gauteng, candidates need to possess a range of key skills, including programming languages such as Java, Python, and C++, as well as experience with databases, cloud computing, and cybersecurity. Strong problem-solving skills, attention to detail, and excellent communication skills are also essential for success in this field. Additionally, knowledge of ITIL (Information Technology Infrastructure Library) and Agile methodologies can be highly valued by employers.
Several major companies and industries in Gauteng are currently hiring IT professionals, including technology firms such as Microsoft, IBM, and Oracle, as well as banks and financial institutions like Standard Bank and Nedbank. The finance industry is also a significant employer of IT staff, with roles ranging from software development to data analytics and cybersecurity.
Career growth opportunities are abundant in the IT industry in Gauteng, with many companies investing heavily in employee training and development programs. With experience and skills, IT professionals can move into senior leadership positions or start their own businesses, while also pursuing further education and certifications. The Johannesburg Stock Exchange (JSE) is also a significant employer of IT staff, with roles ranging from software development to data analytics and cybersecurity.
Overall, the IT industry in Gauteng offers a wide range of exciting career opportunities for job seekers with the right skills and experience. With salaries that are competitive with those offered by major companies globally, there has never been a better time to pursue a career in this dynamic and rapidly evolving field.
This information provides general career guidance. Actual salaries and requirements vary by employer.
GO APPLY NOW
Safe & secure application process
Explore More Opportunities
Get Similar Job Alerts
Job Seeker Tip
Network actively - many jobs are filled through referrals before being advertised.
How to Apply
Click “GO APPLY NOW” to visit the company’s application page.
Follow their instructions carefully.
JVR Jobs connects you with employers – we don’t process applications directly.
Latest Job Opportunities
Western Cape: Junior Electrician Ref: 4192 posted by NDC Personnel
In this role, you will be responsible for the installation, maintenance, and repair of electrical systems and equipment, ensuring reliable…
View JobWestern Cape: Cost Estimator Ref: 4178 posted by NDC Personnel
In this role, you will manage daily costing and tender enquiries within the manufacturing and construction sector. You will also…
View JobWestern Cape: Safety Officer Ref: 4177 posted by NDC Personnel
In this role, you will play a key part in maintaining a safe, healthy, and compliant working environment. You will…
View JobGauteng: Accounts Administrator posted by Keesta HR Solutions (Pty) Ltd
Job Title: Accounts AdministratorSeniority Level: Mid Career (4 - 6 yrs exp)Location: Pretoria (Tshwane)Type: PermanentDuties and Responsibilities:Debtors...
View JobCape Town: Mechanical Supervisor Ref: 4187 posted by NDC Personnel
In this role, you will be instrumental in ensuring that all equipment, machinery, buildings, and facilities are maintained in optimal…
View JobRustenburg: Logistics Manager posted by Skillnet Recruitment
Logistics Manager - Rustenburg NWReport to: Director Seniority Level: Mid Career (Above 6 yrs exp) Type: Permanent Duties and Responsibilities:Manage…
View Job
Browse Employers
Job Alerts