Job Description
Responsibilities:
-
Spark Optimization: Act as the internal SME for Spark internals; manage memory, shuffle tuning, and partitioning for cost-effective performance.
-
Cloud-Agnostic Development: Build pipelines using Python and Delta Lake, decoupling code from specific cloud providers and reducing reliance on GUI tools (e.g., ADF).
-
Refactoring & Modernization: Migrate complex SQL-based ETL into modular, testable, and maintainable Python libraries.
-
Lakehouse Engineering: Manage Medallion Architecture (Bronze/Silver/Gold) using Delta Lake, ensuring storage performance via Z-Ordering and Vacuuming.
-
Code-First Orchestration: Support the transition to code-centric patterns (Airflow, Dagster) to prioritize portability.
-
Technical Excellence: Lead code reviews, mentor junior engineers, and implement automated testing frameworks (Pytest).
Minimum Requirements:
- Education: Bachelors degree in Computer Science, Information Systems, Engineering, or a related field.
-
Spark Mastery: 6+ years of Spark/PySpark experience; expert ability to diagnose bottlenecks via Spark UI and optimize complex DAGs.
-
Advanced Python: Proficiency in production-grade Python, including building reusable libraries and automated testing.
-
Azure Ecosystem: Strong experience with Azure Synapse, Dedicated SQL Pools, and Data Factory.
-
Modern Data Stack: Hands-on experience with Delta Lake, Parquet, and containerization (Docker).
-
Migration Skills: Solid T-SQL skills to interpret and migrate legacy logic into Python-centric environments.
-
Security & Governance: Proven ability to implement high levels of security and compliance across data processes.
Benefits:
- Competitive salary based on experience (salary can potentially be more based on experience/skills)
IF you meet the above requirements and want to make a career-changing move, apply today by emailing your CV to .za
How to Apply
Click the green “Go Apply” button below to apply directly online with the employer.
About Other IT/Computer Jobs in Gauteng
The IT and computer industry is one of the most dynamic and growing sectors in Gauteng, with a thriving job market that offers a wide range of opportunities for career advancement.
In terms of salary ranges, IT professionals in Gauteng can expect to earn anywhere from R800 000 per annum for junior positions to over R2 million for senior executives. The average salary for software engineers is around R1.5 million, while data analysts can earn between R900 000 and R1.2 million. With experience and skills, IT professionals in Gauteng can expect significant salary increases, with some roles offering bonuses and benefits that can increase their overall remuneration.
To succeed in the IT industry in Gauteng, candidates need to possess a range of key skills, including programming languages such as Java, Python, and C++, as well as experience with databases, cloud computing, and cybersecurity. Strong problem-solving skills, attention to detail, and excellent communication skills are also essential for success in this field. Additionally, knowledge of ITIL (Information Technology Infrastructure Library) and Agile methodologies can be highly valued by employers.
Several major companies and industries in Gauteng are currently hiring IT professionals, including technology firms such as Microsoft, IBM, and Oracle, as well as banks and financial institutions like Standard Bank and Nedbank. The finance industry is also a significant employer of IT staff, with roles ranging from software development to data analytics and cybersecurity.
Career growth opportunities are abundant in the IT industry in Gauteng, with many companies investing heavily in employee training and development programs. With experience and skills, IT professionals can move into senior leadership positions or start their own businesses, while also pursuing further education and certifications. The Johannesburg Stock Exchange (JSE) is also a significant employer of IT staff, with roles ranging from software development to data analytics and cybersecurity.
Overall, the IT industry in Gauteng offers a wide range of exciting career opportunities for job seekers with the right skills and experience. With salaries that are competitive with those offered by major companies globally, there has never been a better time to pursue a career in this dynamic and rapidly evolving field.
This information provides general career guidance. Actual salaries and requirements vary by employer.
GO APPLY NOW
Safe & secure application process
Explore More Opportunities
Get Job Alerts
Never miss a job! Get daily alerts matching your preferences.
📧 Why Subscribe?
- Get notified instantly when new jobs match your criteria
- No registration required - just your email
- Daily digest of relevant opportunities
- Free forever - never miss your dream job
- Unsubscribe anytime with one click
Your email is safe with us. We only send job alerts and never spam. Unsubscribe anytime with one click. No registration, no account required.
Jobseeker Tip
Apply early, keep your CV ready, and search using both job title and location. Example: Admin Johannesburg.
Interview tips
How to Apply
Click “GO APPLY NOW” to visit the company’s application page.
Follow their instructions carefully.
JVR Jobs connects you with employers – we don’t process applications directly.
Latest Job Opportunities
Johannesburg: ERP Application Consultant – Finance posted by Hire Resolve
Posted June 10, 2026
View jobCape Town: Sales & Marketing Consultant posted by Hire Resolve
Posted June 10, 2026
View jobOudtshoorn: Accountant posted by GAP Consulting
Posted June 10, 2026
View jobWestern Cape: Senior Accountant posted by Persona Staff
Posted June 10, 2026
View jobBellville: Senior Wealth Manager posted by Hire Resolve
Posted June 10, 2026
View jobStellenbosch: Senior Data Engineer posted by Hire Resolve
Posted June 10, 2026
View job
Browse Employers
Job Alerts