Menu Close

Johannesburg: Senior Data Engineer – JHB posted by Hire Resolve

Advertisement



Job Description

Hire Resolve’s client is looking for a Senior Data Engineer to join their team in Johannesburg, GP. The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines to support telecommunications CDR processing, realtime data ingestion, and analytical workloads.

This role requires expertise in data modeling, ETL development, stream processing, and distributed data systems. The ideal candidate will work closely with developers, DevOps, and analytics teams to transform raw network data into clean, structured, and queryready datasets that power dashboards, machine learning models, and business logic. The candidate will provide technical leadership, optimize data workflows for performance and reliability, and drive best practices in data engineering methodologies

Responsibilities: 

  • Build ETL/ELT pipelines for ingesting, cleansing, and transforming CDRs and telecommunications logs
    from multiple network elements (5G/4G/3G/2G).
  • Design and maintain real-time data flows using Kafka, Apache NiFi and Apache Flink.
  • Work with large-scale distributed file systems for batch and streaming ingestion.
  • Integrate and structure data for analytics platform like Apache Druid, Hudi and Superset.
  • Develop CI/CD pipelines for deploying data workflows and transformation logic.
  • Ensure data quality, schema validation and compliance with retention and security policies.
  • Monitor data pipeline health and optimize performance throughput and cost efficiency.
  • Write complex and performant queries for data validation, transformation, aggregation, and analytics
    across relational and distributed platforms.
  • Develop and optimize big data processing workflows in platforms like Apache Spark, Hive and Druid.
  • Establish efficient issue tracking and workflow processes, enhancing productivity and collaboration
    across engineering teams.
  • Implement security best practices and compliance frameworks to safeguard infrastructure, data, and
    applications from vulnerabilities and threats.
  • Maintain secure role-based access control mechanisms, encryption strategies, and identity management
    solutions to protect sensitive data and ensure regulatory compliance.
  • Mapping data flows from source to transformation to consumption.
  • Design and implement full-text search and indexing solutions for querying and retrieval of structured and unstructured telecommunications data using Apache tools or similar search engines.
  • Analyze and estimate and implement storage requirements and strategies for large-scale CDR datasets
    and real-time data streams, ensuring optimal resource allocation and scalability across environments.
  • Ensure data integrity and consistency across ingestion, transformation and storage layers through
    validation checks, schema enforcement and robust error handling mechanisms.
  • Develop and maintain quality monitoring tools to proactively detect anomalies, missing records or data
    corruption across pipelines.
  • Perform other duties as assigned
View Job  Cape Town: Online Marketing Manager posted by Hire Resolve

Requirements: 

  • Bachelors degree in Computer Engineering, Software Engineering, Computer Science, or a related field.
  • Strong experience in building data pipelines using tools like Apache NiFi, Kafka, Airflow or similar.
  • Proficiency in SQL, Python, database administration and management like PostgreSQL, MySQL.
  • Solid understanding of distributed data systems like Hive and Hudi and Spark.
  • Experience with streaming frameworks like Kafka Streams, Apache Flink and Apache Beam.
  • Familiarity with data serialization formats like JSON.
  • Knowledge of SFTP and secure data transfer mechanisms for ingesting remote files.
  • Proficient with Linux environments, shell scripting and storage systems like Ceph.
  • Experience with data governance, including data privacy and regulatory compliance like GDPR and
    implementing access control, auditing and data usage policies.
  • Experience in maintaining central inventory of data assets, managing metadata and enabling searchable
    discovery across structured and unstructured datasets.
  • Experience in data lineage tracking to map data flows, visualize and track dependencies.
  • Experience with OLAP systems, analytical modelling and columnar databases and designing and
    querying multidimensional cubes.
  • Strong problemsolving skills, ability to work in a fastpaced environment, and manage multiple projects
    efficiently.
  • Strong collaboration skills, adaptability, and a commitment to continuous learning 

How to Apply: 

If you would like to apply for this role, kindly forward your CV to Gaby Turner at *****@*****.co.za or you may forward your CV to *****@*****.co.za 



GO APPLY NOW

Safe & secure application process

Advertisement



Advertisement



View Job  East Rand: Technically Skilled IT Field Technician (East Rand, Johannesburg) posted by AtripleA recruitment & temps


Get Similar Job Alerts
Enter your WhatsApp number (e.g., +27740908132)
You'll receive alerts for new jobs in Johannesburg, Gauteng


Job Seeker Tip

Follow up your job application with a polite email if you haven't heard back within a week.

How to Apply

Click “GO APPLY NOW” to visit the company’s application page.
Follow their instructions carefully.

JVR Jobs connects you with employers – we don’t process applications directly.

Latest Job Opportunities

Gauteng: Graphic Designer posted by Energy at Work Projects

Jobs in Gauteng October 8, 2025
Automotive Technician Energy at Work Projects

What You’ll Do:Design compelling digital experiences across web and mobile platformsCreate intuitive, user-centered UI designs and iterate based on user...

View Job

South Africa: FMCG Machine Technician (Electrician/Millwright – Bakeries) – Cape Town posted by MECS Africa

Jobs in South Africa October 8, 2025
Engineering Management MECS Africa

Specialised FMCG Maintenance and Installations experienceMust be able to read and understand electrical diagrams.Be able to do electrical fault finding.Be...

View Job  South Africa: Anti-Poaching Unit Manager - Luxury Game Reserve - Namibia | Lb posted by HotelJobs
View Job

Delmas: Junior Accountant posted by GAP Consulting

Jobs in Delmas October 8, 2025
Agriculture Aquaculturist Fisheries GAP Consulting

Minimum Requirements:Must have a minimum of 2 years experience as a Junior Accountant within a Finance DepartmentBCom Management Accounting |…

View Job

Middle East: Mining Operations Specialist posted by MECS Africa

Jobs in Middle East October 8, 2025
Health and Env MECS Africa Safety

Duties and Responsibilities:Conduct geotechnical inspections and stability assessments of the residue storage area.Analyze instrumentation data and...

View Job

Centurion: Payroll and Office Administrator posted by Creative Leadership Solutions

Jobs in Centurion October 8, 2025
Creative Leadership Solutions Human Resource

REQUIREMENTSMinimum education (essential):National Senior CertificateMinimum applicable experience (years):5+ years in payroll administration.Minimum...

View Job

Western Cape: Cost and Management Accountant posted by Techbridge Recruitment

Jobs in South Africa October 8, 2025
Manufacturing Production Techbridge Recruitment

Key Responsibilities:Perform costing functions including volume, price, and cost analysisConduct ongoing analysis of process constraints, margin analysis,...

View Job

Browse Employers

View All

Job Alerts


Share this to someone who needs a job:
Posted in Jobs in Central Johannesburg, Jobs in Gauteng, Jobs in Johannesburg, Jobs in South Africa

More Jobs in Your Area