Hire Resolve’s client is seeking a highly skilled Senior Data Engineer to join their dynamic team within the telecommunications domain. The ideal candidate will be responsible for designing, developing, and maintaining scalable and reliable data pipelines to support the ingestion, processing, and analysis of Call Detail Records (CDRs) and real-time data streams.

This role demands advanced technical expertise in stream processing frameworks such as Apache Kafka, Apache NiFi, Apache Flink, and Apache Druid, along with strong collaboration skills and a passion for data-driven solutions.

Responsibilities: 

• Design and implement robust ETL/ELT pipelines for ingesting and transforming CDRs and telecom log data across multiple network types (2G/3G/4G/5G).
• Develop and maintain real-time data pipelines using Apache Kafka, NiFi, and Flink.
• Structure and integrate data for analytics platforms such as Apache Druid, Hudi, and Superset.
• Manage large-scale distributed file systems and ensure efficient batch/stream data processing.
• Build and manage CI/CD pipelines for data workflow deployments.
• Optimize data performance, cost-efficiency, and reliability through continuous monitoring and enhancement.
• Enforce data governance best practices including schema validation, access control, and regulatory compliance (e.g. GDPR).
• Collaborate with development, DevOps, and analytics teams to deliver query-ready datasets for machine learning models and BI dashboards.
• Implement role-based access control, data encryption, and secure identity management.
• Design and maintain metadata repositories, data lineage, and asset discovery frameworks.
• Build quality assurance and anomaly detection tools to monitor data integrity across all pipeline stages.
• Support analytical modelling using OLAP systems and columnar databases.

 Requirements: 

• Bachelors Degree in Computer Science, Software Engineering, Computer Engineering, or related field.
• Minimum of 58 years of experience in data engineering, preferably in telecom or large-scale distributed systems.
• Strong expertise in Apache stack tools: Kafka, NiFi, Flink, Spark, Hive, Druid, Hudi.
• Proficiency in Python, SQL, Shell scripting, and Linux-based systems.
• Experience in database management: PostgreSQL, MySQL, and analytical querying.
• Familiarity with data serialization formats (JSON, Avro, etc.) and secure data transfer protocols (SFTP).
• In-depth knowledge of data quality control, lineage tracking, and data discovery tools.
• Experience in full-text search and indexing of structured/unstructured data using Apache or similar tools.
• Excellent problem-solving abilities, multitasking, and a fast-learning attitude.

How To Apply:

• Contact Hire Resolve today for your next career-changing move
• Our client is offering a highly competitive salary for this role based on experience.
• Send your CV to: *****@*****.co.za or connect with Mischa Bornman via LinkedIn.
• Alternatively, you can also contact me directly at Hire Resolve *****@*****.co.za 

We will contact you telephonically in 3 days should you be suitable for this vacancy. If you are not suitable, we will put your CV on file and contact you regarding any future vacancies that arise.

Click Go Apply to apply online!

---

More related positions

---

Hire Resolve’s client is seeking a talented individual to join their dynamic team in Gauteng. You’ll thrive in a remote-first work culture that prioritizes innovation and collaboration, with monthly in-person get-togethers to foster team bonding and idea sharing.

Responsibilities: 

• Stay Current with SQL: Continuously learn and apply the latest SQL features and programming practices through education, study, and participation in conferences, workshops, and groups.
• Data Management: Oversee data backups from local directories to shared drives and VSTS.
• Issue Resolution: Identify and resolve problems stemming from customer feedback and testing, correcting them or escalating to appropriate personnel.
• Code Quality Assurance: Evaluate code to ensure it meets industry standards, is valid, properly structured, and compatible with various Azure SQL versions.
• Requirements Analysis: Analyze technical requirements to accurately determine user needs.
• System Integration: Manage data analysis and the integration of disparate systems.
• Performance Optimization: Guarantee high quality and optimal performance of delivered code, adhering to internal guidelines.
• Database Expertise: Apply deep understanding of database operations and optimization for Microsoft SQL Server.
• Reporting Development: Develop effective reporting systems that provide accessible information for decision-making.
• Troubleshooting: Conduct thorough troubleshooting on code, tables, indexes, and SQL Server instances.
• Report Generation: Generate and deliver quality reports to customers, performing comprehensive quality assurance checks.
• Cross-functional Collaboration: Partner with various teams to translate business requirements into scalable data solutions.
• Design Participation: Contribute to design reviews, providing valuable input on requirements, product design, and potential issues.
• Software Validation: Test and validate software against major architectural requirements, with a focus on scalability and performance.
• System Quality: Ensure the overall quality of the entire system.
• Technology Advancement: Stay up-to-date with the latest Microsoft data technologies, Azure services, and data engineering best practices.
• Performance Enhancement: Troubleshoot and optimize Azure data pipelines and SQL Server performance, and resolve data quality issues.
• Cost Optimization: Continuously improve data infrastructure to reduce latency, increase throughput, and optimize Azure costs using FinOps principles.
• Tool Integration: Evaluate and integrate new Microsoft data tools and Azure services to enhance our data platform capabilities.
• Mentorship: Mentor and guide junior data engineers, fostering their growth.
• Architectural Oversight: Participate in architecture reviews, ensuring all data solutions meet high-quality standards and best practices.
• Stakeholder Communication: Communicate effectively with stakeholders to understand data requirements and provide technical solutions.
• Independent Delivery: Work autonomously and deliver high-quality work within tight deadlines.

Minimum Requirements: 

• Data Processing: 7+ years of T-SQL/SQL Server expertise, deep Azure Synapse Analytics skills, and C#/.NET for data processing applications.
• Microsoft Data Stack: Strong proficiency in Azure Data Factory, Azure Databricks, SQL Server Integration Services (SSIS), Azure Stream Analytics, and Event Hubs.
• Cloud & Infrastructure: Experience with Azure Synapse Analytics, Power Designer for data modeling, and Azure DevOps for CI/CD.
• Data Modeling: Expertise in dimensional modeling with Power Designer, SQL Server Data Tools (SSDT), schema design for OLAP/OLTP, and performance optimization.
• Business Intelligence: Experience with Power BI integration, SSAS/SSRS, and knowledge of DAX/MDX and semantic layer design.
• DevOps & Orchestration: Proficient with Azure DevOps pipelines, ARM templates/Bicep, Azure Monitor, and Application Insights.
• Testing: Experience with unit testing for T-SQL and data pipelines, data quality frameworks, and integration testing with Azure Data Factory.
• Strong understanding of basic DB concepts, joins, aggregate functions, and commands (DDL, DML, TCL DCL).
• Proficiency in advanced SQL concepts like CTE, Recursive queries, Pivots, Self joins, Temporary functions, UDF, etc.
• Deep understanding of relational database concepts with solid knowledge of MS-SQL.
• Proficiency with version control systems such as Git and Azure Repos.
• Strong problem-solving and troubleshooting skills in Microsoft environments.
• Experience with ETL tools.
• Expertise in SQL Server performance tuning and optimization of large-scale data systems.

Advantageous Skills

• Experience with Azure Machine Learning and Azure Cognitive Services for ML pipelines.
• Experience with Microsoft Fabric and its unified analytics platform.
• Real-time analytics with Azure Stream Analytics and Complex Event Processing.
• Azure Cost Management and FinOps experience optimizing data workloads.
• Power Platform integration (Power Apps, Power Automate) for data workflows.
• Experience with Cosmos DB for multi-model database scenarios.
• Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect certification.

Benefits:

• Competitive salary based on experience.

IF you meet the above requirements and want to make a career-changing move, apply today by emailing your CV to *****@*****.co.za

You can also connect with Daniel on LinkedIn or contact Hire Resolve
If you meet the above requirements we will contact you within 3 working days

Click Go Apply to apply online!

---

More related positions

---