Cape Town: Intermediate Data Engineer posted by iOCO Digital Pty Lty

About the Role

We are seeking an exceptionally talented hands-on Intermediate Data Engineer who combines deep technical expertise in data pipelines, data modelling, and integration. This role is accountable for designing and delivering scalable, reliable, and high-performance data solutions that support both analytics and application use cases.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF), including pipelines, data flows, triggers, and parameterization.
• Integrate data from APIs, flat files, databases, and cloud/on-prem systems.
• Implement robust ingestion patterns for structured and semi-structured data (JSON, XML, CSV).
• Ensure reliable, efficient, and secure movement of data across systems.

• Design and maintain both normalized (OLTP-aligned) and denormalized (analytical / reporting) data models.
• Apply best practices in dimensional modelling (fact/dimension tables) as well as normalized relational design.
• Implement transformations using SQL (T-SQL), stored procedures, and data flows to prepare analytics-ready datasets.
• Ensure data models are scalable, reusable, and aligned with business requirements.

• Optimize SQL queries, ETL pipelines, and data storage for large datasets (millions+ rows).
• Implement indexing strategies, partitioning, and efficient data access patterns.
• Ensure pipelines are resilient with proper error handling, retry logic, and monitoring.
• Design solutions that minimize impact on transactional systems (clear separation of OLTP and reporting workloads).

• Collaborate closely with backend (.NET) teams to support data access patterns and integration with application services.
• Design and deliver aggregated datasets and data structures optimized for API consumption.
• Support frontend (e.g., Vue.js) data requirements by enabling efficient querying, filtering, and pagination.

• Work closely with BI developers, analysts, and stakeholders to translate data requirements into scalable solutions.
• Continuously improve data platform architecture, tooling, and processes.
• Support CI/CD practices for data pipelines and deployments.
• Stay current with evolving data engineering tools, patterns, and best practices.

Requirements

• 5+ years in data engineering, ETL development, or related roles.
• Azure Data Factory (ADF): Strong hands-on experience with pipeline orchestration, data flows, triggers, parameterization, and monitoring.
• SQL / T-SQL: Advanced querying, performance tuning, indexing strategies, and stored procedure development.
• Data Modelling: Strong experience with both normalized (3NF) and denormalized (star/snowflake) data models.
• Data Platforms: Experience with Azure SQL, Synapse Analytics, or Data Lake architectures.
• ETL / ELT: Strong understanding of data pipeline design, incremental loading, and transformation strategies.
• Performance & Scalability: Experience working with large-scale datasets and high-volume data pipelines.
• Data Governance & Quality: Experience implementing data validation, reconciliation, and quality controls.

Qualifications

Bachelors degree in Computer Science, Engineering, Information Systems, or a related field.