Cape Town: Senior Dataops Engineer posted by PBT Group

Date	2024-10-23
Job Title	Senior Dataops Engineer
Employer	PBT Group More Information
Salary	0
Category	Finance Accounting
Location	Western Cape / Cape Town

Job Summary

We are looking for a Senior DataOps Engineer to join our team and lead the development of DataOps practices within our organisation. As this is a relatively new area for us, we need someone with a strong DevOps background combined with software and data experience. The ideal candidate will be responsible for setting the direction for DataOps within the team, building scalable data pipelines, and ensuring efficient data operations in a fully cloud-based environment (AWS). This role is ideal for a senior software professional with a deep understanding of DevOps practices and a solid foundation in data.

Key Responsibilities

Lead DataOps Strategy:
- Define and implement DataOps practices, guiding the team towards a more automated and efficient data pipeline management system.
- Collaborate with data engineers, data scientists, and other stakeholders to establish best practices for data workflows and infrastructure.
Build & Maintain Data Pipelines:
- Design, deploy, and manage scalable and reliable data pipelines to enable efficient data flow and processing across cloud environments.
- Automate and optimise ETL/ELT processes, ensuring seamless data integration from various sources.
DevOps for Data:
- Apply DevOps principles to data workflows, focusing on continuous integration, continuous delivery (CI/CD), and infrastructure-as-code (IaC) for data operations.
- Implement version control and automation for data-related processes, reducing errors and improving data quality.
Cloud Infrastructure Management (AWS):
- Manage and optimise AWS cloud infrastructure to support data workflows, including S3, Redshift, Lambda, and RDS.
- Monitor, maintain, and scale AWS resources for efficient data storage, processing, and analysis.
Data Security & Compliance:
- Ensure data security and compliance with relevant industry standards and regulations, implementing appropriate security protocols and monitoring.
- Collaborate with security teams to ensure secure handling of sensitive data and manage access controls effectively.
Monitoring & Optimisation:
- Set up monitoring, logging, and alerting mechanisms for data pipelines to ensure high availability and performance.
- Identify bottlenecks and inefficiencies in data processes, proposing and implementing optimisations.
Collaboration & Mentorship:
- Provide technical leadership and mentorship to junior team members, helping them to grow in their roles and expand their knowledge of DataOps practices.
- Work closely with data engineering, software development, and IT teams to drive cross-functional initiatives.

View Job Cape Town: Receptionist posted by Isilumko Staffing

Qualifications & Experience

Proven experience as a DevOps Engineer, with significant exposure to data-focused environments.
Strong understanding of data concepts, including data pipelines, ETL/ELT processes, and data storage solutions.
Expertise in building, deploying, and managing data pipelines in cloud environments, especially AWS.
Familiarity with AWS services such as S3, Redshift, Lambda, RDS, and other relevant cloud-based data tools.
Experience with CI/CD pipelines, infrastructure-as-code, and automation tools (e.g., Jenkins, Terraform, Ansible).
Strong knowledge of version control systems (e.g., Git) and containerisation technologies (e.g., Docker, Kubernetes).

Preferred Skills

Experience with DataOps principles and practices, including data versioning, pipeline automation, and data observability.
Strong programming skills in languages such as Python, Java, or Scala, with experience in building data processing solutions.
Familiarity with data processing frameworks like Apache Spark, Kafka, or similar.
Exposure to security best practices for data storage and processing in cloud environments.

Technical Skills

DevOps Tools: Strong understanding of CI/CD tools, infrastructure automation, and version control.
Cloud Expertise: Deep familiarity with AWS cloud infrastructure, including automation and orchestration of services.
Data Management: Knowledge of modern data architectures, pipelines, ETL/ELT processes, and relevant technologies.
Scripting & Automation: Proficiency in automating tasks using scripting languages (Python, Bash) and DevOps tools.

Senior Dataops Engineer position available in Western Cape, Cape Town. This job position was posted by PBT Group. The job has been posted as a premium ad on 2024-10-23 at 13:00:55 in the Finance Accounting category

Click Go Apply to apply online!

You might also like to look at the available jobs in the same area.

View Job Cape Town: Snr Sales Ops Consultant - Travel Industry posted by ABC Worldwide

Select one of the buttons below to browse for more available jobs.

Apply directly on jobmail for this position
Some posts have contact information included with other instructions. All applicants are urged to read the instructions carefully.
We will list jobs at the bottom of each job, so feel free to scroll down and continue your search.

We do not accept any job applications and merely offer the service of daily jobs being mined and displayed on our job portal.
Each job post has a link to take you directly to the original post to apply on their website or the original website where the post came from.

Here are some more related positions