Date | 2024-10-24 |
---|---|
Job Title | Senior Dataops Engineer |
Employer | More Information |
Salary | |
Category | It Computing Software |
Location |
Cape Town Region / Cape Town City Centre |
Job Summary
We are looking for a Senior DataOps Engineer to join our team and lead the development of DataOps practices within our organisation. As this is a relatively new area for us, we need someone with a strong DevOps background combined with software and data experience. The ideal candidate will be responsible for setting the direction for DataOps within the team, building scalable data pipelines, and ensuring efficient data operations in a fully cloud-based environment (AWS). This role is ideal for a senior software professional with a deep understanding of DevOps practices and a solid foundation in data. Key Responsibilities Lead DataOps Strategy : Define and implement DataOps practices, guiding the team towards a more automated and efficient data pipeline management system. Collaborate with data engineers, data scientists, and other stakeholders to establish best practices for data workflows and infrastructure. Build & Maintain Data Pipelines : Design, deploy, and manage scalable and reliable data pipelines to enable efficient data flow and processing across cloud environments. Automate and optimise ETL/ELT processes, ensuring seamless data integration from various sources. DevOps for Data : Apply DevOps principles to data workflows, focusing on continuous integration, continuous delivery (CI/CD), and infrastructure-as-code (IaC) for data operations. Implement version control and automation for data-related processes, reducing errors and improving data quality. Cloud Infrastructure Management (AWS) : Manage and optimise AWS cloud infrastructure to support data workflows, including S3, Redshift, Lambda, and RDS. Monitor, maintain, and scale AWS resources for efficient data storage, processing, and analysis. Data Security & Compliance : Ensure data security and compliance with relevant industry standards and regulations, implementing appropriate security protocols and monitoring. Collaborate with security teams to ensure secure handling of sensitive data and manage access controls effectively. Monitoring & Optimisation : Set up monitoring, logging, and alerting mechanisms for data pipelines to ensure high availability and performance. Identify bottlenecks and inefficiencies in data processes, proposing and implementing optimisations. Collaboration & Mentorship : Provide technical leadership and mentorship to junior team members, helping them to grow in their roles and expand their knowledge of DataOps practices. Work closely with data engineering, software development, and IT teams to drive cross-functional initiatives. Qualifications & Experience Proven experience as a DevOps Engineer, with significant exposure to data-focused environments. Strong understanding of data concepts, including data pipelines, ETL/ELT processes, and data storage solutions. Expertise in building, deploying, and managing data pipelines in cloud environments, especially AWS. Familiarity with AWS services such as S3, Redshift, Lambda, RDS, and other relevant cloud-based data tools. Experience with CI/CD pipelines, infrastructure-as-code, and automation tools (e.g., Jenkins, Terraform, Ansible). Strong knowledge of version control systems (e.g., Git) and containerisation technologies (e.g., Docker, Kubernetes). Preferred Skills Experience with DataOps principles and practices, including data versioning, pipeline automation, and data observability. Strong programming skills in languages such as Python, Java, or Scala, with experience in building data processing solutions. Familiarity with data processing frameworks like Apache Spark, Kafka, or similar. Exposure to security best practices for data storage and processing in cloud environments. Technical Skills DevOps Tools : Strong understanding of CI/CD tools, infrastructure automation, and version control. Cloud Expertise : Deep familiarity with AWS cloud infrastructure, including automation and orchestration of services. Data Management : Knowledge of modern data architectures, pipelines, ETL/ELT processes, and relevant technologies. Scripting & Automation : Proficiency in automating tasks using scripting languages (Python, Bash) and DevOps tools.
Senior Dataops Engineer position available in Cape Town Region, Cape Town City Centre. This job position was posted by . The job has been posted on 2024-10-24 in the It Computing Software category
Click Go Apply to apply online!
You might also like to look at the available jobs in the same area.
Apply directly for this position
Some posts have contact information included with other instructions. All applicants are urged to read the instructions carefully.
We will list jobs at the bottom of each job, so feel free to scroll down and continue your search.
We do not accept any job applications and merely offer the service of daily jobs being mined and displayed on our job portal.
Each job post has a link to take you directly to the original post to apply on their website or the original website where the post came from.
Here are some more related positions
Cape Town: Senior Dataops Engineer posted by PBT Group
We are looking for a Senior DataOps Engineer to join our team and lead the development of DataOps practices within our organisation. As this is a relatively new area for us, we need someone with a strong DevOps background combined with software and data ex
View Job
Senior Dataops Engineer
Cape Town City Centre: Senior Dataops Engineer
We are looking for a Senior DataOps Engineer to join our team and lead the development of DataOps practices within our organisation. As this is a relatively new area for us, we need someone with a strong DevOps background combined with software and data ex
View Job
Senior Dataops Engineer
Email Alerts
Click the button below and submit your email to receive email alerts with the latest jobs.
This service is offered by Result Generation
We do not save your information, and your email is the only information we forward to the service.
If you wish to unsubscribe, please click here