Date | 2024-11-27 |
---|---|
Job Title | Data Engineer |
Employer | Network Finance More Information |
Salary | 0 |
Category | Administration Pa Secretary |
Location |
Gauteng / Johannesburg North |
Job Summary
Responsibilities:
- Process Optimization: Suggest and implement efficiencies to automate manual processes and optimize data delivery for better scalability.
- System Enhancements: Develop and implement new features and enhancements across our data systems, improving the CI/CD process for optimal data pipelines.
- Data Pipeline Development: Assemble and manage complex data sets to meet both functional and non-functional business requirements.
- ETL Processes: Lead the design, development, and optimization of efficient ETL pipelines.
- Testing & Consistency: Develop and execute unit tests to ensure data consistency and reliability across pipelines.
- Monitoring & Reporting: Implement and maintain automated monitoring solutions to support reporting and analytics infrastructure.
- Data Governance & Quality: Ensure high standards of data governance, quality assurance, and the maintenance of data infrastructure systems (AWS, databases, security).
- Metadata & Documentation: Maintain data warehouse and data lake metadata, data catalogs, and user documentation for internal business users.
- Best Practices: Enforce best practices across our database systems (e.g., collations, indices, database engines) to ensure performance and reliability.
Technologies:
- Programming Languages: Python, SQL
- Data Engineering: PySpark, Airflow, Terraform
- Cloud Platforms: AWS (primary), Azure, GCP
- Containerization: Docker
- Version Control & CI/CD: GitHub, Jenkins, etc.
Qualifications and Experience:
- Educational Background: BSc degree in Computer Science, Information Systems, Engineering, or a related technical field (or equivalent work experience).
- Experience: 5+ years of related work experience, including at least 2 years of experience in building and optimizing `big data` data pipelines and maintaining large data sets.
- Technical Proficiency:
- Strong experience with Python and SQL (PostgreSQL, MS SQL).
- Hands-on experience with cloud services, particularly AWS (experience with Azure or GCP is a plus).
- Proficiency in ETL processes and data lifecycle management.
- Experience with Glue and PySpark is highly desirable.
- Skilled in version control, CI/CD pipelines, and GitHub.
- Data Management Expertise: Deep understanding of data quality assurance, data governance, and best practices for managing data transformation processes, metadata, and dependencies.
- Analytical Skills: Strong problem-solving abilities and experience working with unstructured datasets.
- Big Data: Familiarity with message queuing, stream processing, and working with highly scalable `big data` stores.
- Attention to Detail: A meticulous eye for detail and a commitment to maintaining high-quality data systems.
- Communication: Strong communication skills and the ability to work collaboratively in a team.
Apply now!
If you are interested in this opportunity, please apply directly. For more IT jobs, please visit Network Finance
Data Engineer position available in Gauteng, Johannesburg North. This job position was posted by Network Finance. The job has been posted as a char8000 ad on 2024-11-27 at 01:13:10 in the Administration Pa Secretary category
Click Go Apply to apply online!
You might also like to look at the available jobs in the same area.
Apply directly on jobmail for this position
Some posts have contact information included with other instructions. All applicants are urged to read the instructions carefully.
We will list jobs at the bottom of each job, so feel free to scroll down and continue your search.
We do not accept any job applications and merely offer the service of daily jobs being mined and displayed on our job portal.
Each job post has a link to take you directly to the original post to apply on their website or the original website where the post came from.
Here are some more related positions
Johannesburg: Data Analyst/Engineer – Johannesburg – R750k To R950k Per Annum
A group with an inhouse system that runs all their businesses from Life Insurance to Unit Trusts including bringing Impact Farming to Utilities Management. This modern and versatile system is their pride and secret sauce of their success. The Data Analyst/
View Job
Data Analyst/Engineer – Johannesburg – R750k To R950k Per Annum
Pretoria North: Software Engineer – Sql Database Specialist posted by Sastel Packaging
Development and maintenance of inhouse database. Must have qualifications and experience in MySQL, Python and C#. Software Engineer - Sql Database Specialist position available in Gauteng, Pretoria North. This job position was posted by Sastel Packaging. T
View Job
Software Engineer – Sql Database Specialist
Stellenbosch: Data Engineer
Are you passionate about transforming data into actionable insights? Were looking for a skilled Data Engineer to help us build robust data pipelines and drive our data-driven decision-making processes. Key Responsibilities: Design and implement scalable da
View Job
Data Engineer
South Africa: Engineering Data Controller (Permanent)
Brief Role Description To ensure that all activities set out by Supervision are undertaken and completed by the Engineering Data Controller to ensure all data bases are completed on time and to provide product support regarding technical information compan
View Job
Engineering Data Controller (Permanent)
Pretoria: Data Migration Engineer
Developing and implementing comprehensive data migration strategies. Mapping and transforming data from source to target systems. Performing data cleansing and deduplication. Data Migration Engineer position available in Tshwane, Pretoria. This job positio
View Job
Data Migration Engineer
Johannesburg: Data Analyst/Engineer – Johannesburg – R750k To R950k Per Annum
A group with an inhouse system that runs all their businesses from Life Insurance to Unit Trusts including bringing Impact Farming to Utilities Management. This modern and versatile system is their pride and secret sauce of their success. The Data Analyst/
View Job
Data Analyst/Engineer – Johannesburg – R750k To R950k Per Annum
Centurion: Data Science Engineer – Centurion posted by MJM RECRUITMENT
We are hiring:Data science engineer x 1 (centurion)salary: R 30 000 R 35 000pm tctcExperience in the following industies: Vehicle tracking/fleet management/ saas/ big data or telematics highly advantageousbrief description of role:The main purpose of the
View Job
Data Science Engineer – Centurion
Johannesburg: Intermediate Data Engineer – Johannesburg – R400k Up To R600k Per Annum
A long serving bank known as Africa``s biggest lender of assets and financial services group committed to providing banking insurance and asset management services have an open role for an Intermediate Data Engineer to execute data engineering duties accor
View Job
Intermediate Data Engineer – Johannesburg – R400k Up To R600k Per Annum
Cape Town City Centre: Data Engineer
Role: Manager: Data Engineer Minimum Requirements: Bachelors Degree in Computer Science Minimum of 5 years experience in software development Relevant experience in designing and implementing data processing systems In-depth undertaking of database concept
View Job
Data Engineer
Cape Town City Centre: Data Engineer/Analyst – Supply Chain
Degree in IT or Supply Chain/Logistics 4 years in Data Engineering role Ideally experience within a product-led environment, retail, ecommerce or Supply Chain background ideally Data Analysis, Data Gathering Google Big Query Google Data/Looker Automation e
View Job
Data Engineer/Analyst – Supply Chain
Cape Town City Centre: Sap Data Engineer
A well-known insurance company is on the lookout for a SAP Data Engineer to join their innovative team This company drives innovation and excellence in data management and digital transformation. Their team is dedicated to harnessing the power of SAP techn
View Job
Sap Data Engineer
Cape Town City Centre: Senior Data And Vehicle Support Engineer (Telematics) Cape Town
Senior Data and Vehicle Support Engineer (Telematics) Cape Town Minimum requirements and experience: BEng or MEng (Electronic or Mechatronic Engineering) Minimum of 5 years working experience in telematics systems and -components including data managemen
View Job
Senior Data And Vehicle Support Engineer (Telematics) Cape Town
Centurion: Senior Data And Vehicle Support Engineer (Telematics) Centurion
Senior Data and Vehicle Support Engineer (Telematics) Centurion Minimum requirements and experience: BEng or MEng (Electronic or Mechatronic Engineering) Minimum of 5 years working experience in telematics systems and -components including data managemen
View Job
Senior Data And Vehicle Support Engineer (Telematics) Centurion
Cape Town City Centre: Data Engineer
As a Data Engineer at our client, youll play a pivotal role in expanding and optimizing their data architecture and data pipeline systems. Youll work with cutting-edge technology to build efficient data solutions that support cross-functional teams, improv
View Job
Data Engineer
Johannesburg: Data Engineer
We are seeking a skilled and experienced Data Engineer to join our team in Johannesburg. This role involves analyzing, designing, developing, implementing, and maintaining Data/Analytics solutions to meet business requirements, while overseeing the complet
View Job
Data Engineer
Cape Town City Centre: Data Engineer
Are you an experienced Data Engineer with a passion for building robust data pipelines and centralizing data warehouses? Do you thrive in dynamic environments and love working with cutting-edge technologies? If so, we want YOU to join our innovative team K
View Job
Data Engineer
Johannesburg: Ab Initio Data Engineers Etl Developers
PBT Group, leading Data Specialists, has an opening for 2 x experienced Ab Initio ETL Developers in Johannesburg. DUTIES: - Plan and analyse complex business requirements and implement technology enabled solutions to address multi-discipline business oppor
View Job
Ab Initio Data Engineers Etl Developers
Cape Town City Centre: Data Support Engineer
We are seeking a highly skilled Data Support Engineer to join our team in a fully remote capacity. This role requires proficiency in SQL , UNIX , PHP , HTML , and Java , with a focus on ensuring the smooth operation and support of data systems. Key Respons
View Job
Data Support Engineer
Johannesburg: Data Engineer (Johannesburg)
Requirements: Minimum of 3-4 years of experience in data analysis or a similar role Knowledge and Skills: Advanced analytical skills for interpreting and analysing performance data. Python and conducting data science projects is essential. Implementing dat
View Job
Data Engineer (Johannesburg)
Johannesburg: Data Engineer – Johannesburg – R550k To R750k Per Annum
An innovative solar solutions company designed to meet diverse needs has a vacancy for a Data Engineer whose primary role is to oversee all plant data ensuring its accuracy, completeness, and security. This involves devising and executing data collection s
View Job
Data Engineer – Johannesburg – R550k To R750k Per Annum
Cape Town City Centre: Senior Data Engineer
Key Responsibilities: Automate workflows and optimise data delivery. Build and maintain efficient CI/CD processes. Design large, complex data sets for business needs. Maintain data quality, governance, and infrastructure. Tech Stack: Python, PySpark, Palan
View Job
Senior Data Engineer
South Africa: Data Engineer (3Dx) Â?? Marine Sector
Job Title: Data Engineer (3DX) Marine Sector Location: Netherlands Salary: To be discussed in the Interview Vacancy Type: Full-Time Closing date for applications: End Dec 2024 Non- negotiable qualifications and experience required: Education : Bachelors de
View Job
Data Engineer (3Dx) Â?? Marine Sector
Cape Town City Centre: Gcp Data Engineer
Our team is seeking an experienced Google Cloud Platform (GCP) Data Engineer with a passion for building and optimising data pipelines, architectures, and data sets on GCP. This role involves collaborating closely with data analysts, data scientists, and o
View Job
Gcp Data Engineer
Cape Town City Centre: Delivery Lead: Data Engineering
About the role To provide direction and lead a data warehouse team responsible for the full product lifecycle of the software components in the central warehouse, to ensure successful delivery of the required functionality and maintenance related tasks wit
View Job
Delivery Lead
Johannesburg: Data Engineer
What you will be doing The ideal candidate will have extensive experience in data engineering, particularly with Ab Initio, and will be responsible for designing, developing, and maintaining the data infrastructure. Design, build, and maintain scalable dat
View Job
Data Engineer
Email Alerts
Click the button below and submit your email to receive email alerts with the latest jobs.
This service is offered by Result Generation
We do not save your information, and your email is the only information we forward to the service.
If you wish to unsubscribe, please click here