| Date | 2024-09-08 |
|---|---|
| Job Title | Head Of Data Engineering |
| Employer | Carlysle Human Capital More Information |
| Salary | 0 |
| Category | It Computer |
| Location |
Kwazulu Natal / Durban |
Job Summary
You will help build and maintain scalable data pipelines and related systems in a research focused environment. You will be responsible for designing, developing, testing, and deploying data solutions that meet the business requirements and align with the scientific goals. You will collaborate with research scientists, internal IT and other stakeholders to ensure data quality, reliability, accessibility, security and governance, as follows: Design, develop, and maintain end-to-end technical aspects of all data pipelines required to support the research scientists and data managers Support ETL processes including, data ingestion, transformation, validation, and integration processes using various tools and frameworks Optimize data performance, scalability, and security Provide technical guidance and support to data analysts and research scientists. Design data integrations and data quality frameworks Work and collaborate with the rest of the IT department to help develop the strategy for long term scientific Big Data platform architecture Document and effectively communicate data engineering processes and solutions. Make use of and help define the right cutting edge technology, processes and tools needed to drive technology within our science and research data management departments. Minimum Qualifications: Bachelor`s degree or higher in Computer Science, IT, Engineering, Mathematics, or a related field Industry recognized IT related certification and technology qualification such as Databases and Data related certifications. This is a technical role so a strong focus needs to be on technical skills and experience Minimum Experience: 7+ years experience in a Data Engineering, High Performance Computing, Data Warehousing, Big Data Processing Strong experience with high performance computing environments including Unix, Docker, Kubernetes, Hadoop, Kafka, Nifi or Spark or Cloud-based big data processing environments like Amazon Redshift, Google BigQuery and Azure Synapse Analytics At least 5 years advanced experience and very strong proficiency in UNIX, Linux, Windows Knowledge of various data related programming, scripting or data engineering tools such as Python, R, Julia, T-SQL, PowerShell, etc. Knowledge and Abilities: Strong Experience working with various relational database technologies like MS SQL, MySQL, PostgreSQL as well as NoSQL databases such as MongoDB, Cassandra etc. Experience of Big Data technologies such as Hadoop, Spark and Hive Experience with data pipeline tools such as Airflow, Spark, Kafka, or Dataflow Experience working with containerization is advantageous Experience with data quality and testing tools such as Great Expectations, dbt, or DataGrip is advantageous Experience working with Big Data Cloud based (AWS, Azure etc) technologies is advantageous Experience with data warehouse and data lake technologies such as BigQuery, Redshift, or Snowflake advantageous Strong Experience designing end-to-end data pipelines. Strong knowledge of data modeling, architecture, and governance principles Strong Linux Administration skills Programming skills in various languages advantageous Strong data security and compliancy experience Excellent communication, collaboration, and problem-solving skills Ability to work independently and as part of a cross-functional team Interest and enthusiasm for medical scientific research and its applications. SUMMARY: There is large emphasis on the technical element in the role (having experience working with and designing hardware clusters from the ground up). They have a number of technical skills and experience already within the IT department but are looking for someone very strong, and with the past working experience, to help design these clusters and understand the technology in play for advanced scientific computational requirements. This goes beyond just the hardware element though and you will need strong Linux, Data Pipeline and coding/scripting skills. Although this is not a developer role, you should be comfortable with a certain level of coding/scripting and data analysis packages with strong virtualization skills like VMware, HyperV, OpenStack, KVM etc. You will therefore be that technical link with the scientists and will help build platforms (could be hardware, software, cloud etc) and then also manage the Data and Data Pipelines on this system. This would include the compliancy, performance and security of the data too.
Head Of Data Engineering position available in Kwazulu-Natal, Durban. This job position was posted by Carlysle Human Capital. The job has been posted as a premium ad on 2024-09-08 at 13:47:00 in the It Computer category
Click Go Apply to apply online!
You might also like to look at the available jobs in the same area.
Apply directly on jobmail for this position
Some posts have contact information included with other instructions. All applicants are urged to read the instructions carefully.
We will list jobs at the bottom of each job, so feel free to scroll down and continue your search.
We do not accept any job applications and merely offer the service of daily jobs being mined and displayed on our job portal.
Each job post has a link to take you directly to the original post to apply on their website or the original website where the post came from.
Here are some more related positions
Centurion: Head Of Marketing posted by A-Z Business Professionals
KEY RESPONSIBILITIES:1.Strategic Planning2.Service Provider Management3.Campaign Management4.Brand Management5.Team Leadership 6.Budget Management7.Partnerships and Collaboration8.Digital Marketing Head Of Marketing position available in Gauteng, Centurion
View Job
Head Of Marketing
Pretoria: Head Of Research Department
Together with our client, we are hiring a Head of Research to manage all aspects of the research department. The successful applicant will lead the department’s overall clinical research programme and be provided with the freedom to build the client’s clin
View Job
Head Of Research Department
Johannesburg: Customer Hardships: Head Of Technology
Responsibilities and KRAs: Implementation and maintenance of customer hardships information technology solutions and integration components Updates to the collection strategies, segments, queues, correspondence templates and call strategies in the decision
View Job
Customer Hardships
Port Elizabeth: Head Of Distribution
Our client, a renowned International and national Wealth management Group is URGENTLY seeking to appoint a suitably qualified and experienced Head of Distribution to be based in Cape Town. PLEASE NOTE : A benchmarked CTC Salary Package of R1.2M R1.5M per a
View Job
Head Of Distribution
Johannesburg: Head Of Finance
In this high-impact role, you`ll work closely with the Managing Director, taking the reins of the finance department. From driving precise financial reporting and ensuring ironclad compliance to conducting in-depth financial analysis, your leadership will
View Job
Head Of Finance
South Africa: Group Head Of Internal Audit
The ideal candidate is a strategic thinker with strong assurance experience, currently in a senior leadership role. With a base degree as a minimum requirement and professional membership as a plus, the candidate must be adept at engaging at both Board and
View Job
Group Head Of Internal Audit
Pretoria: Head Of Distribution
Our client, a renowned International and national Wealth management Group is URGENTLY seeking to appoint a suitably qualified and experienced Head of Distribution to be based in Cape Town. PLEASE NOTE : A benchmarked CTC Salary Package of R1.2M R1.5M per a
View Job
Head Of Distribution
Johannesburg: Head Of Distribution
Our client, a renowned International and national Wealth management Group is URGENTLY seeking to appoint a suitably qualified and experienced Head of Distribution to be based in Cape Town. PLEASE NOTE : A benchmarked CTC Salary Package of R1.2M R1.5M per a
View Job
Head Of Distribution
South Africa: Head Of Distribution
Our client, a renowned International and national Wealth management Group is URGENTLY seeking to appoint a suitably qualified and experienced Head of Distribution to be based in Cape Town. PLEASE NOTE : A benchmarked CTC Salary Package of R1.2M R1.5M per a
View Job
Head Of Distribution
Edenvale: Head Of Opreations – Fmcg posted by Leonie Sussens c.c.
Head of Oprations - EdenvaleMechanical/Industrial Engineering degree5 years relevant experience in FMCG Call Leonie Leonie Sussens c.c. Head Of Opreations - Fmcg position available in Gauteng, Edenvale. This job position was posted by Leonie Sussens c.c..
View Job
Head Of Opreations – Fmcg
Johannesburg: Head Of Ppc (Fully Remote)
Who we are: We are a marketing agency with a focus in the healthcare industry working with medical and healthcare providers. What we do: We create medical and healthcare professionals to build and promote their business to new patients. What we are looking
View Job
Head Of Ppc (Fully Remote)
Johannesburg: Head Of Internal Audit
An established financial services company is looking for an experienced individual to head up its internal audit department that has operations spread through other African countries. The prosperous candidate will be required to bring systematic and discip
View Job
Head Of Internal Audit
Johannesburg: Head Of Credit & Risk – Financial Services Group In Sandton.
Head of Credit & Risk Financial Services group in Sandton . Superb opportunity for a dynamic B Com with 10 to 12 years of Collections, Credit and Risk exposure within Retail Banking developing to workflows and scoring models, data analytics, AI and risk mo
View Job
Head Of Credit & Risk – Financial Services Group In Sandton.
Johannesburg: Head Of Tax
A leading workforce solutions company, driven to grow people, businesses and economies through connections that allow them to realize their potential has an exciting opportunity for a Head of Tax in Gauteng. The Head of Tax will be accountable for formulat
View Job
Head Of Tax
Johannesburg: Head Of Supplier Management
SA Citizen Min 5yrs experience in the travel industry specialising in supplier management or service provider management Ability to increase GDS & Net fares Responsible for all trade supplier-related activities Contract analysis Degree in Business or Finan
View Job
Head Of Supplier Management
Johannesburg: Head Of Risk And Compliance (Insurance)
This role is essential in ensuring that the organization adheres to industry regulations, mitigates risks, and fosters a culture of compliance. The ideal candidate needs to have a strong background in the insurance sector and a proven track record in risk
View Job
Head Of Risk And Compliance (Insurance)
Johannesburg: Head Of Web Design And Development (Fully Remote)
Who we are: We are a marketing agency with a focus in the healthcare industry working with medical and healthcare providers. What we do: We create medical and healthcare professionals to build and promote their business to new patients. What we are looking
View Job
Head Of Web Design And Development (Fully Remote)
Johannesburg: Head Of Customer Support
Our client within the FinTech space is seeking a Head of Customer Services to join their team in JHB North on a permanent basis. The Head’s core focus will be to bring exceptional customer satisfaction and enhancing user experience through multiple support
View Job
Head Of Customer Support
Aeroton: Head Of Blood And Transfusion Therapies
Head of Blood and Transfusion Therapies - (POS24186) Area: Aeroton Salary : R950 R1k Key insights to the portfolio. The portfolio consists of 3 main areas, Transfusion, Blood and Haematology. Within Transfusion, we have a few drugs but these are not really
View Job
Head Of Blood And Transfusion Therapies
Johannesburg: Head Of Operations & Sustainability
Key Responsibilities: Manage Budgets: Oversee and control spending to ensure resources are used effectively. Enhance Productivity: Analyse and optimise staffing and production to boost efficiency. Industry Awareness: Stay informed about industry trends and
View Job
Head Of Operations & Sustainability
Johannesburg: Senior Head Of Advisory posted by Construct Executive Search
Applicants must have the following requirements:Must have a Bachelor`s Degree in Finance, Economics, Business, or Development Finance or EngineeringMust have 8-10 years of experience in Management Consulting, and Commercial BankingMust have experience in p
View Job
Senior Head Of Advisory
Other: Head Of Commercial posted by Jobs 4 All
Head Of Commercial TanzaniaEmail CV to Jobs 4 All Head Of Commercial position available in Other, Other. This job position was posted by Jobs 4 All. The job has been posted as a free ad on 2024-08-21 at 17:34:46 in the Building Construction Mining categor
View Job
Head Of Commercial
Johannesburg: Head Of Licensing posted by Jobs 4 All
Email CV to Jobs 4 All Head Of Licensing position available in Gauteng, Johannesburg. This job position was posted by Jobs 4 Al
View Job
Head Of Licensing
Centurion: Head Of Sales – Farm Automotive
A general degree 6 years MIN experience within automotive Farm 5 In a senior leadership capacity A strong dealership network and clear understanding of dealership functioning. Project management certification a bonus Proven track record of sales and revenu
View Job
Head Of Sales – Farm Automotive
Johannesburg: Jr 418731 – Head Of Supplier Relations (Travel Industry Experience Essential)
Employer Description Travel Management company in JHB North Job Description Responsible for negotiating, developing, communicating and managing preferred air, hotel and other (rail, car, etc.) supplier-related programmes within region which will benefit th
View Job
Jr 418731 – Head Of Supplier Relations (Travel Industry Experience Essential)