| Date | 2024-11-24 |
|---|---|
| Job Title | Head Of Data Engineering |
| Employer | Carlysle Human Capital More Information |
| Salary | 0 |
| Category | It Computer |
| Location |
Kwazulu Natal / Durban |
Job Summary
This role is for a Senior Data Engineer to help build and maintain scalable data pipelines and related systems in a research focused environment. You will be responsible for designing, developing, testing, and deploying technical platforms and data solutions that meet the business requirements and align with the scientific goals. You will collaborate with research scientists, internal IT and other stakeholders to ensure data quality, reliability, accessibility, security and governance. This is technical role (not just a DBA or Data management role) that needs a high degree of core IT technical background and skills Design, develop, and maintain the end-to-end technical aspects (hardware – on prem and/or cloud) of a high performance compute and storage platform designed specifically for data pipelines in a research data intensive environment Design, develop, and maintain the end-to-end data pipelines (software, DBs and processes) required to support the research scientists and data managers. Support ETL processes including, data ingestion, transformation, validation, and integration processes using various tools and frameworks Optimize data performance, scalability, and security Provide technical guidance and support to data analysts and research scientists. Design data integrations and data quality frameworks Work and collaborate with the rest of the IT department to help develop the strategy for long term scientific Big Data platform architecture Document and effectively communicate data engineering processes and solutions. Make use of and help define the right cutting edge technology, processes and tools needed to drive technology within our science and research data management departments. Minimum Qualifications: Bachelor`s degree or higher in Computer Science, IT, Engineering, Mathematics, or a related field Industry recognized IT related certification and technology qualification such as Databases and Data related certifications. This is a technical role so a strong focus needs to be on technical skills and experience Minimum Experience: 7+ years experience in a Data Engineering, High Performance Computing, Data Warehousing, Big Data Processing Strong experience with technologies such as Hadoop, Kafka, Nifi or Spark or Cloud-based big data processing environments like Amazon Redshift, Google BigQuery and Azure Synapse Analytics At least 5 years advanced experience and very strong proficiency in UNIX, Linux, Windows Operating systems and preferably containerization technologies like Docker, Kubernetes etc Working knowledge of various data related programming, scripting or data engineering tools such as Python, R, Julia, T-SQL, PowerShell, etc. Strong working experience with software compute and virtualization platforms such as VMware, Hyper-V, OpenStack, KVM etc. Strong working experience with hardware compute platforms including high performance compute cluster hardware and related technologies. Knowledge and Abilities: Strong Experience working with various relational database technologies like MS SQL, MySQL, PostgreSQL as well as NoSQL databases such as MongoDB, Cassandra etc. Experience of Big Data technologies such as Hadoop, Spark and Hive Experience with data pipeline tools such as Airflow, Spark, Kafka, or Dataflow Experience working with containerization is advantageous Experience with data quality and testing tools such as Great Expectations, dbt, or DataGrip is advantageous Experience working with Big Data Cloud based (AWS, Azure etc) technologies is advantageous Experience with data warehouse and data lake technologies such as BigQuery, Redshift, or Snowflake advantageous Strong Experience designing end-to-end data pipelines including the compute hardware infrastructure. Strong knowledge of data modeling, architecture, and governance principles Strong Linux Administration skills Programming skills in various languages advantageous Strong data security and compliancy experience Excellent communication, collaboration, and problem-solving skills Ability to work independently and as part of a cross-functional team Interest and enthusiasm for medical scientific research and its applications SUMMARY: There is large emphasis on the technical element in the role (having experience working with and designing hardware clusters from the ground up). They have a number of technical skills and experience already within the IT department but are looking for someone very strong, and with the past working experience, to help design these clusters and understand the technology in play for advanced scientific computational requirements. This goes beyond just the hardware element though and you will need strong Linux, Data Pipeline and coding/scripting skills. Although this is not a developer role, you should be comfortable with a certain level of coding/scripting and data analysis packages with strong virtualization skills like VMware, HyperV, OpenStack, KVM etc. You will therefore be that technical link with the scientists and will help build platforms (could be hardware, software, cloud etc) and then also manage the Data and Data Pipelines on this system. This would include the compliancy, performance and security of the data too.
Head Of Data Engineering position available in Kwazulu-Natal, Durban. This job position was posted by Carlysle Human Capital. The job has been posted as a premium ad on 2024-11-24 at 13:39:27 in the It Computer category
Click Go Apply to apply online!
You might also like to look at the available jobs in the same area.
Apply directly on jobmail for this position
Some posts have contact information included with other instructions. All applicants are urged to read the instructions carefully.
We will list jobs at the bottom of each job, so feel free to scroll down and continue your search.
We do not accept any job applications and merely offer the service of daily jobs being mined and displayed on our job portal.
Each job post has a link to take you directly to the original post to apply on their website or the original website where the post came from.
Here are some more related positions
Klerksdorp: Head Of Department: Admin Head And Head Receptionist – Urgently Needed posted by Drs Alberts Bouwer en Jordaan Incorporated
Minimum requirements:Grade 12Management Qualification (degree or diploma)At least 5 years of work experience in the medical field.At least 2 years of work experience in the medical field as a head of department or manager. The role requires a people/patien
View Job
Head Of Department
Stellenbosch: Snr Talent Acquisition Manager – Head Of Talent Acquisition posted by YAM Legacy
E-mail CV`s to YAM Legacy Snr Talent Acquisition Manager - Head Of Talent Acquisition position available in Western Cape, Stellenbosch. This job position was posted by YAM Legacy. The job has been posted as a free ad on 2024-10-03 at 14:29:42 in the Man
View Job
Snr Talent Acquisition Manager – Head Of Talent Acquisition
Africa: Head Of Communications (Jb4806) posted by Kontak Recruitment
"Head of Communications (JB4806) Remote for applicants residing in Africa Market-RelatedDuration: Permanent We are seeking a highly skilled Head of Communications to lead and manage both internal and external communication strategies for a clinical researc
View Job
Head Of Communications (Jb4806)
Gauteng: Head Of Regulatory Advocacy & Market Access (Jb4807) posted by Kontak Recruitment
"Head of Regulatory Advocacy & Market Access (JB4807) Remote, for applicants residing in Africa Market RelatedDuration: PermanentWe are seeking a highly experienced Head of Regulatory Advocacy & Market Access to lead regulatory strategies and advocacy effo
View Job
Head Of Regulatory Advocacy & Market Access (Jb4807)
Johannesburg: Head Of Operations (Solar)
Company Description: We are seeking a highly motivated and experienced individual to join our team as the Solar Head of Operations. As the Solar Head of Operations, you will be responsible for overseeing the efficient and effective operation of our solar e
View Job
Head Of Operations (Solar)
Cape Town City Centre: Head Of Programmes
Our client is looking for a Head of Programmes. This role is responsible for the overall planning, execution, and delivery of the organization`s programmes. This includes developing and managing the programme portfolio, ensuring that programmes are aligned
View Job
Head Of Programmes
Hoedspruit: Head Of Housekeeping
Key Responsibilities: Manage and coordinate housekeeping operations to maintain high standards of cleanliness and hygiene. Train, supervise, and motivate housekeeping staff to deliver excellent service. Conduct regular inspections of guest rooms and public
View Job
Head Of Housekeeping
Cape Town City Centre: Head Of Operations
Our client is seeking a Head of Operations to join their team located in Cape Town. The Head of Operations is responsible for overseeing the company`s daily operations. This role involves optimising operational workflows to ensure timeous delivery of goods
View Job
Head Of Operations
Kwazulu-Natal: Head Of Hr – South Africa
Head of HR (POS24345) AREA: South Africa SALARY: Market Related Job Description: In this role you will be responsible for executing the entire HR Strategy (ca. 500 employees), including an oversight to the Site from setting the agenda and people priorities
View Job
Head Of Hr – South Africa
Johannesburg: Head Of Department – Intermediate Phase (Tembisa)
Qualifications and Skills: - Bachelor of Education (BEd) in Intermediate Phase, with SACE registration. - At least 3 years of experience in school management or a similar leadership role. - Excellent communication skills in both English and Afrikaans, able
View Job
Head Of Department – Intermediate Phase (Tembisa)
Johannesburg: Head Of Legal And Compliance
JOB DESCRIPTION INFORMATION: Job Title: Head of legal and compliance Employment Type: Permanent, On-site Work Location: Johannesburg - Sandton JOB CONTEXT: We`re on the lookout for a Head pf legal and compliance to be the driving force behind our legal pro
View Job
Head Of Legal And Compliance
Sandton: Head Of Legal And Compliance
The Head of Legal and Compliance will be responsible in the strategic delivery of the legal and compliance team and function. This will include partnering with top management to deliver legal solutions to mitigate risks relating to strategic priorities, bu
View Job
Head Of Legal And Compliance
Johannesburg: Head Of Finance
Head of Finance Media & Entertainment Industry Key Responsibilities: Lead financial planning, budgeting, and forecasting to support strategic goals. Oversee financial reporting, ensuring accuracy and compliance. Collaborate with executive leadership on inv
View Job
Head Of Finance
Port Elizabeth: Head Of Quality & Responsible Pharmacist
Main Purpose of the Job: Management of the Quality Development of Quality Culture Strategic Development & Improvement of Quality Compliance to meet international standards Assuring of Compliance with international statutory & GMP requirements Assuring of C
View Job
Head Of Quality & Responsible Pharmacist
Johannesburg: Head Of Finance
Main Areas of Responsibility: Lead the finance team, managing financial operations, accounting, and internal controls. Develop and refine financial models, ensuring that capital allocation yields favorable returns. Prepare business cases and investment pro
View Job
Head Of Finance
Cape Town City Centre: Head Of Data And Ops
The alternative candidate we’re interested in searching the market for is a Senior Manager of Data Operations to lead our data strategy, optimize contact sourcing, and manage our data team within a B2B outbound sales environment. This role will oversee dat
View Job
Head Of Data And Ops
South Africa: Head Of Employee Rewards And Recognition
Bachelors degree in Human Resources, Business Administration, Finance, or a related field. Certified Compensation Professional (CCP) and Global Remuneration Professional (GRP) (advantageous). 10 years in Compensation and Benefits (3 years in a leadership r
View Job
Head Of Employee Rewards And Recognition
Johannesburg: Head Of Tax
This company is a prominent financial institution dedicated to providing comprehensive banking solutions and services to individuals, businesses, and communities. With a rich history and a strong commitment to innovation. This company prides itself on fost
View Job
Head Of Tax
Cape Town City Centre: Executive Head Of Project Development
The Royal Portfolio is seeking an Executive Head of Project Development to work alongside our owners and management teams on new construction and renovation projects. The ideal candidate must have a strong background in property development and infrastruct
View Job
Executive Head Of Project Development
South Africa: Head Of Cards & Business Development (Kenyan Diaspora)
Key Responsibilities: Strategic Leadership & Management: Develop and execute a comprehensive strategy for the card banking division to drive market share, profitability, and customer satisfaction. Oversee the development of credit, debit, and prepaid card
View Job
Head Of Cards & Business Development (Kenyan Diaspora)
Johannesburg: Head Of Customer Sales And Marketing
The Opportunity We are looking for a customer facing Sales Leader who has proven ability to execute against ambitious sales targets as well as supporting the development of the programme and outreach to the investment community. They must be able to work i
View Job
Head Of Customer Sales And Marketing
Johannesburg: Head Of Finance
My client is a global leader in the pharmaceutical industry. This organization is dedicated to discovering, developing, and delivering innovative medicines that improve patient health and quality of life. Operating in over 150 countries, the company has a
View Job
Head Of Finance
Johannesburg: Head Of Admin
Kindly apply if you meet the minimum requirements. Should you not hear back from us within 2 weeks consider your application as unsuccessful. Head Of Admin position available in Gauteng, Johannesburg. This job position was posted by . The job has been post
View Job
Head Of Admin
Cape Town City Centre: Head Of Port Operations – Expat Mining
Qualification and REQUIREMENTS Degree in Civil/mechanical engineering or equivalent. At least 18-20 years ` experience in managing Port construction projects and port operations , ideally in the mining sector . Certificates in port management to ensure tha
View Job
Head Of Port Operations – Expat Mining
Midrand: Head Of High School (Private School)
Honours Degree in Education (recommended) or related field. 10 years` experience in a management position at an educational institution. At least 10 years high school teaching experience. Knowledge of High School curriculum. Ability to lead and nurture you
View Job
Head Of High School (Private School)
Email Alerts
Click the button below and submit your email to receive email alerts with the latest jobs.
This service is offered by Result Generation
We do not save your information, and your email is the only information we forward to the service.
If you wish to unsubscribe, please click here
