Gauteng: Senior Data Engineer posted by Psybergate

About the Role

At Psybergate, we are seeking an experienced Senior Data Engineer to design, build, and maintain scalable data pipelines and lakehouse structures that support our analytics, BI, machine learning, and Generative AI applications. The ideal candidate will have a strong understanding of enterprise data lake and lakehouse architecture, as well as experience building and operating production-grade data platforms.

Key Responsibilities

• Design, build, and maintain scalable data pipelines and lakehouse structures
• Deliver data solutions supporting analytics, BI, machine learning, and Generative AI applications
• Apply enterprise data lake and lakehouse principles to ensure solutions are reliable, secure, governed, and fit for downstream consumption
• Translate business and analytical requirements into production-ready data solutions
• Build and operate solutions using Databricks, including Delta Lake, Databricks Jobs & Workflows, Unity Catalog, Databricks Bundles, notebooks, and shared libraries
• Enable data consumption for GenAI use cases such as RAG, AI services, and agent workflows
• Support analytics platforms, reporting tools, and downstream operational systems
• Build data pipelines for Generative AI applications, including curated knowledge datasets, structured and semi-structured data, metadata, and lineage management
• Enable GenAI data patterns including Retrieval Augmented Generation (RAG), prompt/context preparation, and AI model input/output flows
• Work closely with AI Engineers and Product Owners to align engineering deliverables to AI and GenAI use cases
• Develop production-grade pipelines using Python, PySpark, SQL, and Apache Spark
• Implement automated testing and CI/CD practices for data engineering workloads
• Ensure data solutions are observable, resilient, performant, and cost-efficient
• Support operational stability, incident resolution, and root cause analysis
• Collaborate within Agile, cross-functional product squads alongside AI/ML engineers, analytics teams, platform teams, and security stakeholders
• Contribute to engineering reviews, standards, and design discussions
• Maintain documentation, operational runbooks, and governance compliance

Requirements

• Relevant Degree or Diploma in Computer Science, Information Technology, Data Engineering, or related field
• 6+ years’ experience as a Senior / Lead Data Engineer
• 2+ years’ hands-on experience working in Databricks environments
• Strong understanding of enterprise data lake and lakehouse architecture
• Strong proficiency in Python, SQL, and Apache Spark
• Experience building and operating production-grade data platforms
• Experience working in enterprise or regulated environments
• Strong understanding of data governance, security, and operational best practices
• Experience working in Agile, product-aligned squads
• Strong analytical and problem-solving skills
• Excellent collaboration and communication skills

Salary & Benefits

Salary information not available.

Qualifications

No formal education or certifications required.