Johannesburg: Databricks Developer posted by Crayon Technologies Ltd

How you’ll role The Databricks Developer is a key role focused on designing and implementing scalable data pipelines. The developer will transform raw data into analytics-ready datasets and support AI workloads using the Databricks platform. The position involves collaborating closely with data engineering, integration, and cloud architecture teams to further enterprise-level data projects. Candidates should bring extensive experience in Apache Spark, cloud platforms, and modern data engineering practices. What you’ll do Develop, maintain, and optimize ETL/ELT pipelines using Databricks, PySpark, and Spark SQL Work on batch and streaming data processing using Delta Lake and structured streaming Design and maintain data models aligned to Lakehouse principles and implement Delta Lake tables Integrate Databricks pipelines into enterprise CI/CD platforms Develop unit tests, data quality checks, and validation frameworks to ensure pipeline reliability What you’ll need Proficiency with Databricks and Spark, including Databricks Workflows and Notebooks Strong knowledge of Python (primary), SQL, and Scala (nice-to-have) Experience with cloud platforms, particularly Azure Databricks, or AWS/GCP equivalents Understanding of data engineering tools such as Airflow, Kafka, and Databricks Repos Ability to implement secure data access using IAM, Unity Catalog, and cluster policies