Johannesburg: BI_BA Developer (Databricks, Spark SQL, PySpark) (JHB) posted by DataFin Recruitment

ENVIRONMENT:

A dynamic Healthcare Platform seeks a Business Intelligence/Business Analyst Developer to fill a hands-on role focused on building and maintaining reporting and analytics solutions on a modern lakehouse platform. The primary focus will be the development of management reporting and operational analytics for key data partners, alongside contributing to the broader data platform. The successful candidate will work to specifications agreed with the General Manager and associated data partners and deliver against a defined backlog of reporting, modelling, and pipeline development work. You will combine strong SQL and BI development capability with the analytical mindset needed to translate business requirements into well-modelled, accurate, and maintainable data products. Applicants will require a Bachelors Degree in IT/ Computer Science/Information Systems or related field with 3-6 years experience and proficiency with Databricks, Spark SQL, PySpark, T-SQL, Power BI, Python, Git & Advanced Excel skills including Power Query and Power Pivot.

DUTIES:

BI Development & Reporting –

• Design, build, and maintain management and operational reports using Power BI.
• Develop semantic models and DAX measures against the Databricks lakehouse.
• Build dashboards and KPI views for management stakeholders.
• Tune Power BI datasets and reports for performance, accuracy, and usability.
• Maintain and extend existing reports, extracts, and dashboards as requirements evolve.
• Validate report outputs against source data and resolve discrepancies.

Data Pipeline Development on Databricks –

• Develop and maintain notebooks and workflows across the medallion architecture.
• Write Spark SQL and Python to transform pharmacy claims, dispensing, and product data.
• Support Lakeflow Connect ingestion from Azure SQL and other source systems.
• Apply Unity Catalog governance, access controls, and data classification standards.
• Optimise notebook and workflow performance for large datasets.
• Contribute to platform documentation, code reviews, and version control discipline via GitHub.

Business Analysis & Requirements –

• Work with the General Manager and internal stakeholders to clarify reporting and analytics requirements.
• Translate business questions and specifications into clear data and reporting designs.
• Document data definitions, business rules, and report specifications for ongoing maintenance.
• Conduct ad-hoc analysis to answer specific operational and commercial questions as required.
• Validate deliverables internally before release and incorporate feedback iteratively.

Data Quality & Documentation –

• Investigate and resolve data quality issues across the pipeline.
• Maintain documentation of models, measures, pipelines, and business logic.
• Apply consistent naming conventions and coding standards across all development work.
• Participate in peer review of code, models, and reports.

REQUIREMENTS:

Qualifications

• Bachelor Degree in Information Technology, Computer Science, Information Systems, Data Science, Statistics, or a related field, or equivalent demonstrable experience.

Experience/Skills

• 3-6 Years of experience in Business Intelligence Development, Data Analytics, or a closely related role.
• Demonstrated experience with Power BI in a production environment.
• Working experience with Databricks or a comparable lakehouse platform.
• Proven SQL development experience on a relational database platform.

Required Technical Skills

Databricks & Lakehouse Platform:

• Working experience with Databricks for data processing and analytics.
• Proficiency in Spark SQL and PySpark for data transformation.
• Understanding of the medallion architecture pattern.
• Familiarity with Unity Catalog, Delta Lake, and lakehouse concepts.
• Experience developing and maintaining production Databricks notebooks and workflows.

SQL & Data Modelling:

• Strong proficiency in SQL and T-SQL, including complex joins, window functions, and query optimisation.
• Experience with dimensional modelling, star schema design, and semantic layer construction.
• Understanding of data warehousing concepts and modern lakehouse equivalents.
• Experience with Azure SQL Database as a source system.

Business Intelligence & Reporting:

• Strong proficiency in Power BI for dataset design, report development, and DAX measure authoring.
• Experience designing semantic models that connect Power BI to a lakehouse backend.
• Ability to design dashboards that match the needs of executive, manager, and operational user personas.
• Advanced Excel skills including Power Query and Power Pivot.

Programming & Tooling:

• Working proficiency in Python for data processing and notebook development.
• Comfortable with Git and pull request workflows for version control.
• Familiarity with CI/CD concepts for data and BI deployment.

Cloud Platform:

• Working knowledge of Azure data services, particularly Azure SQL and Azure storage.
• Understanding of cloud security and access control fundamentals.

Preferred to haves

• Pharmaceutical or healthcare domain knowledge (highly preferred): Direct experience with pharmaceutical, pharmacy, or healthcare data including claims, prescriptions, dispensing records, or market research data.
• Databricks certification (Data Engineer Associate or Data Analyst Associate).
• Microsoft Power BI Certification.
• Azure Data Certifications (Azure Data Engineer Associate or Azure Data Fundamentals).
• Familiarity with pharmaceutical data products.
• Exposure to legacy SQL Server BI stack (SSIS, SSRS, SSAS) for transition and integration scenarios.
• Knowledge of POPIA and privacy-preserving analytics concepts as applied to healthcare data.

ATTRIBUTES:

• Strong analytical and problem-solving abilities.
• Strong attention to detail and a quality-first mindset on data accuracy.
• Clear written and verbal communication, particularly in documenting work and explaining technical detail.
• Ability to manage multiple priorities and meet delivery commitments.
• Self-directed, with the ability to scope and progress work independently once requirements are clear.
• Strong organisational skills and disciplined documentation habits.