The team seeks a skilled Data Scientist to assist with transitioning a complex microsimulation model into a robust, scalable, and well-governed production environment. The production environment is in Databricks and needs to support streamlined scenario testing input.
Advanced skills in writing custom R functions are essential for the successful candidate, as the microsimulation model is written in R with supporting pipelines in Python.
Key Responsibilities:
- Conduct code analysis and peer review of a microsimulation model, including refactoring, optimisation, and ensuring code readiness for operational use.
- Optimise compute usage and improve performance across Spark jobs and model workloads.
- Maintain scalable data pipelines and workflows using Azure Databricks, Delta Lake, and related services.
- Implement best practice data engineering standards, including version control, testing frameworks, logging, monitoring, and error handling processes.
- Collaborate closely with other data scientists, analysts, engineers and domain experts to understand model structure and ensure accurate, performant implementation.
- Work with cloud and platform support teams to ensure the deployed product aligns with organisational standards, operational requirements, and platform capabilities.
- Document workflows, architecture, model dependencies, and operational procedures.
- Strong experience writing custom functions using R and the ability to understand complex code.
- Experience in delivering complex models into production, including validation and unit testing.
- Demonstrated capability in designing and implementing scenario testing methodologies and repeatable testing processes.
- Proficiency in Spark and related data engineering technologies, with experience optimising computational performance and scalability for large-scale data processing.
- Solid understanding of software engineering practices, including automated testing, modular code design, and version control.
- Ability to work collaboratively with technical and non‑technical stakeholders.
- Familiarity with Azure Data Factory, Azure Storage, or Unity Catalog.
- Experience working with labour market data, ABS reports and ABS TableBuilder.
- Experience in the public sector or a regulated environment.
Security Required: Baseline Security clearance required
Location - Canberra, Sydney, Melbourne, Adelaide, Perth, Hobart and Brisbane
How to Apply - Please upload your resume to apply. Candidates will need to be willing to undergo pre-employment screening checks which may include, ID and work rights, security clearance verification and any other client requested checks
Closing date: Friday 13 November 2026
Call Joanne Finchett on 0480 002454 or email Joanne@whizdom.com.au for any further information
