Lead Data Engineer for Data Analytics
The Department requires data engineering capability to design and evolve analytics capability across a hybrid environment, encompassing continued development of on‑premises analytics capability alongside establishment of a modern cloud-based analytics platform. The role has a strong emphasis on production-grade engineering, and DevOps practices. It delivers priority data pipelines using a medallion-style architecture to support advanced analytics and AI use cases.
Key duties and responsibilities
The Lead Data Engineer for Data Analytics will:
- Work collaboratively with a small multidisciplinary team to stand up a cloud analytics platform that enables analytics and advanced analytics use cases, using contemporary cloud-native and hybrid data engineering patterns (e.g., Azure, Microsoft Fabric).
- Design and implement priority data pipelines aligned to a Medallion-style layered architecture, supporting repeatable ingestion, transformation and analytics workflows.
- Implement modern DataOps/DevOps practices for data engineering, including CI/CD for pipelines and notebooks, infrastructure-as-code, environment promotion, automated testing, release management, and operational runbooks.
- Implement monitoring/observability and operational controls, including logging, alerting, data quality checks, lineage capture, and metadata management patterns aligned to governance tools (e.g., Microsoft Purview).
- Build scalable data engineering solutions using distributed processing technologies, such as Spark‑based compute engines, appropriate to the hybrid environments.
- Apply awareness of enterprise integration patterns (e.g., streaming ingestion, mirroring/CDC, event‑driven orchestration, hybrid data movement and virtualisation patterns) to deliver robust and reliable integration across cloud and on‑premise environments.
- Implement reusable patterns, standards and templates to support consistent analytics delivery across datasets and use cases.
- Collaborate closely with architects, analysts and data science practitioners to enable analytics and data science capability on top of shared, well-governed data foundations.
- Operate effectively within secure or sensitive environments, applying appropriate security, privacy and compliance considerations when designing and delivering data engineering solutions.
- Contribute to uplift of analytics engineering capability through documentation, knowledge sharing and pragmatic, maintainable design choices.
- Experience working within modern Azure, Microsoft Fabric and Purview ecosystems, with an understanding of cloud‑native analytics and governance capabilities.
- Experience enabling data science and advanced analytics capability on cloud or on-prem data platforms.
- Experience working with graph databases for graph-enabled analytics.
- Experience applying enterprise integration patterns (e.g., streaming ingestion, CDC/mirroring, event‑driven orchestration, hybrid integration or virtualisation) to support reliable data movement and interoperability across systems.
- Experience applying data mesh type architectural patterns to support governed, scalable and domain‑aligned data products.
- Experience with MLOps.
- Experience working within an Intelligence context.
- Demonstrated experience designing and building modern analytics platforms that support analytics use cases.
- Strong experience implementing data pipelines and data transformations for repeatable ingestion and curation workflows.
- Demonstrated experience with modern data engineering toolchains in cloud environments, with skills transferable across cloud providers.
- Experience working with distributed data processing technologies (e.g., Spark or equivalent) to develop scalable batch or streaming data solutions.
- Demonstrated ability to translate analytical and data science requirements into robust data engineering solutions.
Security Required: NV1 Security Clearance required
Location- Canberra based
How to Apply - Please upload your resume to apply. Candidates will need to be willing to undergo pre-employment screening checks which may include, ID and work rights, security clearance verification and any other client requested checks
Closing date: Monday 6 April 2026
Call Joanne Finchett on 0480 002454 or email Joanne@whizdom.com.au for any further information
