We are seeking a dedicated Data Engineer to join our dynamic development team that manages solutions on the Departments AWS Cloud Ecosystem. You will be working with a large group of technical specialists across different areas and will be instrumental in undertaking data integration and building scalable data pipelines to support AI projects.
It is essential that the candidate has experience integrating multiple source data systems with AWS Cloud platforms and developing scalable data pipelines and workflows.
The primary project that you will be working on relates to developing AI tools that augment and enhance staff capabilities.
Established domain knowledge with data processing for AI systems and experience with relevant AWS Cloud services is highly desirable
About the Role
Key Responsibilities:
- Build, maintain, and optimise batch and streaming data pipelines for ingesting, processing, and transforming large volumes of data using AWS services (e.g. S3, Glue, Lambda, Step Functions, Kinesis, Redshift, Athena), ensuring performance, reliability, and cost efficiency.
- Develop, integrate and manage ETL/ELT workflows from multiple source systems (APIs, databases, files, streaming sources)
- Ensure data quality, integrity, and reliability through validation, monitoring, and testing
- Monitor, support, test, and troubleshoot data pipeline and performance issues
- Promote and implement data engineering best practices using modern, automated, and repeatable approaches.
- Ensure all work complies with relevant legislation, policies, requirements, and compliance standards
- Maintain awareness of emerging data engineering practices and industry trends and apply relevant innovations to improve organisational capability.
- Collaborate effectively with delivery teams, platform teams, security teams, and stakeholders to understand data requirements and implement data governance, security, and access controls
- Communicate clearly and effectively with technical and non‑technical stakeholders, including documenting solution designs, pipelines, and ongoing support procedures.
- Actively contribute to a positive team culture by mentoring, coaching, and upskilling colleagues in AI data processing practices.
- Participate effectively in Agile delivery environments, contributing to planning, estimation, sprint activities, and continuous improvement within SCRUM or similar frameworks.
Our Ideal Job Seeker
Skills and experience:
Mandatory:
- 5+ years’ experience delivering enterprise data integration and pipeline solutions in AWS
- Hand-on experience building, maintaining and optimising batch and streaming data pipelines for ingesting, processing, and transforming large volumes of data using AWS services (e.g. S3, Glue, Lambda, Step Functions, Kinesis, Redshift, Athena)
- Hand-on experience integrating multiple data source systems with an AWS cloud platform e.g. data lake/warehouse/API
- Hand-on experience building end‑to‑end data pipelines for processing unstructured content for use by AI tools e.g. chunking, embedding and vector database storage (OpenSearch)
- Hands-on experience with ETL/ELT tools and frameworks
- Strong proficiency with SQL and Python
- Experience with Infrastructure as Code (Terraform/Ansible)
- Experience building and running workloads on Kubernetes
- Strong knowledge of AWS security principles and knowledge of data security and privacy concepts
- Strong experience with DevOps/GitOps, automation/scripting tools and CI/CD and version control.
- Minimum tertiary level qualification in Computer Science, Information Technology or equivalent
Desired:
- Relevant AWS certifications or equivalent e.g. AWS Certified Data Engineer – Associate
- Experience working in complex enterprise cloud environments
- Highly developed problem-solving skills and ability to work independently
- Experience working in an Agile delivery environment
- Strong collaboration and communication skills
- Open minded and able to pivot to diverse cloud engineering work
Location and Contract Term: This role can be based in Canberra, Newcastle, Sydney, Brisbane, Adelaide and Melbourne . 12 month contract with 2 x 12- month contract extension options pending budget and delegate approval.
Security Requirements: Applicants must be Australian Citizens and hold a baseline security clearance.
How to Apply: Applications close 4 June.
Please upload your resume to apply. Candidates will need to be willing to undergo pre-employment screening checks which may include, ID and work rights, security clearance verification and any other client requested checks
Please note you may need to complete selection criteria to complete this application process. We will be in touch with instructions for suitably skilled candidates
Call Kelly 0480 002 457 or email kelly@whizdom.com.au for any further information
