Data Engineer

Data Engineer

Job Title: Data Engineer
Contract Type: Contract
Location: Canberra
Salary: Competitive
Start Date: 2021-11-29
Reference: V-40403
Contact Name: Kelly Shockley
Contact Email:
Job Published: November 30, 2021 11:54

Job Description


The Role:

The Data Engineer will focus on the development of data streams from Department source systems that flow data into a variety of analytical tools and data stores. The role will consist of data pipeline development, pipeline deployment & monitoring (ideally leveraging experience with DevOps automation) and vendor engagement/product evaluation. It is expected that successful candidates will have an appropriate understanding on when to use existing functionality vs develop new functionality

Key Duties: 

•    Design, construct, install, test and maintain data management systems.
•    Build high-performance algorithms, predictive models, and prototypes.
•    Ensure that all systems meet the business/company requirements as well as industry practices.
•    Integrate up-and-coming data management and software engineering technologies into existing data structures.
•    Develop set processes for data mining, data modelling, and data production.
•    Create custom software components and analytics applications.
•    Research new uses for existing data.
•    Employ an array of technological languages and tools to connect systems together.
•    Collaborate with members of your team (eg, data architects, the IT team, data scientists) on the project’s goals.
•    Install/update disaster recovery procedures.
•    Recommend different ways to constantly improve data reliability and quality.
•    Work effectively in a small team environment
•    Provide knowledge / skills transfer to other team members
•    Record and maintain files / information in accordance with Department requirements


Skills and Experience Required:

•    Proven experience with the development of data streaming and processing pipelines using Python, Apache Kafka, Apache NiFi, any Directed Acyclic Graph (DAG) based python data orchestration platform or similar technologies.
•    Data development and management experience including but not limited to data migration, modelling and analytics.


•    Proven experience implementing role based access control within a data platform.
•    Proven system implementation and integration skills, including but not limited to; systems configuration and change management process and documentation.
•    Demonstrated ability to communicate and work effectively within a team
•    Expertise in any of the following technologies:
o    RHEL Linux (or similar) administration and support experience
o    Graph databases.
o    Database administration and management experience
•    Data modelling
•    Experience in, or the desire to learn :
o    administration and support of a Big Data Hadoop platform
o    Containerisation technologies
o    Cloud automation
o    Using DevOps practices in a large infrastructure environment
•    Experience working in teams with an Agile Development way of working
•    Knowledge of Australian Government information security practices and procedures


Location and Contract Term: This is a Canberra based role. 12 month contract from start date with options to extend up to 24 months pending budget and delegate approval.


Security Requirements: Applicants must be Australian Citizens to apply. The ability to obtain and maintain a NV1 security clearance is required.


How to Apply:

Applications close 9am 6 December.

Please upload your resume to apply. Please note you may need to complete selection criteria to complete this application process. We will be in touch with instructions for suitably skilled candidates

Call Kelly 1300 944 936 for any further information