A Renowned public research university based in Macquarie Park is looking for a Senior Data Engineer to join their amazing team to develop and optimise data and analytics architecture, as well as optimising data flow and collection to deliver integrated and quality business information and reporting solutions.
- Setting up the Development and Test environments with CI/CD processes using DBT for transformation and Bitbucket for version control
- Write, test and validate data transformation and ETL routines within Apache Airflow, Python, Alteryx.
- Develop, test and maintain data and analytics architecture, including databases and large-scale processing systems, for optimal extraction, transformation and upload of data from multiple sources and different technologies
- Experience with cloud infrastructure and working within a Lean Engineering Agile Environment.
Skills and Experience Required:
- Expertise in DBT, Bitbucket mandatory
- Knowledge of databases, operating systems, current internet technologies and reporting toolsets.
- Knowledge of Business Information tools such as Tableau, Apache Superset or Power BI.
- Strong understanding of BI data structures and how to optimise data delivery for different BI Reporting tools
- Expertise in SQL, SQL tuning, schema design, Snowflake, Python, Kubernetes.
- Advanced knowledge and skills in Python, Apache Airflow, R, ETL and integration.
- Solid experience in building RESTful APIs and microservices, e.g., with Flask.
- Experience with solution building and architecting with public cloud offerings Azure, AWS and Google for services such as S3, Blob, BigQuery, SQL DB, Data Lake, Cosmo DB, Batch, Kubernetes and Data Factory
- ERWIN knowledge is an advantage
- Demonstrated experience in developing, implementing, and maintaining data pipelines and data models with work flow tools such as Airflow, Oozie or Luigi.
- Knowledge of the Office of Business Intelligence & Reporting policies, systems, processes, and procedures.
If you would like to find out more, please contact Smitha Wali on 0468496706 or click "Apply Now".
To be considered for the role click the 'apply' button or for more information about this and other opportunities please contact Smitha Wali on 02 9464 5849 or email firstname.lastname@example.org and quote the above job reference number.
Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process please contact me on the above contact details.