Lead Big Data Engineer

Job details

Posted
28 April 2025
Salary
AU$800 - AU$900 per day
Location
Sydney CBD
Job type
Permanent
Discipline
Paxus Jobs
Reference
267677

Job description

We are seeking a highly skilled Lead Engineer - Big Data to join our team and work on a cutting-edge data platform. This role requires hands-on experience in designing and developing scalable data solutions, implementing modern data architectures, and leading data projects in a complex enterprise environment.

Key Responsibilities

  • Design, develop, test, and support future-ready data solutions across industry verticals.
  • Produce detailed solution designs and provide technical leadership for data engineers and testers.
  • Develop and support end-to-end batch and near real-time data pipelines.
  • Apply expertise in big data architectures, cloud platforms, AI/ML, analytics, and data governance.
  • Develop Proof of Concepts (PoCs) and Working Demos to showcase innovative solutions.
  • Collaborate with internal and external stakeholders on consulting, workshops, and delivery engagements.
  • Mentor junior engineers and consultants.

Mandatory Requirements

  • Proven leadership experience in project teams.
  • Ability to navigate large enterprise governance and processes.
  • Strong interpersonal and presentation skills for large technical forums.
  • 5+ years of experience in solution design, technical leadership, development, testing, or support roles in IT consulting (Banking/FSS preferred).
  • 3+ years of hands-on experience with Big Data/Data Lake services (cloud and on-premise).
  • Expertise in NoSQL databases (HBase) and Apache Kafka for real-time data streaming.
  • Strong understanding of data modeling (ER modeling, Third Normal Form, Dimensional Modeling).
  • Knowledge of Data Mesh Architecture and Domain Data Products.
  • Hands-on experience in Scala (Expert), Python, SQL, and shell scripting.
  • Expertise in cloud platforms such as Azure, AWS, GCP, and IBM Cloud.
  • Experience in ETL, Data Warehousing, BI solutions, and high-performance system implementations.

To be considered for the role click the 'apply' button or for more information about this and other opportunities please contact Yash Kumar Jain on [email protected] and quote the above job reference number.

Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process, including the application form in an alternate format, please contact me on the above contact details.