Transport For NSW - Senior Data Engineer (AWS Big Data) (6 Months temporary Opportunity)

Data Scientist is needed in Sydney, Australia.

Client: Transport For NSW

Location: Sydney NSW, Australia

Contract: No Job Type Listed

Job Description

In this role, you will be responsible for working as part of a team delivering Data Architecture & Data Analytics. You will build data pipelines by integrating various data sources, maintaining, improving, cleaning, and manipulating data from divisional and other databases. Additionally, you will create and maintain optimal data pipelines from internal and external sources into a large cloud environment, implementing against reference designs to meet functional requirements.

Requirements

• Expertise in Cloud ETL development experience using various ETL tools and extensive knowledge in Data Warehousing principles.
• Design, develop, and implement ETL pipelines using cloud ETL tools (e.g. Matillion) to extract, transform, and load data into a cloud data warehouse (e.g. Snowflake).
• Design and maintain data models within Snowflake for query optimization and data analysis.
• Strong SQL skills to write efficient queries within Snowflake (SnowSQL).
• Develop and implement data quality checks and monitoring processes to ensure data accuracy and consistency.
• Utilize AWS S3 for data storage and retrieval in the data pipeline.
• Automate data pipeline workflows and scheduling using tools like cron or CloudWatch.
• Troubleshoot and resolve issues related to data pipelines, Snowflake, and AWS services.
• Collaborate with engineers, analysts, and stakeholders to understand data needs and deliver solutions.
• Document data pipelines and processes for clear communication and future reference.
• Experience deploying and maintaining data pipelines successfully in a production environment.

Other Information

• This is a full-time opportunity until Feb 2025 with potential for extensions.
• The role is based at Parramatta with flexible work practices, offering hybrid work options.
• Transport for NSW provides a safe, integrated, and efficient transport system, connecting people, communities, and industry daily.
• Support for people with disabilities is available throughout the recruitment process and at work.