Hastha Solutions - AWS Data Engineer - Contract - Sydney

Data Scientist is needed in Sydney, Australia.

Client: Hastha Solutions

Location: Sydney, NSW, au

Contract: Contract

Job Description

We are seeking an AWS Data Engineer for a contract position in Sydney. The role involves designing and implementing scalable data pipelines, integrating with Amazon Redshift, managing data lakes using Amazon S3, and building batch processing jobs with AWS services. The engineer will also support streaming data workflows, ensure secure data handling, and collaborate with cross-functional teams to deliver end-to-end solutions.

Key Responsibilities

• Design and implement scalable data pipelines using Python or PySpark, and boto3.
• Integrate with Amazon Redshift for data warehousing and perform data ingestion and transformation tasks.
• Work with Amazon S3 to architect and manage data lakes.
• Build and manage batch processing jobs using AWS Glue, EMR, and Lambda.
• Support streaming data workflows using Amazon Kinesis.
• Leverage AWS Secrets Manager and KMS for secure data handling and encryption.
• Write efficient SQL for data querying, transformations, and performance tuning.
• Collaborate with data engineering, product, and DevOps teams to deliver end-to-end solutions.
• Participate in code reviews, debugging, and testing activities to ensure high-quality software delivery.
• Use Git for version control and follow best practices in CI/CD.
• Continuously explore and recommend improvements to enhance system performance, scalability, and maintainability.

Required Skills

• Advanced proficiency in Python, including libraries such as Python/PySpark, Pandas, and boto3.
• Experience designing workflows in Apache Airflow (or MWAA).
• Hands-on experience with Amazon Redshift for analytics and data warehousing.
• Solid SQL skills and experience with data modeling and transformation.
• Comfortable with Amazon S3, including setting up data lake architecture.
• Familiar with AWS core services: EC2, S3, IAM, Lambda.
• Experience using AWS Secrets Manager and AWS KMS for secure data access.
• Strong understanding of batch processing (AWS Glue, EMR) and streaming (Kinesis).
• Version control using Git.
• Excellent problem-solving and communication skills.

Good to Have

• Knowledge of additional ETL/ELT tools like AWS Glue, AWS Athena.
• Familiarity with DevOps practices and CI/CD pipelines.
• Experience working in agile teams and data-focused environments.
• Knowledge in reading and understanding JavaScript (Node.js/Express).