Hastha Solutions - Data Engineer - Contract - Melbourne/Sydney
AI Engineer is needed in Melbourne, Australia.
Client: Hastha Solutions
Location: Melbourne, VIC, au
Contract: Contract
Job Description
We have an urgent requirement for a Data Engineer on a contract basis in Melbourne/Sydney. The successful candidate will be responsible for designing, developing, and optimizing data pipelines using Azure Databricks and Apache Spark, as well as maintaining robust ETL/ELT frameworks and ensuring data integrity and reliability.
Requirements
Essential Experience:
- 5+ years of Data Engineering experience in enterprise environments.
- 3+ years of hands-on experience working with Azure Databricks and Apache Spark.
- Proven experience delivering data solutions on Azure cloud platforms.
- Experience designing and implementing scalable data pipelines and data lake architectures.
- Strong experience with Agile delivery methodologies.
- Ability to work independently and contribute immediately within a project-based environment.
- Strong stakeholder management and communication skills.
Key Responsibilities
- Design, develop, and optimize data pipelines using Azure Databricks and Apache Spark.
- Build and maintain robust ETL/ELT frameworks to ingest, transform, and load data from multiple source systems.
- Develop scalable data solutions leveraging Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
- Implement data quality, validation, and monitoring frameworks to ensure data integrity and reliability.
- Collaborate with solution architects, business analysts, data scientists, and stakeholders to deliver project outcomes.
- Optimize Spark workloads, cluster configurations, and data processing performance.
- Support deployment activities through CI/CD pipelines and DevOps practices.
- Troubleshoot production issues and provide timely resolution of data platform incidents.
- Contribute to technical documentation, knowledge transfer, and operational handover activities.
- Ensure adherence to security, governance, compliance, and enterprise data standards.
No longer accepting applications
Get instant notifications for new AI Engineer jobs. Enter your email:
How It Works
🔍Get quality leads
Review job leads for free, filter by local or global clients, and get real time notifications for new opportunities.
🎉Apply with ease
Pick the best leads, unlock contact details, and apply effortlessly with Twine's AI application tools.
📈Grow your career
Showcase your work, pitch to the best leads, land new clients and use Twine’s tools to find more opportunities.