Upstaff - DataOps/Cloud Data Engineer – Senior

Data Engineer is needed in Toronto, Canada.

Client: Upstaff

Location: Toronto, ON, ca

Contract: Contract

Job Description

We are seeking a Senior DataOps/Cloud Data Engineer to design, develop, optimize, and support modern cloud data solutions within a large-scale public-sector environment. The successful candidate will have strong hands-on expertise with Azure Data Factory, Databricks, Informatica, Python, SQL, and Medallion Architecture, along with proven experience building enterprise-scale data pipelines, lakehouses, data models, and cloud data integrations.

The consultant will play a key role in modernizing existing ETL processes, including the migration of Informatica ETL to Azure Data Factory and Databricks ELT, while ensuring data quality, security, governance, performance, and operational reliability.

Requirements

Mandatory Technical Skills

• Informatica ETL
• Azure Data Factory (ADF)
• Azure Databricks
• Databricks Medallion Architecture
• Python
• SQL
• T-SQL
• PL/SQL
• SSIS
• Microsoft Fabric
• Lakeflow
• SQL Optimization
• Delta Lake
• Data pipeline development, workflow orchestration, deployment, and automation
• Dataflow, parallelism, data movement, and Delta Lake indexing
• Cloud data engineering and cloud data platforms
• Data Lake, Data Warehouse, and Lakehouse solutions
• Relational and dimensional data modelling
• Fact/dimension modelling and star schema
• Enterprise data warehouse and data lake/lakehouse development
• Cloud and on-premises data integration
• CI/CD and automated data provisioning
• Azure DevOps
• Data quality, profiling, cleansing, validation, and monitoring
• Data governance and data lineage
• Microsoft Entra ID and RBAC
• Data anonymization and masking
• High-volume/VLD data migration and transformation
• OLTP and OLAP environments

Core Qualifications

• Strong analytical, problem-solving, decision-making, communication, presentation, and interpersonal skills.
• Proven experience designing and developing complex data and business intelligence solutions.
• Strong DataOps best practices and Agile development/deployment experience.
• Experience working with large and diverse structured and unstructured datasets.
• Experience with data analysis, data profiling, KPI development, and data visualization.
• Strong production troubleshooting and issue-resolution experience.
• Experience coordinating modernization of large, complex, multi-platform and multi-tier technology environments.
• Experience creating detailed business, system, technical, and solution design documentation.
• Experience with AODA/WCAG compliance.
• Demonstrated ability to work collaboratively across multiple teams and meet strict deadlines.

Project & Delivery Experience

• Strong experience with SDLC, Agile, and Waterfall methodologies.
• Experience conducting unit testing, system testing, SIT, SAT, and UAT.
• Experience tracking, reporting, and resolving project issues and risks.
• Experience gathering and consolidating business and technical requirements.
• Experience leading requirements-gathering sessions and supporting change requests and project artifacts.
• Experience coordinating multiple projects and competing priorities.
• Ability to provide technical leadership and guidance to development and business teams.
• Ability to ensure business requirements are accurately reflected in solution designs and technical specifications.

Leadership & Collaboration

• Lead and coordinate technical teams and workstreams.
• Provide technical advice and mentorship to team members.
• Work effectively with business users, project managers, architects, developers, and IT teams.
• Facilitate solution integration across multiple teams and platforms.
• Communicate technical concepts clearly to both technical and non-technical stakeholders.
• Demonstrate strong ownership, accountability, and delivery focus.

Public Sector Experience

• Previous experience working in a public-sector organization of comparable size is strongly preferred.
• Experience delivering enterprise-scale data modernization, cloud migration, or data integration initiatives within government or large regulated environments is an asset.

Certifications

Databricks and/or Microsoft Fabric certifications are considered an asset.

Additional Information

The consultant will be responsible for transferring knowledge to the designated Community Services Cluster resource. Knowledge transfer must be completed one week before the project end date or one week before the consultant leaves the Ministry, whichever applies.

Knowledge transfer will include:

• All deliverables and supporting documentation.
• Design, support, release, and training documentation.
• Project requirements and solution design documentation.
• Documentation stored in designated repositories such as Git, SharePoint, HPQC, TFS, or other approved project repositories.
• Written communication to the Project Manager and designated Ministry staff identifying where documentation has been stored.
• At least one walkthrough of the project documentation.
• One-on-one knowledge-transfer sessions, email communication, document updates, and documentation reviews.

Ideal Candidate Profile

The ideal candidate is a senior hands-on Cloud Data Engineer/DataOps professional who combines deep expertise in Informatica, Azure Data Factory, Databricks, Python, SQL, and Medallion Architecture with strong enterprise data modelling, cloud migration, DataOps, CI/CD, data governance, and production support experience.

Important: This is a 100% onsite position requiring the consultant to work from 777 Bay Street, Toronto, five days per week.