Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

Data engineer with 4+ years of experience building scalable, secure, high-performing Azure and Databricks data platforms. I deliver end-to-end ETL/ELT pipelines for reliable ingestion, transformation, and analytics-ready data using Azure Data Factory, Azure Databricks, and ADLS Gen2. I specialize in batch and streaming architectures with PySpark, SQL/T-SQL, and change-data-capture workflows. I focus on data quality through validation/reconciliation, optimize query and Delta Lake performance, and support production monitoring, troubleshooting, and governed lakehouse implementations with Unity Catalog.…Data engineer with 4+ years of experience building scalable, secure, high-performing Azure and Databricks data platforms. I deliver end-to-end ETL/ELT pipelines for reliable ingestion, transformation, and analytics-ready data using Azure Data Factory, Azure Databricks, and ADLS Gen2. I specialize in batch and streaming architectures with PySpark, SQL/T-SQL, and change-data-capture workflows. I focus on data quality through validation/reconciliation, optimize query and Delta Lake performance, and support production monitoring, troubleshooting, and governed lakehouse implementations with Unity Catalog.

Dharmil Nileshkumar Patel

Data Engineer, Cloud Developer, Programmer, +3





Data engineer with 4+ years of experience building scalable, secure, high-performing Azure and Databricks data platforms. I deliver end-to-end ETL/ELT pipelines for reliable ingestion, transformation, and analytics-ready data using Azure Data Factory, Azure Databricks, and ADLS Gen2. I specialize in batch and streaming architectures with PySpark, SQL/T-SQL, and change-data-capture workflows. I focus on data quality through validation/reconciliation, optimize query and Delta Lake performance, and support production monitoring, troubleshooting, and governed lakehouse implementations with Unity Catalog.…Data engineer with 4+ years of experience building scalable, secure, high-performing Azure and Databricks data platforms. I deliver end-to-end ETL/ELT pipelines for reliable ingestion, transformation, and analytics-ready data using Azure Data Factory, Azure Databricks, and ADLS Gen2. I specialize in batch and streaming architectures with PySpark, SQL/T-SQL, and change-data-capture workflows. I focus on data quality through validation/reconciliation, optimize query and Delta Lake performance, and support production monitoring, troubleshooting, and governed lakehouse implementations with Unity Catalog.

Available to hire

Data engineer with 4+ years of experience building scalable, secure, high-performing Azure and Databricks data platforms. I deliver end-to-end ETL/ELT pipelines for reliable ingestion, transformation, and analytics-ready data using Azure Data Factory, Azure Databricks, and ADLS Gen2.

I specialize in batch and streaming architectures with PySpark, SQL/T-SQL, and change-data-capture workflows. I focus on data quality through validation/reconciliation, optimize query and Delta Lake performance, and support production monitoring, troubleshooting, and governed lakehouse implementations with Unity Catalog.

Skills

Experience Level

Expert

Expert

Expert

Expert

Expert

Expert

Intermediate

Intermediate

Intermediate

Intermediate

Language

Work Experience

Azure Data Engineer at RBC.

January 1, 2025 - Present

Developed Azure data-platform components to support migration of legacy workloads to an Azure Databricks Lakehouse environment. Built and maintained scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks, Python, PySpark, and SQL Server to process millions of records in daily batch workloads. Implemented change data capture pipelines using Kafka and Spark Structured Streaming to propagate low-latency updates to downstream consumers. Created PySpark and SQL transformations for data cleansing, validation, and preparation for reporting and analytics. Optimized SQL queries and Delta table design (partitioning and data-layout/Delta optimization) to reduce pipeline execution time by ~25%. Built automated SQL script testing to detect errors before deployment, reducing deployment failures by ~30%. Supported CI/CD-style promotion across development/testing/production environments, monitored production pipelines, and performed root-cause analysis to resolve incidents and maintain

Azure Data Engineer at RBC

January 1, 2025 - Present

Developed Azure-based data platform components to migrate legacy workloads into an Azure Databricks Lakehouse environment. Built scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks, Python, PySpark, and SQL Server to process millions of records in daily batch workloads. Implemented change data capture pipelines using Apache Kafka and Spark Structured Streaming to deliver low-latency updates to downstream consumers. Created data transformation workflows in PySpark and SQL for cleansing, validation, and preparation for reporting and analytics. Optimized SQL queries and Delta tables using query tuning, partitioning, and data layout improvements, reducing pipeline execution time by ~25%. Built an automated database-script testing framework to detect SQL errors before deployment, reducing deployment-related failures by ~30%. Monitored production pipelines, performed root-cause analysis for incidents, and resolved data quality and reliability issues to maintain service leve

Data Engineer at Digi QARC.

July 1, 2021 - July 1, 2023

Developed and maintained ETL workflows using SSIS, Python, and SQL to extract data from relational databases, flat files, and business applications. Wrote complex SQL queries, stored procedures, views, and functions to transform data and support reporting needs. Automated recurring extraction and file-processing/validation tasks with Python to reduce manual effort by ~25%. Performed data cleansing, mapping, and transformations for accurate operational reports, and implemented data-quality checks for nulls, duplicates, schema mismatches, and invalid formats to reduce recurring data errors by ~20%. Assisted with loading structured and semi-structured data into SQL Server and related storage environments. Monitored scheduled jobs, investigated failures, collaborated on resolutions with senior engineers, and supported validated datasets for Power BI dashboards along with testing, documentation, and production deployments.

Data Engineer at Digi QARC

July 1, 2021 - July 1, 2023

Developed and maintained ETL workflows using SSIS, Python, and SQL to extract data from relational databases, flat files, and business applications. Wrote complex SQL queries, stored procedures, views, and functions to transform data and support reporting requirements. Created Python scripts to automate recurring data extraction, file processing, and validation, reducing manual effort by ~25%. Performed data cleansing, mapping, and transformations to produce accurate datasets for operational reports and business analysis. Implemented data-quality checks to detect nulls, duplicates, schema mismatches, and invalid formats, reducing recurring data errors by ~20%. Loaded structured/semi-structured data into SQL Server and related data storage environments following ETL development standards. Monitored scheduled ETL jobs, investigated failures, and collaborated with senior engineers to resolve loading and transformation issues. Partnered with reporting teams to prepare validated datasets fo