Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

Azure Data Engineer with 4+ years of experience designing and delivering enterprise-scale data platforms using Azure Data Factory, Databricks, Synapse Analytics, and Microsoft Fabric. Proven expertise in building Delta Lakehouse architectures, metadata-driven ETL frameworks, and scalable PySpark pipelines across financial services and retail clients. Strong background in data governance, implementing Unity Catalog, RBAC, and GDPR-compliant solutions. DP-203 certified, with a track record of optimizing data pipelines, enabling real-time analytics, and supporting business-critical decision-making.…Azure Data Engineer with 4+ years of experience designing and delivering enterprise-scale data platforms using Azure Data Factory, Databricks, Synapse Analytics, and Microsoft Fabric. Proven expertise in building Delta Lakehouse architectures, metadata-driven ETL frameworks, and scalable PySpark pipelines across financial services and retail clients. Strong background in data governance, implementing Unity Catalog, RBAC, and GDPR-compliant solutions. DP-203 certified, with a track record of optimizing data pipelines, enabling real-time analytics, and supporting business-critical decision-making.

Subhashini P

Data Engineer, Web Developer, Programmer, +2





Azure Data Engineer with 4+ years of experience designing and delivering enterprise-scale data platforms using Azure Data Factory, Databricks, Synapse Analytics, and Microsoft Fabric. Proven expertise in building Delta Lakehouse architectures, metadata-driven ETL frameworks, and scalable PySpark pipelines across financial services and retail clients. Strong background in data governance, implementing Unity Catalog, RBAC, and GDPR-compliant solutions. DP-203 certified, with a track record of optimizing data pipelines, enabling real-time analytics, and supporting business-critical decision-making.…Azure Data Engineer with 4+ years of experience designing and delivering enterprise-scale data platforms using Azure Data Factory, Databricks, Synapse Analytics, and Microsoft Fabric. Proven expertise in building Delta Lakehouse architectures, metadata-driven ETL frameworks, and scalable PySpark pipelines across financial services and retail clients. Strong background in data governance, implementing Unity Catalog, RBAC, and GDPR-compliant solutions. DP-203 certified, with a track record of optimizing data pipelines, enabling real-time analytics, and supporting business-critical decision-making.

Available to hire

Azure Data Engineer with 4+ years of experience designing and delivering enterprise-scale data platforms using Azure Data Factory, Databricks, Synapse Analytics, and Microsoft Fabric. Proven expertise in building Delta Lakehouse architectures, metadata-driven ETL frameworks, and scalable PySpark pipelines across financial services and retail clients.

Strong background in data governance, implementing Unity Catalog, RBAC, and GDPR-compliant solutions. DP-203 certified, with a track record of optimizing data pipelines, enabling real-time analytics, and supporting business-critical decision-making.

Skills

Experience Level

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Intermediate

Work Experience

Data Engineer at Innovative Logical Solutions

August 1, 2024 - Present

Architected metadata-driven Azure Data Factory pipelines on Microsoft Fabric integrating structured and semi-structured sources, enabling end-to-end pipeline traceability. Designed Databricks + Unity Catalog framework for bronze/silver/gold processing with lineage tracking to support governance and GDPR compliance. Led requirements workshops translating business KPIs into analytical models and delivering same-day insights. Built and optimized PySpark transformations using caching, parallel processing, and partition pruning to reduce runtime. Implemented Unity Catalog and RBAC across three environments with audit trail coverage for sensitive retail datasets. Created Power BI dashboards integrated with Azure SQL Database and Dataverse for real-time KPI monitoring. Deployed Microsoft Fabric pipelines for unified ingestion across hybrid/multi-cloud environments and automated build/test/release via Azure DevOps CI/CD.

Data Engineer at VA Solutions

August 1, 2023 - May 31, 2024

Migrated legacy SSIS/SSAS workflows to ADF and Databricks, improving SAP source integration throughput and reducing maintenance effort. Implemented a hybrid lakehouse on ADLS, Synapse, and Cosmos DB for near real-time risk analytics from high-volume daily financial events. Increased ingestion throughput and data consistency using PySpark broadcast joins, checkpointing, and adaptive partition pruning. Built Python and Airflow monitoring/alerting to reduce mean time to detect and resolve pipeline failures. Delivered Power BI dashboards integrating SAP BW and Azure SQL for real-time drilldown analytics. Implemented IAM/RBAC policies for sensitive datasets and maintained compliance with enterprise security standards.

Data Engineer at Commerzbank

November 1, 2022 - July 31, 2023

Partnered with stakeholders to design Azure-based warehouse solutions using Databricks, Unity Catalog, and Synapse Analytics. Built metadata-driven Delta Lake + ADF pipelines to improve modularity and schema adaptability in changing environments. Designed a modern lakehouse on ADLS and Delta Lake supporting both batch and real-time workloads under unified governance. Optimized ETL using PySpark caching, checkpointing, and partition tuning for scalability on complex enterprise workloads. Implemented IAM/RBAC for secure access to sensitive datasets while keeping developer/analyst flexibility. Automated data quality frameworks and ETL runs with dynamic exception tracking and real-time notifications. Developed interactive Power BI dashboards integrating SAP BW and Azure SQL for drilldowns and trend analytics.

BI Developer at State Auto Insurance

May 1, 2022 - October 31, 2022

Gathered BI and reporting requirements and translated KPIs into data warehouse models. Developed optimized ETL pipelines using SSIS and Alteryx integrating SAP and SQL sources for accurate and timely analytics feeds. Created Power BI and Tableau dashboards with drill-down visualizations and automated refresh. Built SSAS cubes with MDX calculations to improve multidimensional query performance. Optimized calculations using DAX and SQL to reduce dashboard latency. Automated scheduling, alerts, and audit processes via Power BI Service and SQL Server Agent. Maintained data governance using access controls, metadata documentation, and user-level permissions.