Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

Senior Data Engineer with 7+ years of experience designing and building scalable config-driven ETL/ELT pipelines and enterprise data platforms on Microsoft Azure. Strong expertise in SQL, PySpark, dimensional modeling (star/snowflake), query optimization, and data quality enforcement across Bronze/Silver/Gold medallion layers. Proven record of improving onboarding time and pipeline throughput by building governed, production-ready solutions using Databricks, Delta Live Tables, ADF/Synapse, Unity Catalog and Microsoft Purview. Adept at cross-functional collaboration and mentoring engineers to deliver reliable data products with CI/CD automation and strong governance.…Senior Data Engineer with 7+ years of experience designing and building scalable config-driven ETL/ELT pipelines and enterprise data platforms on Microsoft Azure. Strong expertise in SQL, PySpark, dimensional modeling (star/snowflake), query optimization, and data quality enforcement across Bronze/Silver/Gold medallion layers. Proven record of improving onboarding time and pipeline throughput by building governed, production-ready solutions using Databricks, Delta Live Tables, ADF/Synapse, Unity Catalog and Microsoft Purview. Adept at cross-functional collaboration and mentoring engineers to deliver reliable data products with CI/CD automation and strong governance.

SHOBIYA RANI





Senior Data Engineer with 7+ years of experience designing and building scalable config-driven ETL/ELT pipelines and enterprise data platforms on Microsoft Azure. Strong expertise in SQL, PySpark, dimensional modeling (star/snowflake), query optimization, and data quality enforcement across Bronze/Silver/Gold medallion layers. Proven record of improving onboarding time and pipeline throughput by building governed, production-ready solutions using Databricks, Delta Live Tables, ADF/Synapse, Unity Catalog and Microsoft Purview. Adept at cross-functional collaboration and mentoring engineers to deliver reliable data products with CI/CD automation and strong governance.…Senior Data Engineer with 7+ years of experience designing and building scalable config-driven ETL/ELT pipelines and enterprise data platforms on Microsoft Azure. Strong expertise in SQL, PySpark, dimensional modeling (star/snowflake), query optimization, and data quality enforcement across Bronze/Silver/Gold medallion layers. Proven record of improving onboarding time and pipeline throughput by building governed, production-ready solutions using Databricks, Delta Live Tables, ADF/Synapse, Unity Catalog and Microsoft Purview. Adept at cross-functional collaboration and mentoring engineers to deliver reliable data products with CI/CD automation and strong governance.

Available to hire

Senior Data Engineer with 7+ years of experience designing and building scalable config-driven ETL/ELT pipelines and enterprise data platforms on Microsoft Azure. Strong expertise in SQL, PySpark, dimensional modeling (star/snowflake), query optimization, and data quality enforcement across Bronze/Silver/Gold medallion layers.

Proven record of improving onboarding time and pipeline throughput by building governed, production-ready solutions using Databricks, Delta Live Tables, ADF/Synapse, Unity Catalog and Microsoft Purview. Adept at cross-functional collaboration and mentoring engineers to deliver reliable data products with CI/CD automation and strong governance.

Skills

Experience Level

Expert

Expert

Expert

Expert

Expert

Expert

Intermediate

Intermediate

Intermediate

Beginner

Work Experience

Senior Data Engineer at Cognizant

March 1, 2025 - Present

Designed and built a scalable, config-driven ETL ingestion framework using JSON configurations and schema DDLs, reducing custom pipeline effort for new source onboarding by 50%+. Implemented PySpark transformations including casting, null handling, deduplication via primary-key checks, and referential integrity validation across Bronze/Silver/Gold layers. Tuned SQL and PySpark performance through execution plan analysis, partitioning strategies, and join optimizations for high-volume datasets. Modeled star-schema dimensional structures at the Gold layer for OLAP reporting and data marts. Enforced data quality using Delta Live Tables expectations (quarantine/flag/drop) for null, deduplication, and referential integrity violations. Handled Auto Loader _RESCUED_DATA parsing/normalization for schema evolution and table-level bespoke transformations. Established governance with Unity Catalog and Microsoft Purview (cataloguing, lineage, ownership) and automated CI/CD deployments across DEV/S

Technology Lead — Data Engineer at Infosys

October 1, 2024 - February 28, 2025

Led ADF pipelines for large-scale batch and API-based marketing data ingestion and transformation, transferring high-volume datasets to SQL databases daily. Directed migration of ETL/ELT logic from Databricks notebooks to Azure Synapse, applying SQL performance tuning and execution plan analysis to maintain data integrity and performance post-migration. Resolved Azure tenant security (S360) issues and executed Databricks FY rollover path migration with zero data loss while preserving governance standards. Provided production support and root-cause analysis for ICM pipeline incidents to ensure high availability of critical data solutions.

Technology Analyst — Data Engineer at Infosys

January 1, 2023 - October 1, 2024

Integrated data from REST APIs using PySpark transformations in Azure Databricks and orchestrated ETL/ELT pipelines using ADF; developed Azure Function Apps for batch processing of large datasets. Optimized Azure Synapse dedicated SQL pool performance using execution plan analysis, query rewriting, indexing improvements, and dimensional modeling fixes to reduce response times. Migrated classic Azure DevOps pipelines to YAML CI/CD to improve release reliability and version-controlled deployment governance across environments. Maintained a BI platform end-to-end for data acquisition to reporting, including Power BI dashboards and scorecards, and resolved data quality issues using SQL integrity checks and dimensional modeling corrections.

Analyst — Data Engineer at Satven

October 1, 2018 - December 31, 2022

Processed complex datasets using advanced SQL querying and statistical analysis (correlation, hypothesis modeling) and supported reporting/visualization using Power BI. Translated business requirements into technical data solutions in collaboration with manufacturing and customer data teams to identify improvement strategies. Developed clustering and prediction analytics for vehicle components using CART and Random Forest approaches for crushing behavior under NCAP test conditions. Built a linear regression model to predict optimal CAPEX amounts for vendor components to maximize profitability across business scenarios.