End-to-end data warehousing work: built pipelines in Azure and Databricks; implemented Unity Catalog for centralized governance and access control; migrated pipelines from Azure Data Factory to Databricks; optimized PySpark jobs, reducing processing time by 40%; supported production incidents and data scientists.

Data Engineer Consultant at Tata Consultancy Services Bank of Montreal

May 1, 2024 - January 31, 2025

Designed data models and built ingestion pipelines using AWS Glue, Lambda, and S3 with PySpark and Python; developed end-to-end ingestion and transformation workflow (S3 to Redshift L1-L3); implemented data quality checks and SCD2; leveraged Azure and AWS lake services for medallion architecture; improved processing with Parquet and Airflow-based orchestration.

Big Data Engineer at Canadian Imperial Bank of Commerce

July 1, 2022 - May 31, 2024

Built data models and pipelines using Azure Data Factory and Databricks (PySpark/SQL); delivered delta tables and end-to-end data ingestion from source to warehouse; implemented incremental loads and SCD2; supported analytics via Azure Synapse, AWS Athena; visualized real-time data with Power BI.

Associate Lead at NEC Software Solutions India Limited

October 1, 2020 - June 30, 2022

Ingested data from files and databases via Azure Data Factory and Databricks; created Hive tables in Spark and Python; managed data lake environments; implemented end-to-end capital markets flows for Broadridge, Morningstar, Wells Fargo; implemented SCD Type II; improved performance by PySpark optimizations (40%).

Technical Lead I at CitiusTech HealthCare Pvt Ltd

November 1, 2018 - September 30, 2020

Validated healthcare savings/spends calculations and patient extracts in Apache Spark; designed Hive warehouse tables on HDFS; participated in data modeling with BA; built scalable data solutions using Hadoop, Python, and PySpark; applied Spark optimizations reducing processing time by 40%.

Sr. Software Engineer at Mastek Limited

July 1, 2017 - October 31, 2018

Extracted and verified data from source files; performed unit testing; loaded data from CSV to staging and base tables via ODI; transformed and aggregated data; tested reports in OBIEE; implemented PySpark/Python logic for data processing.

Project Engineer at Wipro Technologies

March 1, 2013 - July 31, 2017

Developed Python-based logic to meet business requirements; wrote SQL scripts for data transformations; updated release notes and deployment docs; worked on Citi Bank capital markets data project; contributed to unit and integration testing.