Lead Data Engineer at DevLabs Alliance Pvt Ltd

Led the design and operations of ~50 data pipelines ingesting from multiple sources and loading into Azure Synapse for Power BI reporting. Implemented Azure Data Factory pipelines for ingestion and processing, built ETL workflows from SQL Server, Oracle, Sybase, DB2, and SFTP. Developed PySpark notebooks in Databricks for data validation and cleansing. Implemented SCD1 and SCD2, exception handling, and logging; managed linked services, datasets, triggers, and ADLS Gen2 storage. Secured data with Key Vault and configured automated notifications via Logic Apps. Monitored production pipelines and tuned performance; managed CI/CD with Azure DevOps. Created Synapse views to support Power BI reporting.

Lead Data Engineer at BencKart

Orchestrated end-to-end ingestion and validation of data from multiple systems for McKesson, using ADF and Databricks, with data stored in Snowflake. Developed Snowflake stored procedures and ingestion pipelines. Implemented email notifications via Logic Apps. Built Databricks notebooks for orchestration and data validation. Created linked services and managed Key Vault secrets.

Senior Data Engineer at Accenture

Designed and executed AWS data pipelines using S3 and Glue to extract and transform data. Implemented SCD2 and delivered gold layer data for QlikView reporting. Responsibilities included Python/SQL scripting, storage optimization with RDS/DynamoDB, and building data warehouses in Redshift/Athena. Implemented IAM security and CloudWatch monitoring; automated workflows with Lambda and Step Functions.

Senior Data Engineer at Infosys

Processed high-volume healthcare data, implemented SCD1 and SCD2 in Delta Lake, and loaded into a Data Warehouse for analytics. Created and managed Databricks clusters; developed Spark ETL workflows; established JDBC connections to Synapse. Worked with CSV, Parquet, JSON, and XML; integrated Key Vault and security scopes; optimized Spark SQL queries; automated ETL with Databricks Workflows.