Architected an enterprise lakehouse platform on Azure Data Lake Gen2, Databricks, and Synapse, implementing bronze, silver, and gold data layers and partnering with BI teams for standardized reporting. Developed Python and PySpark ETL processes using Azure Data Factory with Databricks to ingest structured and semi-structured data from SQL Server and PostgreSQL into ADLS Gen2, processing 1 TB+ daily. Orchestrated reliable event-driven pipelines with Azure Event Hubs and Databricks Structured Streaming to ingest financial transactions and customer activity data, reducing latency by 70% for risk and regulatory dashboards. Implemented data quality checks in Databricks with PySpark and SQL validations to ensure reliability of governed datasets. Provisioned dedicated Synapse SQL pools and serverless endpoints, creating materialized views, partitioned tables, and workload isolation policies, boosting query throughput by 35% while controlling compute costs. Partnered with data scientists to pr

Data Engineer at Cigna

July 1, 2021 - July 1, 2023

Engineered a healthcare data platform on AWS S3, Glue, and Snowflake, consolidating claims, EHR, provider, and payer data into a unified data lakehouse with staging, curated, and aggregated layers, streamlining query execution by 45% and enabling enterprise-wide analytics. Developed scalable ETL pipelines using Python, PySpark, AWS Lambda, AWS Glue, and EMR to process 10 million healthcare records from JSON, CSV, and Parquet, consolidating data from DynamoDB and SQL Server to S3. Executed Change Data Capture (CDC) tasks using AWS DMS to replicate changes from SQL Server and DynamoDB into S3 and Snowflake, improving data freshness in near real-time and minimizing batch load dependencies. Built real-time streaming applications with Apache Kafka and Confluent Schema Registry to ingest high-volume claims, EHR, and API feeds, boosting data freshness by 60% and accelerating analytics insights for regulatory teams. Automated data quality checks with Great Expectations, enforcing schema valida

Data Engineer at Spirit Airlines

January 1, 2019 - July 1, 2021

Built a scalable data lake architecture on AWS S3, integrating Glue, Redshift, and Athena to handle 1 TB structured and semi-structured data daily, boosting reliable data availability by 40%. Streamlined ETL workloads using Spark-Scala on AWS EMR, processing complex data from legacy Oracle systems and external sources in JSON, CSV, and Parquet formats into Redshift and S3 for Athena-based analysis. Orchestrated and optimized batch workflows using Apache Airflow DAGs to efficiently schedule, monitor, and manage critical enterprise data jobs, elevating overall reliability and decreasing manual interventions by 60%. Performed data modeling and optimized Amazon Redshift schemas using star and snowflake designs for analytical workloads, accelerating query efficiency and enabling effective partition pruning to reduce scan costs. Developed enterprise-grade real-time streaming dataflows using AWS Kinesis and Lambda to ingest high-volume transactional data into Redshift, lowering latency by up