Available to hire
Results-driven Senior Data Engineer with 10+ years designing, developing, and optimizing enterprise-scale data platforms and high-performance ETL/ELT pipelines across Microsoft Azure, AWS, and GCP. Expertise across Azure Data Factory, Azure Databricks, PySpark, Delta Lake, Synapse, ADLS Gen2, and cloud data warehouses like Snowflake/Redshift, delivering reliable, secure, and scalable analytics infrastructure.
Skilled in lakehouse/warehouse architecture (Medallion Bronze/Silver/Gold, Star/Snowflake schemas, SCD Type 1 & 2), implementing CDC, schema evolution, ACID transactions, and automated data quality frameworks. Experienced with orchestration (ADF, Databricks Workflows, Airflow, Step Functions), CI/CD (Azure DevOps, Git), governance (Key Vault, IAM/RBAC, Entra ID, Unity Catalog), and performance tuning for streaming and batch workloads.
Work Experience
Senior Azure Data Engineer at JPMorgan Chase
May 1, 2023 - PresentDesigned and implemented enterprise-scale cloud data engineering solutions using Azure Data Factory, Azure Databricks, PySpark, Delta Lake, ADLS Gen2, Azure SQL Database, and Azure Synapse Analytics for large-scale analytics and reporting. Architected scalable ETL/ELT pipelines using Medallion Architecture (Bronze/Silver/Gold) and built high-performance Spark SQL/PySpark transformations with optimization techniques (partitioning, caching, broadcast joins, AQE). Developed metadata-driven parameterized ADF pipelines and incremental processing frameworks using CDC/watermarking, Delta Lake MERGE upserts, and Auto Loader. Implemented Delta Lake ACID features and governance/optimization routines (schema evolution, time travel, OPTIMIZE, ZORDER, VACUUM). Built batch and near real-time pipelines using Databricks Structured Streaming with Azure Event Hubs. Delivered dimensional modeling solutions in Synapse (Star schema, Fact/Dimension tables, SCD Type 1 & 2) and performance-tuned SQL objects.
Senior Data Engineer at American Express
August 1, 2021 - April 1, 2023Designed and developed enterprise data pipelines using Azure Data Factory, Azure Databricks, PySpark, SQL Server, Snowflake, and ADLS Gen2 for healthcare analytics and reporting. Built scalable ETL/ELT workflows integrating relational sources, REST APIs, flat files, and cloud storage into centralized platforms. Developed reusable PySpark frameworks for cleansing, enrichment, and validation while improving processing efficiency. Implemented Delta Lake pipelines with incremental ingestion, schema evolution, ACID transactions, and MERGE-based upsert strategies. Delivered Snowflake warehouse solutions (databases/schemas/tables/views/stored procedures) with optimized SQL for reporting needs. Implemented CDC-based pipelines to capture source changes and propagate updates to analytics layers. Orchestrated with ADF using parameterization, dynamic expressions, looping, error handling, and dependency management. Created Python utilities for validation/reconciliation/exception handling and built
AWS Data Engineer at United Airlines
June 1, 2018 - July 1, 2021Designed and developed AWS-based data pipelines using Amazon S3, AWS Glue, Amazon EMR, Redshift, Python, PySpark, and Spark SQL for enterprise analytics and reporting. Built batch ingestion frameworks integrating Oracle, SQL Server, Teradata, REST APIs, and flat files into S3 data lake environments. Developed distributed transformations using PySpark/Spark SQL on EMR to process large-scale datasets. Created Glue ETL jobs and Glue Crawlers to transform data and maintain catalog metadata for downstream analytics. Designed and maintained Redshift warehouse objects (tables/views, distribution/sort strategies, SQL transformations). Implemented event-driven automation using AWS Lambda for validation, data movement, and lightweight processing. Orchestrated workflows with AWS Step Functions and Glue workflows/scheduling. Optimized Spark and Redshift workloads using partitioning, caching, broadcast joins, file-size/distribution/sort key tuning, and SQL query optimization. Built data quality and
Data Engineer at Cisco
June 1, 2016 - May 1, 2018Developed enterprise data pipelines using Python, SQL, PySpark, Apache Spark, Hadoop, Hive, and SSIS to integrate structured/semi-structured data from relational databases and APIs. Built large-scale batch processing solutions using HDFS, Hive, Sqoop, Oozie, Spark SQL, and PySpark across distributed environments. Worked with AWS services (S3/EMR/Redshift/EC2/Lambda/Glue) and Azure services (Azure SQL Database, ADF, HDInsight, Azure Functions) for data movement, orchestration, and migration. Supported GCP workloads using GCS, BigQuery, Dataproc, and Dataflow. Wrote complex SQL/T-SQL and transformations across multiple systems (SQL Server, Oracle, Teradata, Hive, Redshift, BigQuery). Designed dimensional models (Star schema, Snowflake schema, Fact/Dimension tables, surrogate keys, SCD Type 1 & 2) and improved performance via Spark partitioning/caching/broadcast joins, executor tuning, Hive partitioning, indexing, query optimization, and workload monitoring. Implemented automated data qua
ETL Developer at CSAA Insurance Group
May 1, 2014 - May 1, 2016Developed and maintained enterprise ETL solutions using SQL Server, SSIS, Oracle, Teradata, and T-SQL for insurance reporting systems. Designed SSIS packages for extracting, transforming, and loading policy, claims, and customer data into data warehouses. Built stored procedures, views, functions, and ETL scripts to automate reporting. Integrated data from relational databases and files (XML/CSV/Excel) and legacy systems into centralized reporting platforms. Built Hadoop batch workflows using HDFS, Hive, Sqoop, and Spark for growing analytics workloads; developed Hive tables and Spark-Scala transformations to improve performance. Implemented incremental and full-load ETL with logging, auditing, validation, and exception handling. Performed data profiling, cleansing, and reconciliation for production validation; supported SIT/UAT and production releases with issue resolution. Scheduled SQL Server Agent jobs and SSIS workflows with dependency management, restartability, notifications, an
Education
Master of Science in Data Science at United States of America
January 11, 2030 - August 19, 2026Qualifications
Industry Experience
Financial Services, Healthcare, Other
Hire a Data Engineer
We have the best data engineer experts on Twine. Hire a data engineer in Irvine today.