Rahul Parthy

Senior Data/ Cloud Engineer (AWS) at Target Corporation

Led the modernization of legacy data systems into a cloud-native platform on AWS, designing petabyte-scale data lakes and data pipelines. Implemented real-time analytics for fraud detection, order tracking, and inventory forecasting, achieving sub-minute latency across millions of daily events. Migrated 200+ legacy Teradata ETL jobs to PySpark, Airflow, and dbt, eliminating on-prem dependencies and delivering significant cost savings. Built interactive dashboards for merchandising and operations, and established CI/CD pipelines with Terraform, Jenkins, GitHub Actions, and CodePipeline. Enforced end-to-end data governance and security (IAM, KMS, Lake Formation, CloudTrail), ensuring PCI-DSS and SOX compliance. Mentored a team of 7 engineers and partnered with stakeholders to align engineering with business goals.

Senior Data Engineer (Azure) at HSBC

Led cloud-based data platform on Azure Data Lake Gen2, Data Factory, Synapse Analytics, Databricks, Azure SQL, Event Hub, and Key Vault, consolidating 10TB+ of structured and semi-structured retail, sales, and supply chain data. Developed automated ETL pipelines with PySpark, Airflow, and dbt, integrating validation, testing frameworks, and lineage tracking. Designed and deployed interactive dashboards with Power BI and Tableau for real-time insights. Implemented CI/CD pipelines with Azure DevOps, GitHub Actions, and Terraform, automating testing, versioning, and environment promotion. Migrated legacy SQL Server systems to Snowflake on Azure, achieving a 3x performance boost. Established robust data governance and security with Key Vault, RBAC, masking, auditing, and PCI DSS compliance. Mentored a team of engineers and partnered with business units to develop ML-ready data pipelines.

Sr. Data Engineer at Dominion Energy

Led modernization of Dominion Energy’s data ecosystem through a hybrid AWS–Azure architecture, building real-time grid performance monitoring, predictive analytics, and enterprise data platforms to support smart grid, asset management, and regulatory reporting. Migrated legacy regulatory analytics to Snowflake with Spark-based processing, reducing query times by 40%. Automated ETL workflows with AWS Glue, integrating grid telemetry, customer usage, and asset data into Redshift/Snowflake. Delivered real-time dashboards for grid stability and demand forecasting; orchestrated pipelines with Airflow; integrated data from smart grids and ERP into ADLS for enterprise analytics. Implemented robust data governance and security practices.

Big Data Engineer at Chevrolet

Designed real-time data ingestion and processing pipelines across Azure and Databricks using Kafka and Azure Event Hubs. Enabled IoT device data ingestion from POS systems and in-store sensors for real-time store operations. Developed metadata-driven ingestion pipelines, automated ingestion with ADF and Delta Lake, and optimized analytics with Synapse. Implemented Terraform IaC, CI/CD, and governance with Azure Purview. Built modular ETL templates to improve reusability and reliability; deployed monitoring with Azure Monitor and Log Analytics; implemented CDC and incremental loading with ADF Mapping Data Flows.

Data Engineer at Pfizer

Designed on-premises big data pipelines on CDH using Sqoop, Flume, Spark, and Hive to process terabytes of data daily from lab, MES, and ERP systems. Implemented near real-time streaming with Kafka and Spark Streaming for anomaly detection in manufacturing. Built data quality frameworks with Oozie and Python, created analytics data marts with Hive/Impala, and automated batch workflows with Oozie/Azkaban. Integrated legacy Oracle/SQL Server via Sqoop for secure CDC into HDFS and prepared ML-ready datasets for predictive maintenance.

Data Analyst at State Farm Insurance

Collected, cleaned, and analyzed large datasets from policy, claims, and customer systems to identify trends and improve underwriting accuracy. Built interactive dashboards with Tableau and Excel; conducted statistical analyses and predictive modeling with SQL and Python. Automated routine data extraction and validation processes, reducing manual reporting time by 30% and improved data governance across business units.

Sr Data/ Cloud Engineer (AWS) at Target Corporation

Led cloud-native modernization of legacy data systems, designing a petabyte-scale data lakehouse on AWS (S3, Glue, Redshift, Lake Formation, Athena, EMR, Kinesis, Lambda). Designed and implemented real-time analytics pipelines leveraging Kafka, Spark Structured Streaming, EMR, Kinesis Data Streams, and Lambda to power fraud detection, order tracking, and inventory forecasting, reducing data latency from 15 minutes to under 20 seconds across 15M+ daily events. Migrated 200+ legacy Teradata ETL jobs to PySpark, Airflow, and dbt, eliminating on-prem dependencies and achieving annual cost savings of $1.8M. Implemented CI/CD pipelines with Terraform, Jenkins, GitHub Actions, and AWS CodePipeline; established end-to-end data governance with IAM, KMS encryption, Lake Formation policies, CloudTrail, CloudWatch Logs, and row-level security to ensure PCI-DSS and SOX compliance. Mentored a team of 7 engineers and partnered with stakeholders to define data strategy and architecture, delivering mul

Senior Data Engineer at Dominion Energy

Led modernization of Dominion Energy’s data ecosystem through a hybrid AWS–Azure architecture, building real-time grid performance monitoring, predictive analytics, and enterprise data platforms to enhance operational reliability and compliance. Designed event-driven pipelines with Kinesis, Step Functions, and Lambda to process smart meter, IoT sensor, and SCADA data with sub-second latency. Migrated legacy regulatory and compliance reporting infrastructure to Snowflake, optimizing energy distribution analytics and reducing query execution time by 40% via Spark; automated ETL workflows with AWS Glue, integrating grid telemetry, customer usage, and asset management data into Redshift/Snowflake. Delivered real-time dashboards on grid stability, load forecasting, and energy consumption using Kinesis Data Analytics, Glue, Redshift, DynamoDB, and EMR. Orchestrated pipelines with Apache Airflow; integrated Azure Data Factory pipelines into ADLS for enterprise analytics. Built data marts