I'm a results-oriented Lead Data & Cloud Engineer with 12 years of IT experience, including 11+ years designing and deploying full-scale data pipelines and cloud-native architectures across AWS, GCP, Azure, and on-prem Hadoop ecosystems. I excel at building scalable, secure, and high-throughput solutions for real-time analytics and data modeling, and I lead cross-functional teams with a strong focus on governance and automated pipeline orchestration. I've architected multi-cloud, AI-powered streaming platforms using Kafka, Flink, PySpark, and Beam; developed MLOps-friendly data products and LLM-powered solutions, and steered migrations from legacy on-prem systems to cloud-native platforms to optimize cost and performance.

Darsh Premchandani

I'm a results-oriented Lead Data & Cloud Engineer with 12 years of IT experience, including 11+ years designing and deploying full-scale data pipelines and cloud-native architectures across AWS, GCP, Azure, and on-prem Hadoop ecosystems. I excel at building scalable, secure, and high-throughput solutions for real-time analytics and data modeling, and I lead cross-functional teams with a strong focus on governance and automated pipeline orchestration. I've architected multi-cloud, AI-powered streaming platforms using Kafka, Flink, PySpark, and Beam; developed MLOps-friendly data products and LLM-powered solutions, and steered migrations from legacy on-prem systems to cloud-native platforms to optimize cost and performance.

Available to hire

I’m a results-oriented Lead Data & Cloud Engineer with 12 years of IT experience, including 11+ years designing and deploying full-scale data pipelines and cloud-native architectures across AWS, GCP, Azure, and on-prem Hadoop ecosystems. I excel at building scalable, secure, and high-throughput solutions for real-time analytics and data modeling, and I lead cross-functional teams with a strong focus on governance and automated pipeline orchestration.

I’ve architected multi-cloud, AI-powered streaming platforms using Kafka, Flink, PySpark, and Beam; developed MLOps-friendly data products and LLM-powered solutions, and steered migrations from legacy on-prem systems to cloud-native platforms to optimize cost and performance.

See more

Experience Level

Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
See more

Language

English
Fluent

Work Experience

Lead Data Engineer at American Electric Power
December 1, 2024 - Present
Designed and implemented end-to-end data pipelines for ingestion of smart meter, SCADA, and outage management data into cloud analytics platforms. Built streaming pipelines with Kinesis and Spark Structured Streaming for near real-time energy consumption and grid telemetry; implemented CDC via AWS DMS and Spark jobs to sync changes to Redshift and S3-based data lakes. Implemented SCD Type 1/2, cross-region Kafka replication, and serverless components to automate workflows for fraud detection, regulatory reporting, and executive dashboards.
Lead Data & Cloud Engineer at Citigroup
January 1, 2023 - November 1, 2024
Migrated legacy ETL and Airflow workloads to Cloud Composer; built real-time ingestion pipelines with Kafka and Spark Streaming; leveraged GCP services (BigQuery, DataProc, Pub/Sub) and Dataprep/ADF for Bronze-Silver-Gold data pipelines, materialized views, and partitioning to support regulatory reporting and dashboards.
Senior Data & Cloud Engineer at Nordstrom
November 1, 2020 - December 1, 2022
Designed real-time transaction and IoT-driven data platform using Kafka and Azure Event Hubs; ingested data from in-store devices via Azure IoT Hub; built metadata-driven ingestion with YAML/JSON configs; automated end-to-end ingestion with ADF and SQL Server; optimized performance in Azure Synapse Analytics and Delta Lake for retail analytics and BI dashboards.
Cloud Data Engineer at Gilead Sciences
August 1, 2017 - October 1, 2020
Developed real-time adverse event monitoring using AWS Kinesis, Lambda, and Databricks; migrated regulatory reporting infrastructure to Snowflake with Spark-based transformations; automated ETL with AWS Glue; established HIPAA-compliant infrastructure with secure data pipelines and governance.
Hadoop Developer at Lucid Motors
September 1, 2014 - July 1, 2017
Built end-to-end Big Data solutions for automotive domain, integrating production, inventory, and quality data into a unified data lake; implemented real-time streaming with Kafka; optimized Hive/Impala-based analytics, Oozie-based workflows, and Dockerized components with CI/CD pipelines.
Software Developer at Oracle
January 1, 2013 - August 1, 2014
Supported enterprise financial applications; provided L2/L3 production support; designed and optimized backend ETL workflows using Oracle DB/SQL Server/SSIS; developed REST microservices and Agile-delivered features; built data ingestion workflows for Hadoop ecosystem.

Education

Master of Business Administration (Business Analytics) at Illinois Institute of Technology, Chicago
January 11, 2030 - January 13, 2026
Integrated Program in Business Analytics at Stern School of Business, New York University (NYU), NY
January 11, 2030 - January 13, 2026
Bachelor of Commerce (Finance and Accounting) at Stern School of Business, New York University (NYU), NY
January 11, 2030 - January 13, 2026
Master of Business Administration (Business Analytics) at Illinois Institute of Technology, Chicago
January 11, 2030 - January 13, 2026
Integrated Program in Business Analytics at Stern School of Business, New York University (NYU), NY
January 11, 2030 - January 13, 2026
Bachelor of Commerce (Finance and Accounting) at Stern School of Business, New York University (NYU), NY
January 11, 2030 - January 13, 2026
Master of Business Administration (Business Analytics) at Illinois Institute of Technology
January 11, 2030 - January 13, 2026
Integrated Program in Business Analytics at Stern School of Business, New York University (NYU)
January 11, 2030 - January 13, 2026
Bachelor of Commerce (Finance and Accounting) at Stern School of Business, New York University (NYU)
January 11, 2030 - January 13, 2026

Qualifications

Microsoft Certified: Azure Fundamentals (AZ-900)
January 11, 2030 - January 13, 2026
Basic Hadoop Certification
January 11, 2030 - January 13, 2026
Microsoft Certified: Azure Fundamentals (AZ-900)
January 11, 2030 - January 13, 2026
Basic Hadoop Certification
January 11, 2030 - January 13, 2026
Microsoft Certified: Azure Fundamentals (AZ-900)
January 11, 2030 - January 13, 2026
Basic Hadoop Certification
January 11, 2030 - January 13, 2026

Industry Experience

Energy & Utilities, Financial Services, Healthcare, Manufacturing, Retail, Software & Internet, Professional Services, Life Sciences, Media & Entertainment, Other