Nandini Gunda

Led migration of 50+ legacy ETL jobs to AWS MWAA, reducing job failures by 45% and cutting operational overhead by over $100K annually. Built real-time streaming pipelines with Apache Kafka and Spark Structured Streaming to monitor SaaS metrics, achieving sub-minute latency. Optimized Spark batch pipelines processing 3TB+ daily, reducing transformation runtimes by 40% and lowering infra costs. Designed scalable data models in AWS Redshift for SaaS KPIs (ARR, CAC, CLTV) and implemented reusable ingestion/transformation frameworks. Introduced Great Expectations-based automated validation and anomaly detection, improving event-level data quality. Migrated S3 Parquet workflows to Apache Hudi for incremental upserts, boosting pipeline efficiency and maintainability. Implemented CI/CD with GitLab/Jenkins/Terraform/Helm and integrated Apache Atlas for lineage. Participated in cross-cloud PoCs comparing Spark performance on AWS EMR vs Azure Databricks and supported Delta Lake workflows.

Designed scalable, fault-tolerant data pipelines and ETL/ELT workflows for a SaaS environment, enabling batch and real-time analytics. Orchestrated Airflow DAGs with monitoring and alerting to ensure SLA adherence across environments. Delivered end-to-end streaming and batch solutions on Azure by ingesting high-volume Kafka data into Data Lake, Synapse Analytics, and Blob Storage. Optimized Azure Databricks pipelines processing 4TB+ daily, reducing latency by 35% and saving over $10K/month in compute costs. Built real-time event-stream processing with Kafka Streams and Spark Streaming on Azure Event Hubs, enabling near real-time user signals. Developed self-service datasets and Power BI dashboards for 100+ stakeholders, cutting insight delivery time by 50%. Led migration of 40+ batch jobs to stream-first architecture with zero data loss. Implemented CI/CD with Git, Terraform, Docker, and Azure DevOps. Mentored junior engineers in Python/Scala and Azure-native architecture.

Designed and delivered scalable cloud-native pipelines on GCP (BigQuery, Cloud Storage, Composer), enabling global strategy and product teams to make timely, data-driven decisions. Built end-to-end ETL/ELT workflows with Cloud Composer, orchestrating Python and Spark jobs to process terabytes of data with high reliability. Architected real-time and batch data processing pipelines using Kafka, Spark Streaming, Scala, and Python, ingesting data into BigQuery and GCS for analytics. Refactored and optimized distributed Spark applications, boosting throughput by 45% and reducing Dataproc costs by $15K per quarter. Created self-service datasets and data libraries, reducing dependency on engineering teams. Integrated diverse SaaS sources (Salesforce, Stripe, Mixpanel) with internal logs into unified BigQuery datasets. Migrated legacy batch ETL to real-time event-driven architectures on GCP, significantly reducing latency and improving observability. Implemented Great Expectations for automate