Suvarna Chandrika Reddy

Managed end-to-end ETL pipelines on GCP, coordinating data movement from multiple sources into Cloud Storage and loading into BigQuery for enterprise analytics. Built scalable batch and streaming ingestion workflows using Python, Dataflow, Pub/Sub, and Compute Engine to streamline data imports into BigQuery warehouses; migrated legacy cron jobs to Airflow for improved scheduling and monitoring. Designed staging layers in Cloud Storage, performed data validation and transformations with Dataflow and Dataproc, and implemented IAM-based access control and query optimization to enhance performance and cost efficiency. Led real-time data processing using Pub/Sub and Spark on GKE; developed AI/ML data pipelines with Vertex AI monitoring and model-tracking.

Migrated on-prem Hadoop to GCP; built scalable batch and streaming data pipelines using Python, PySpark, Hive SQL, and Presto; implemented configurable data delivery pipelines for customer-facing stores; built dashboards in Tableau; optimized data ingest from relational/non-relational sources; built stored procedures in MS SQL for enterprise reporting.

End-to-end ETL and big data pipelines using AWS Glue/EMR; real-time streaming with Kinesis and Kafka; data preprocessing for ML with Spark/EMR; migrated production infrastructure to AWS using serverless patterns (Lambda, Kinesis); CI/CD with CodePipeline; data warehousing with Redshift; integration with SageMaker for ML models; data quality frameworks and validation.