Ganesh Kumar

Architected and delivered real-time inventory and streaming ETL systems using Kafka/Kafka Streams and Azure Data Factory, supporting sub-second latency across large retail operations. Built multi-tier lakehouse pipelines (Bronze/Silver/Gold) with automated schema evolution and data quality validation, and implemented Delta Lake on Azure for ACID transactions and time-travel queries. Designed a star schema warehouse in Azure Synapse to improve query performance for executive reporting and built RESTful APIs using Azure Functions for real-time inventory delivery to mobile and partners. Developed fraud detection pipelines using Kafka and Spark Streaming with ML-driven anomaly alerts, integrated with PCI-DSS-aligned security. Implemented batch ingestion pipelines for nightly sales processing and strengthened CI/CD with Azure DevOps/GitHub, including PySpark testing and data-quality gates before releases. Also contributed to AWS and GCP lake and streaming implementations (S3/Glue/EMR/Lake F

Designed scalable big data processing workflows on AWS, Azure, and GCP for retail and operational analytics, including batch and real-time event ingestion from POS, order management, inventory, and clickstream sources. Implemented Azure Data Lake Storage (ADLS Gen2) with RBAC, Key Vault encryption, lifecycle policies, and automated Azure Data Factory pipelines with incremental loads, retries, and error handling. Built transformations and analytics using Databricks/Spark and Azure Synapse with partitioning, distribution strategies, and optimized SQL. Delivered streaming solutions using Event Hubs/Stream Analytics/Azure Functions and developed monitoring dashboards for operational and data-quality metrics. Architected data warehouses in Redshift with dimensional modeling and SCD strategies, and implemented resilient streaming pipelines with Kinesis/Kafka Connect. Created automated ETL workflows and reconciliation/validation frameworks using Glue, Talend, Sqoop, Informatica, and schedulin

Built and operationalized ETL pipelines on AWS and Azure and GCP across logistics/shipment domains, including fault-tolerant ingestion from multiple sources (JSON/XML, relational systems, MongoDB). Configured secure S3 storage with encryption, versioning, lifecycle policies, and access logging. Developed AWS Glue pipelines and streaming ingestion using MSK/Kinesis/Lambda/S3 with low-latency validation, error handling, and event-driven workflows. Implemented Redshift analytics with partitioning/distribution key optimization for improved historical query performance. Created Talend jobs with quarantine queues for malformed records and used MongoDB for metadata/configurations. Delivered Azure Data Factory pipelines and near-real-time shipment tracking using Event Hubs/Stream Analytics/Databricks, integrating with Azure Synapse for reporting. Automated deployments using Terraform/ARM and implemented monitoring with CloudWatch dashboards and alerts. Also built batch/streaming pipelines on G