Shiva Kumar

Designed and implemented scalable, secure AWS-based data platform consolidating healthcare data sources to enable advanced analytics, compliance reporting, and near real-time decision support. Built data lakes on S3 and integrated with Redshift for analytics; developed PySpark ETL on EMR for large-scale transformations; applied HIPAA and PIPEDA governance with encryption, access control, and data masking. Optimized pipelines for performance and cost using partitioning and AWS Cost Explorer; collaborated with data scientists to provision curated datasets for ML/AI; established automated monitoring and alerts with CloudWatch.

Building a scalable and secure AWS-based data platform to consolidate healthcare data sources, enabling analytics, compliance reporting, and near real-time insights for decision-making. Design and develop data pipelines using AWS Glue, Lambda, and Step Functions to ingest and process structured and unstructured healthcare data. Build and manage data lakes on Amazon S3 and integrate with Redshift for analytics and reporting. Implement PySpark-based ETL jobs on EMR for large-scale transformations and data cleansing. Apply data governance and compliance standards (HIPAA, PIPEDA) by enforcing encryption, access control, and data masking. Optimize pipeline performance and cost using AWS cost monitoring tools and partitioning strategies. Collaborate with data scientists to deliver curated datasets for ML and AI-driven use cases. Develop automated monitoring and alerting for pipeline health using CloudWatch.

Enterprise Data Lake Migration on Azure: Migrated legacy on-premise data warehouse to Azure Data Lake and Synapse Analytics, enabling enterprise-wide access to a single source of truth for financial and customer data. Developed ETL pipelines using Azure Data Factory to ingest financial and claims data from multiple sources (SQL Server, Oracle, flat files). Implemented PySpark and Databricks notebooks for data transformations, validations, and aggregations. Modeled data in Azure Synapse Analytics to support BI and financial reporting. Integrated ADLS Gen2 with governance standards. Automated workflows with ADF triggers/pipelines for real-time and batch ingestion. Collaborated with BI teams to deliver Power BI dashboards from Synapse datasets. Tuned SQL queries and optimized storage formats (Parquet, Delta Lake) for performance and cost.