Anjana Jha

Architected secure, high-throughput streaming pipelines using Apache Kafka (on-prem) and Azure Event Hubs to ingest real-time meter and grid telemetry data (>150K events/sec) with minimal latency. Implemented end-to-end encryption and access controls (TLS, SASL, and Azure RBAC). Built a centralized data lake on Azure Blob Storage Gen2 and curated warehouse layers in Azure Synapse, enabling sub-second queries and <5-minute SLA for analytics. Optimized Databricks Spark ETL to process streaming data into analytics-ready tables, cutting batch times ~40% while handling >1 PB of historical data. Orchestrated pipelines with Airflow and Azure Data Factory; implemented data lineage; performed large-scale backfills on autoscaling Databricks clusters with spot instances, reducing compute costs by ~50%. Enforced governance with Azure AD RBAC and NSGs; codified infrastructure with Terraform and ARM templates; monitored end-to-end with Azure Monitor. Implemented serverless Azure Functions to automat

Designed end-to-end ETL pipelines using Azure Data Factory and Azure Databricks (PySpark/Scala) to ingest, cleanse, and standardize data into an Azure Data Lake. Optimized Spark jobs with partitioning and caching, reducing daily processing times by >40%. Modeled data warehouses in Azure Synapse Analytics, later integrating with Snowflake hosted on AWS for advanced analytics. Implemented secure data access policies across clouds (Azure RBAC and AWS IAM/KMS) with encryption in transit and at rest; automated policy audits via CI/CD. Established proactive monitoring with Azure Monitor and AWS CloudWatch; automated deployments with Azure DevOps, Terraform, and ARM; trained teams on Databricks and Snowflake, boosting release velocity by ~60%.

Designed ETL with AWS Glue, EMR, and PySpark/Scala transforming TBs of telecom data daily. Built real-time streaming pipelines with Kafka + Spark Streaming on EMR for low-latency insights. Designed data lakes on S3 with partitioning; enabled efficient querying via Athena and Redshift Spectrum; modeled enterprise data warehouses in Redshift. Implemented AWS Step Functions and Data Pipeline for orchestration with robust error handling. Monitored Spark jobs, crawlers, and query performance with CloudWatch; delivered Power BI dashboards backed by Redshift and Athena. Authored runbooks and best practices; achieved ~50% ETL runtime reduction and supported telecom compliance.