Lead Azure DataBricks Data Engineer at EPAM Systems

Led enterprise-scale Azure DataBricks implementations for both batch and real-time platforms. Designed and optimized Medallion architecture pipelines using PySpark, Spark Structured Streaming, Delta Lake, Delta Live Tables, and Delta Sharing concepts. Built data ingestion and transformation frameworks using Databricks Auto Loader, file notification modes, schema evolution, and incremental processing. Implemented CDC/SCD Type 2 using Delta Lake Merge, alongside performance optimizations such as Optimize, Vacuum, Z-Ordering, partition tuning, and file compaction. Implemented governance with Unity Catalog and secure data access patterns. Automated deployments using Azure DevOps and Databricks Asset Bundles (DAB), and integrated AI solutions via Azure OpenAI and FastAPI-based services.

Senior Data Engineer at Ticketmaster

Worked on Azure DataBricks and Delta Lake pipelines for scalable ingestion and transformation. Delivered CDC and SCD frameworks using Delta Lake Merge operations. Developed and maintained streaming and batch ETL using Spark Structured Streaming and Delta Live Tables. Performed performance tuning and monitoring using Spark UI/executor logs, and contributed to CI/CD pipelines for reliable production releases. Supported production operations, troubleshooting, and ongoing optimization for data workflows.