Data Engineer at Recruit First

Worked with stakeholders to gather requirements, understand reporting needs, and design scalable enterprise data architectures. Developed and maintained ETL/ELT pipelines using Azure Data Factory and Databricks for ingestion into ADLS. Designed transformation workflows using Synapse Analytics, Spark notebooks, and Microsoft Fabric dataflows. Implemented Delta Lake medallion architecture (Bronze/Silver/Gold) and optimized transformations with Python, PySpark, and Spark SQL for large datasets. Integrated external systems via REST APIs with pagination, authentication, retry logic, and rate limiting. Delivered data governance using Azure Purview and Unity Catalog (metadata, lineage, access control, compliance). Built Power BI dashboards for KPI monitoring and business decision-making. Automated deployments using Azure DevOps CI/CD, ARM templates, and Databricks Asset Bundles. Provided production support with Azure Monitoring, resolving incidents and ensuring platform stability.

Data Engineer at Blue Dart

Collaborated with stakeholders to define data requirements and design a scalable cloud-based data warehouse for enterprise analytics and BI. Built automated ingestion pipelines using Azure Data Factory and Apache Airflow from relational databases, APIs, and cloud storage into centralized data lakes. Designed and optimized large-scale transformations using Databricks and PySpark. Implemented Synapse Analytics SQL pools (Dedicated and Serverless) to support high-performance reporting workloads. Integrated Cosmos DB, Azure SQL DB, and ADLS. Implemented Delta Lake for incremental processing, ACID transactions, and historical management. Created reusable Python/SQL scripts for validation and cleansing. Developed interactive Power BI dashboards. Automated CI/CD deployments using Azure DevOps and Git and monitored/optimized workflows for availability and performance.

Data Engineer at Razorpay

Designed real-time ingestion pipelines using Azure Data Factory and Databricks streaming for near real-time analytics and event-driven processing. Built event-driven workflows with Azure Functions and Logic Apps. Developed PySpark streaming applications for continuous ingestion, transformation, and processing of high-volume event streams with low latency. Led migration from on-premise systems to Azure Data Lake Storage and Snowflake. Built REST API-based ingestion frameworks for flexible third-party integrations. Implemented governance using Unity Catalog and Azure Purview. Created CI/CD pipelines using Azure DevOps. Built Power BI real-time dashboards and optimized Spark streaming jobs for scalability, fault tolerance, and production reliability. Provided monitoring and production support for streaming systems.

MI Analyst at Aurobindo Pharma

Analyzed legacy SAP systems with SAP teams and defined migration strategies to transition enterprise data to modern cloud platforms. Developed ETL pipelines using SSIS and Azure Data Factory to extract from SAP BW and SAP BO and load into ADLS. Built transformation workflows using Databricks and PySpark for high-performance processing of large SAP datasets. Designed SSAS OLAP models for multidimensional analysis and fast reporting queries. Integrated SAP systems with Azure SQL DB and ADLS. Implemented Delta Lake architecture for incremental loads and consistent historical tracking. Developed Power BI reports for validating migrated data. Automated deployment processes using Git and CI/CD and optimized SQL and transformation logic to improve migration performance. Provided production support and troubleshooting for migration-related issues.