Senior Data Engineer at Accenture

Designed and implemented a scalable, cloud-native data ingestion framework on AWS Glue consolidating 15+ data sources into a single source of truth on Amazon S3. Orchestrated and automated complex data transformations using PySpark, processing over 2 TB of daily data to support real-time analytics and ML models. Led migration of legacy on-premise databases to Amazon Redshift, achieving 30% warehouse cost reduction and 60% improvement in query performance. Built data quality frameworks using AWS Glue DataBrew and custom scripts, automating anomaly detection and improving reliability for stakeholders by 95%. Engineered a near real-time streaming solution using Amazon Kinesis with latency under 5 seconds. Mentored a team of 4 data engineers in an Agile environment, implementing CI/CD with AWS CodePipeline/CodeBuild, accelerating project delivery by 20%. Collaborated with cross-functional teams to define data models and SLAs.

Data Engineer at Tata1mg

Developed scalable ETL/ELT pipelines automating data extraction from 10+ systems into a centralized Amazon S3 data lake. Designed and implemented star-schema data models in Amazon Redshift, reducing report generation time from hours to minutes. Built near-real-time inventory monitoring using AWS Lambda and event-driven processes handling 500,000+ daily events, reducing stock-outs by 25%. Optimized SQL queries to support ad-hoc analysis; identified key customer segments improving marketing campaign conversions by 15%. Automated data validation and reporting tasks using Python, saving ~20 person-hours per week. Managed and monitored pipelines with MWAA ensuring 99.8% uptime. Implemented data partitioning and distribution in Redshift improving analytical query performance by 35%.