Mahnoor Zahid
I’m Mahnoor Zahid, a data engineering consultant with over 4 years of hands-on experience building scalable ETL pipelines on AWS and Databricks.
About
I’m Mahnoor Zahid, a data engineering consultant with over 4 years of hands-on experience building scalable ETL pipelines on AWS and Databricks. I enjoy turning complex data into reliable, secure analytics platforms and collaborating with cross-functional teams to deliver cloud-native solutions.
From migrating HDP to CDP on AWS to designing event-driven real-time data lakes and cross-region ingestion frameworks, I focus on governance, security, and cost optimization while delivering robust data platforms.
- 5 years' experience
- Joined May 2024
Industry experience
- Computers & Electronics
- Software & Internet
- Professional Services
Work experience· 4 roles
Data Engineering (Feb 2023 – Present). Led HDP → CDP migration on AWS; designed configuration-driven routing for ingestion and target schemas; built idempotent, exactly-once processing with DLQ handling; parallelized PySpark jobs; instrumented CloudWatch and incident routing; achieved ingestion latency reduction and >99% job reliability. Implemented event-driven real-time data lake ingestion using MWAA, EventBridge, Lambda, and SQS; developed EU CRM Spark-based ETL for cross-region CRM data; established governance and security baselines.
Data Platform Infrastructure Operations. Managed day-to-day data operations for HDP, EMR, and Databricks during Hortonworks to AWS-managed services migration; oversaw governance, cost-reduction initiatives; provided Redshift, RDS, and Hive DBA support; automated reporting via Python.
Internal POC: end-to-end ML pipeline prototype. Stack included Azure, MLFlow, Python, MS SQL, Docker, Kubernetes, Jenkins; designed drift management, model retraining, and performance monitoring; demonstrated end-to-end ML workflow.
Education
Certifications and awards
Hire Mahnoor Zahid today
Post a job and invite Mahnoor, or contact them directly. No hidden commission.