Available to hire
8+ years of experience designing, building, and optimizing data platforms, warehouses, and ETL pipelinesStrong expertise in Apache Kafka, Apache NiFi, Spark, Airflow, ELK Stack, Docker, Kubernetes. Expert in cloud platforms: AWS, GCP, and Azure. Led large-scale data integration and warehousing projects for global organizations, delivering scalable and reliable data solutions. Proven experience in Python-based automation, data architecture, and real-time streaming pipelines. Focused on improving data processing efficiency to support informed, data-driven decisions. Experienced in leading and mentoring cross-functional teams to generate actionable insights.
Skills
Experience Level
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Intermediate
Language
English
Fluent
Work Experience
Principal Data Engineer at MicroDev Solutions
May 4, 2022 - Present• Spearheaded the development and maintenance of ETL pipelines and a centralized data warehouse, ensuring seamless data integration and accessibility.
• Engineered containerized data pipelines using Confluent Kafka, Mendix, SIPAT, and Python microservices, reducing data processing time by 40%.
• Designed a scalable, high-performance data architecture supporting real-time analytics across 10+ business units.
• Developed robust streaming platforms for multi-source data ingestion, increasing pipeline efficiency by 35% for data lakes and warehouses.
• Built ETL pipelines using Apache Nifi to ingest data from JSON, CSV, and XML sources, transforming 100+ TB of raw data into optimized Parquet format.
• Developed a multi-layer data architecture on AWS (S3, Athena, EMR) that improved query performance by 30%.
• Integrated processed datasets into Amazon Redshift to support real-time business intelligence and reporting needs
• Collaborated with cross-functional teams to improve data accessibility, reliability, and overall data engineering best practices.
• Tech Stack / Tools: Apache Kafka, Apache Airflow, Apache Nifi, ELK Stack (Elasticsearch), Python, SQL, Bash scripting, Docker, Kubernetes, Azure Cloud, REST APIs, Git, Linux
Senior Data Engineer at TechnoGenics SMC PVT LTD
February 9, 2021 - April 14, 2022• Designed and maintained large-scale ETL pipelines and data warehouses, improving data processing speed by 30%.
• Implemented Python, Kafka, Elasticsearch, FluentD, and GCP (GKE) solutions, supporting 5+ TB daily data ingestion.
• Worked on Python, Kafka, Elasticsearch, FluentD, and GCP, with a particular focus on Google Kubernetes Engine (GKE).
• Achieved a 30% improvement in data processing speed by designing and implementing a robust data warehouse and ETL pipelines that consolidated data from multiple sources.
• Built a data warehouse from scratch in AWS Redshift and Python microservices, consolidating 10+ heterogeneous data sources.
• Developed Python-based microservices for automated data ingestion and transformation, improving pipeline efficiency and reducing manual intervention.
• Tech Stack / Tools: Python, AWS Redshift, AWS S3, REST APIs, Microservices Architecture, SQL, Docker, Git, Linux
Principal Data Engineer at MicroDev Solutions (Remote)
May 1, 2022 - PresentSpearheaded development and maintenance of ETL pipelines and a centralized data warehouse, ensuring seamless data integration and accessibility. Engineered containerized data pipelines using Confluent Kafka, Mendix, SIPAT, and Python microservices, reducing data processing time by 40%. Designed scalable, high-performance data architecture supporting real-time analytics across 10+ business units. Built streaming platforms for multi-source data ingestion, increasing pipeline efficiency by 35% for data lakes and warehouses. Built ETL pipelines using Apache Nifi to ingest data from JSON, CSV, and XML sources, transforming 100+ TB of raw data into optimized Parquet format. Developed a multi-layer data architecture on AWS (S3, Athena, EMR) that improved query performance by 30%. Integrated processed datasets into Amazon Redshift to support real-time BI and reporting needs. Collaborated with cross-functional teams to improve data accessibility, reliability, and best practices.
Software Engineer at Ebryx Pvt. Ltd
February 1, 2017 - January 1, 2021Developed Python automation scripts, ETL pipelines, and Django-based web apps, managing 90+ servers with a self-healing monitoring framework. Engineered a reporting portal to manage and execute end-to-end data workflows using Python Django and MySQL. Built a Python framework to monitor ~90 servers with self-healing capabilities and automatic alerting. Utilized Python, Linux, Bash scripting, Elasticsearch, and SQL to design, develop, and optimize ETL pipelines. Built automated pipelines for a ML-based phishing detection system, increasing detection throughput by 50%.
Education
MS in Data Science for Business at University of Stirling
April 5, 2023 - July 18, 2024BS in Computer Science at FAST National University
March 6, 2013 - January 4, 2017Qualifications
Enhanced Data Processing Efficiency
May 15, 2025 - January 8, 2026Gold Medal in Computer Science
January 4, 2017 - January 4, 2017Industry Experience
Software & Internet, Education, Computers & Electronics, Professional Services
Designed and implemented a fully automated, end-to-end ETL pipeline to ingest data from public APIs and store it in a Hive data warehouse for analytical use. Using Apache NiFi for orchestration, the pipeline collects data, transforms it, and loads it into Hive for fast querying and analysis. The system ensures scalability, modularity, and daily updates with real-time data transformations.
Hire a Database Developer
We have the best database developer experts on Twine. Hire a database developer today.