SR

Sheshanth Reddy

Available to hire

I am a Data Engineer with over 7 years of experience designing and implementing cloud-based and on-premises data solutions.

About

I am a Data Engineer with over 7 years of experience designing and implementing cloud-based and on-premises data solutions. I specialize in Databricks, PySpark, and Kafka, and have a proven track record migrating warehouses to cloud-native platforms, building scalable batch and streaming pipelines, and delivering BI-ready datasets for analytics and ML.

I enjoy collaborating with platform and product teams to create data products, implementing governance and security, and enabling self-service analytics through dashboards and reporting. My work spans from data ingestion and transformations to data modeling (Dimensional, Data Vault, Star) and modern architectures such as Medallion, Delta Lake, and lakehouse patterns.

  • 9 years' experience
  • Joined October 2025

Languages

  • EnglishFluent

Industry experience

  • Software & Internet
  • Professional Services

Work experience· 7 roles

Data Engineer · Atlas AirJul 2023 – Present · 3 yrs 5 mos

Worked on on-premises BI solutions using SSIS, SSRS, and Power BI to build ETL pipelines, operational reports, and dashboards. Migrated to Azure Cloud, adopting Azure Databricks to modernize pipelines and process large-scale semi-structured and unstructured data. Designed and optimized workflows as the central platform for big data processing, Spark-based transformations, and ML-ready pipelines. Developed and deployed PySpark jobs for batch ingestion from APIs, flat files, and RDBMS sources. Streamed ingestion from Kafka into Cosmos DB and ADLS, handling high-velocity JSON/NoSQL data for low-latency analytics. Implemented Azure Key Vault, RBAC, and Managed Identities to safeguard sensitive datasets across production pipelines. Built automated pipelines for schema and data deployments using GitHub Actions and GraphQL Inspector, with Git as the version control system to manage and back up code. Delivered Power BI dashboards and reports by integrating with modernized cloud data sources, e

Data Engineer · Molina HealthCareJun 2023 – Oct 2025 · 2 yrs 5 mos

Migrated on-premises databases and data warehouses into the Databricks Lakehouse on GCP using Google Cloud Storage (GCS) as the landing zone, establishing a scalable and cloud-native foundation. Designed and developed robust PySpark pipelines in Databricks for data ingestion, cleansing, enrichment, and transformation, producing high-quality reusable datasets. Implemented near real-time streaming pipelines using Kafka and Spark Structured Streaming, later refactored with GCP Pub/Sub and Databricks Structured Streaming for cloud-native scalability. Optimized Databricks performance and cost efficiency by right-sizing clusters, enabling autoscaling, and tuning Spark jobs for large-scale batch and streaming workloads. Automated ETL workflows with Apache Airflow DAGs, ensuring reliability and reducing manual interventions in production pipelines. Built a centralized logging and monitoring framework using ELK Stack and GCP Stackdriver, enabling proactive detection and faster resolution of fai

Data Engineer · Molina HealthCareAug 2021 – Jun 2023 · 1 yr 11 mos

Migrated on-premises databases and data warehouses into the Databricks Lakehouse on GCP using Google Cloud Storage (GCS) as the landing zone. Designed and developed PySpark pipelines for data ingestion, cleansing, enrichment, and transformation. Implemented near real-time streaming pipelines using Kafka and Spark Structured Streaming, later refactored with GCP Pub/Sub for cloud-native scalability. Optimized Databricks performance and cost efficiency through right-sizing clusters and autoscaling. Automated ETL workflows with Apache Airflow, and built centralized logging/monitoring using ELK Stack and Stackdriver. Partnered with platform and product teams to deliver curated data products in Databricks Lakehouse for downstream consumers.

Big Data Developer · Atom IT ServicesDec 2020 – Oct 2025 · 4 yrs 11 mos
Big Data Developer · Atom IT ServicesJun 2019 – Dec 2020 · 1 yr 7 mos
ETL Engineer · Dhruvsoft servicesMay 2019 – Oct 2025 · 6 yrs 6 mos
ETL Engineer · Dhruvsoft ServicesOct 2017 – May 2019 · 1 yr 8 mos

Education

Bachelor of Technology in Computer Science · Jawaharlal Nehru Technological University, Hyderabad2030 – 2017
Bachelor of Technology in Computer Science · Jawaharlal Nehru Technological University, Hyderabad2030 – 2017

Hire Sheshanth Reddy today

Post a job and invite Sheshanth, or contact them directly. No hidden commission.

Post a job