Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

Hello, I’m Varun Raj K, a Senior Data Engineer with over 10 years of experience in big data, cloud data engineering, data architecture, and analytics solutions. I excel at building scalable data pipelines, data ingestion, modeling, ETL/ELT processes, and managing enterprise data platforms across Azure, AWS, and GCP.\n\nI enjoy solving complex data challenges, collaborating with cross-functional teams, and migrating on-premises data workloads to the cloud. I’m passionate about turning data into actionable insights through robust architectures and effective dashboards.…Hello, I’m Varun Raj K, a Senior Data Engineer with over 10 years of experience in big data, cloud data engineering, data architecture, and analytics solutions. I excel at building scalable data pipelines, data ingestion, modeling, ETL/ELT processes, and managing enterprise data platforms across Azure, AWS, and GCP.\n\nI enjoy solving complex data challenges, collaborating with cross-functional teams, and migrating on-premises data workloads to the cloud. I’m passionate about turning data into actionable insights through robust architectures and effective dashboards.

Varun Raj K

Full Stack Developer, Back-End Developer, Front-End Developer, +2





Hello, I’m Varun Raj K, a Senior Data Engineer with over 10 years of experience in big data, cloud data engineering, data architecture, and analytics solutions. I excel at building scalable data pipelines, data ingestion, modeling, ETL/ELT processes, and managing enterprise data platforms across Azure, AWS, and GCP.\n\nI enjoy solving complex data challenges, collaborating with cross-functional teams, and migrating on-premises data workloads to the cloud. I’m passionate about turning data into actionable insights through robust architectures and effective dashboards.…Hello, I’m Varun Raj K, a Senior Data Engineer with over 10 years of experience in big data, cloud data engineering, data architecture, and analytics solutions. I excel at building scalable data pipelines, data ingestion, modeling, ETL/ELT processes, and managing enterprise data platforms across Azure, AWS, and GCP.\n\nI enjoy solving complex data challenges, collaborating with cross-functional teams, and migrating on-premises data workloads to the cloud. I’m passionate about turning data into actionable insights through robust architectures and effective dashboards.

Available to hire

Hello, I’m Varun Raj K, a Senior Data Engineer with over 10 years of experience in big data, cloud data engineering, data architecture, and analytics solutions. I excel at building scalable data pipelines, data ingestion, modeling, ETL/ELT processes, and managing enterprise data platforms across Azure, AWS, and GCP.\n\nI enjoy solving complex data challenges, collaborating with cross-functional teams, and migrating on-premises data workloads to the cloud. I’m passionate about turning data into actionable insights through robust architectures and effective dashboards.

Skills

Experience Level

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Language

English

Fluent

Work Experience

Senior Data Engineer at US Bank

February 1, 2023 - Present

Led large-scale migrations from on-prem SQL databases to Azure Data Lake, Azure SQL Database, Azure Synapse Analytics, and Databricks using Azure Data Factory. Designed scalable BI and analytics architectures leveraging ADLS Gen1/Gen2, Synapse, HDInsight, Databricks, Stream Analytics, and Cosmos DB. Served as SME for data architecture, modeling, and pipeline design aligned to business requirements; built and optimized enterprise-grade pipelines with StreamSets, ADF, and Databricks. Implemented CI/CD pipelines with Azure DevOps and Jenkins; integrated Databricks with ADLS, Cosmos DB, Event Hubs, and DevOps workflows. Created ingestion pipelines for hybrid data integration and supported dashboards via Power BI.

Senior Data Engineer at Centene Corporation

January 1, 2021 - January 1, 2023

Designed and developed scalable Spark applications (PySpark/Scala) to process data from RDBMS, APIs, Kafka, and streaming sources; migrated legacy ETL to scalable Spark pipelines with optimized S3 integrations; built API-driven ingestion pipelines and land curated datasets into S3; migrated Informatica workloads to Informatica IDMC using Terraform; implemented data governance and metadata management; built Snowflake-based data lake solutions; developed Spark SQL pipelines and real-time streaming using Spark Streaming and Kafka; automated CI/CD deployments with Jenkins; containerized apps with Docker and Kubernetes.

Data Engineer at Walgreens - Deerfield, Illinois

October 1, 2019 - December 1, 2020

Delivered data engineering solutions by building and optimizing Hadoop/Spark-based pipelines in Cloudera environment; implemented advanced data preprocessing, feature engineering, and data validation; built and tuned Hive and Pig workflows for batch ingestion; set up real-time ingestion using Kafka and Storm into HBase; developed Spark applications (Scala/PySpark) for analytics; built dashboards and monitoring using Kibana; migrated on-prem workloads to cloud, and implemented MapReduce jobs for operational reporting.

Data Engineer at Phenom People Pvt Ltd

November 1, 2017 - September 1, 2019

Developed Spark-based big data analytics and ML applications (Python/Scala), including NLP sentiment analysis and time-series forecasting; created conceptual, logical, and physical data models; built BI dashboards with Tableau; implemented data pipelines and ETL using Spark, Hive, and Python; leveraged AWS/GCP data infrastructure for data ingestion and storage; coordinated with stakeholders to define data requirements; implemented data quality checks and metadata management.

Data Engineer at Global Software Solutions

June 1, 2015 - October 1, 2017

Wrote shell scripts to extract data from Unix servers to HDFS, implemented microservices with Spring Boot, and built data pipelines using Spark, Pig, and Oozie to automate ETL; loaded transformed data into MongoDB; deployed applications on Pivotal Cloud Foundry; built Hive tables and MapReduce jobs; managed Hadoop clusters on AWS and Cloudera; built data pipelines for batch processing, data migration, and reporting.