Experienced Data Engineer

Lokesh Rayavarapu

Experienced Data Engineer

Available to hire

Experienced Data Engineer

Experience Level

Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert

Work Experience

Senior Data Engineer at Johnson & Johnson
March 1, 2024 - Present
Designed a cloud-native Azure Lakehouse architecture using Azure Data Factory, Azure Databricks, ADLS Gen2, and Azure Synapse to support enterprise clinical and commercial analytics. Built scalable ETL/ELT ingestion frameworks with Python and SQL to onboard data from clinical trials, ERP, and external APIs. Implemented Delta Lake with ACID, schema evolution, and time travel; developed PySpark-based data processing pipelines for large-scale datasets; designed dimensional models (star and snowflake) for downstream analytics in Power BI and Tableau. Enabled near real-time ingestion via Azure Event Hubs and Structured Streaming. Built reusable data-validation libraries; implemented data quality frameworks; tuned Databricks clusters for performance; enforced RBAC and encryption; integrated Snowflake for downstream consumption; built CI/CD pipelines with Azure DevOps; orchestrated workflows with ADF triggers and Databricks Jobs; monitored with Azure Monitor; automated tasks with PowerShell a
Senior Data Engineer at Freddie Mac
August 1, 2021 - February 1, 2024
Architected an enterprise risk analytics platform using Azure Databricks, Azure Data Factory, and Azure Data Lake. Built high-volume ETL/ELT pipelines to ingest loan, servicing, and market datasets (batch and near-real-time). Developed Delta tables to support regulatory reporting and auditability; created RESTful and SOAP API integrations with Python, WSDL, and Azure Logic Apps, secured with Azure AD/OAuth. Tuned Spark jobs for improved performance; designed dimensional models (star/snowflake) for risk, pricing, and portfolio analytics; built curated data marts for Power BI and Tableau dashboards; automated workflow orchestration with Apache Airflow and cloud schedulers; implemented data validation, governance, lineage, and metadata management for regulatory compliance; supported ad-hoc analytics via Databricks SQL Analytics; integrated Teradata legacy systems; automated deployments via Git CI/CD; provided production support and performance troubleshooting; collaborated with risk analy
Data Engineer at Insurance Services Office
January 1, 2019 - July 1, 2021
Designed and implemented an enterprise-scale insurance analytics data platform using Hadoop, Apache Spark, and Snowflake to support risk modeling, underwriting, pricing, and regulatory reporting. Built end-to-end ETL/ELT pipelines with Python, SQL, and Spark to ingest high-volume policy, claims, and third-party data. Migrated large datasets from Teradata to Snowflake using Sqoop and Python; implemented SnowPipe and SnowSQL for continuous ingestion; parsed JSON and XML feeds; designed dimensional models (star and snowflake) for underwriting and claims analytics; automated workflow orchestration with Apache Airflow and Oozie; implemented data quality checks, reconciliation logic, and exception handling; implemented IAM, RBAC, and encryption for secure access; built analytics-ready datasets for Tableau; supported feature engineering for actuarial and predictive risk models; contributed to cloud migration and modernizing legacy platforms; provided production support and incident resolution
Data Engineer at Aditya Birla Retail
June 1, 2016 - September 1, 2018
Designed and delivered a centralized retail analytics platform using Hadoop, Spark, and SQL to support sales, inventory, and customer insights. Built ETL pipelines with Python, Pig, Hive; ingested data into HDFS with Sqoop and Flume handling 10+ TB of data. Performed distributed transformations using Spark SQL/Scala; modeled relational and dimensional data for BI; built batch processing for daily/weekly/monthly reports; performed data cleansing and deduplication; loaded curated datasets into Teradata and Redshift; optimized SQL queries and ETL jobs; built executive dashboards in Tableau and Power BI; established data quality checks; supported forecasting and demand planning; assisted with legacy migrations; automated batch jobs with Unix; prepared documentation; ensured data security; provided production support; collaborated with business and analytics teams.

Education

Add your educational history here.

Qualifications

Add your qualifications or awards here.

Industry Experience

Healthcare, Financial Services, Professional Services, Software & Internet