Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

I'm Anurag Srivastava, a data engineer with over 19 years of IT experience, specializing in building robust data platforms across industries such as Energy, Education, Aviation, Retail, Utilities, Banking and Insurance. I design and deliver end-to-end data solutions using Snowflake, AWS, Python, Apache Airflow and other modern ETL/ELT and big data technologies, with a strong focus on data modelling, scalability, and governance. In my roles, I lead and contribute to data ingestion patterns, medallion-architecture data lakes, and cloud-based pipelines. I enjoy collaborating with cross-functional teams, mentoring teammates, and applying best-practice design principles to accelerate insights while ensuring quality and maintainability.…I'm Anurag Srivastava, a data engineer with over 19 years of IT experience, specializing in building robust data platforms across industries such as Energy, Education, Aviation, Retail, Utilities, Banking and Insurance. I design and deliver end-to-end data solutions using Snowflake, AWS, Python, Apache Airflow and other modern ETL/ELT and big data technologies, with a strong focus on data modelling, scalability, and governance. In my roles, I lead and contribute to data ingestion patterns, medallion-architecture data lakes, and cloud-based pipelines. I enjoy collaborating with cross-functional teams, mentoring teammates, and applying best-practice design principles to accelerate insights while ensuring quality and maintainability.

Anurag Srivastava

Full Stack Developer, Database Developer, Web Developer, +2





I'm Anurag Srivastava, a data engineer with over 19 years of IT experience, specializing in building robust data platforms across industries such as Energy, Education, Aviation, Retail, Utilities, Banking and Insurance. I design and deliver end-to-end data solutions using Snowflake, AWS, Python, Apache Airflow and other modern ETL/ELT and big data technologies, with a strong focus on data modelling, scalability, and governance. In my roles, I lead and contribute to data ingestion patterns, medallion-architecture data lakes, and cloud-based pipelines. I enjoy collaborating with cross-functional teams, mentoring teammates, and applying best-practice design principles to accelerate insights while ensuring quality and maintainability.…I'm Anurag Srivastava, a data engineer with over 19 years of IT experience, specializing in building robust data platforms across industries such as Energy, Education, Aviation, Retail, Utilities, Banking and Insurance. I design and deliver end-to-end data solutions using Snowflake, AWS, Python, Apache Airflow and other modern ETL/ELT and big data technologies, with a strong focus on data modelling, scalability, and governance. In my roles, I lead and contribute to data ingestion patterns, medallion-architecture data lakes, and cloud-based pipelines. I enjoy collaborating with cross-functional teams, mentoring teammates, and applying best-practice design principles to accelerate insights while ensuring quality and maintainability.

Available to hire

I’m Anurag Srivastava, a data engineer with over 19 years of IT experience, specializing in building robust data platforms across industries such as Energy, Education, Aviation, Retail, Utilities, Banking and Insurance. I design and deliver end-to-end data solutions using Snowflake, AWS, Python, Apache Airflow and other modern ETL/ELT and big data technologies, with a strong focus on data modelling, scalability, and governance.

In my roles, I lead and contribute to data ingestion patterns, medallion-architecture data lakes, and cloud-based pipelines. I enjoy collaborating with cross-functional teams, mentoring teammates, and applying best-practice design principles to accelerate insights while ensuring quality and maintainability.

Skills

Experience Level

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Intermediate

Intermediate

Language

English

Fluent

Work Experience

Senior Data Engineer at JBS Australia

May 1, 2024 - Present

Design and build the data ingestion pattern into Snowflake Data Lake from multiple sources using Qlik Replicate; develop data ingestion pipelines using Pentaho (GUI-based tool) and AWS Glue; set up API-based ingestion using Python Snowpark in Snowflake; define Bronze, Silver and Gold (Medallion) architecture for the data lake; implement Change Tracking Mechanism (SCD Type 2); build incremental data load framework in Snowflake; provide high-level estimations of data ingestion across sources; guide the team on data design and solution design principles; perform code reviews; collaborate with the data team to establish ways of working; automate repetitive development tasks using Python.

Senior Data Engineer at Super Retail Group

October 1, 2022 - May 1, 2024

Enhance and maintain the metadata-driven acquisition layer that classifies and loads data from source files to Snowflake using S3, AWS Lambda, DynamoDB, IAM Roles, SNS and Matillion. Design and maintain data models (fact and dimensional models) using Kimball methodology; design an event-driven job scheduling framework in Matillion; resolve data-related issues in ingestion pipelines and the data warehouse. Lead the exploration of migrating from Snowflake to Databricks with vendor support, define RBAC for Databricks, and collaborate with the team to establish standards and templates.

Senior Data Engineer at Lendi

August 1, 2021 - October 1, 2022

Part of a cloud DW build to satisfy reporting needs; applied Kimball modelling to design fact and dimension tables; pulled data from Salesforce/Google Sheets (via Fivetran), Kafka events, and other sources; redesigned broker commission data tool from Snowflake objects to Airflow-driven pipeline for easier maintenance; migrated legacy AWS components to modern orchestration; created Airflow DAGs to orchestrate DBT runs; built ETL pipelines in Python (Airflow) and supported DBT/AIRFLOW on Kubernetes; deployed Snowflake objects using Terraform; implemented CI/CD pipelines and aligned with agile practices.

Senior Data Engineer at Versent

February 1, 2021 - August 1, 2021

Assess and migrate current state data pipeline architecture in AWS (S3, Athena, Lambda, Aurora MySQL, CloudWatch, CloudTrail, EC2) and SnapLogic; propose data pipeline optimizations and decommissioning of components; build new infrastructure for migrated pipelines using Terraform; prepare test/implementation/rollback plans; lead HOTS with support teams to ensure self-sufficiency; contribute to a data architecture roadmap with a focus on governance and storage.

Data Engineer / Modeller at Macquarie University

August 1, 2019 - March 1, 2020

Contributed to building a Snowflake-based data warehouse; collaborated with data modellers to develop data models and load data from staging (ODS) to Data Vault/Data Mart (Kimball) for reporting; built ETL pipelines using Python (Airflow), Snow SQL and Snow Pipe to extract data from APIs, flat files, SharePoint, and emails; implemented ELT processes and JSON parsing in Snowflake; ran cost optimization measures for moving data from Oracle to Snowflake; configured Airflow on Kubernetes (EKS) and implemented automated testing with pytest; established Git practices and CI/CD integration.

Data Engineer at Qantas

March 1, 2017 - August 1, 2019

Built and enhanced ETL pipelines with Airflow to extract data from files, databases, and web services; developed backend using Python Flask for the Frequent Flyer seat allocation module; performed SQL performance tuning; participated in code reviews and PR approvals; implemented Bamboo-based CI/CD and blue/green deployment; built automated unit tests for ETL processes; enhanced deployment pipelines for ETL deployments.

Data Engineer / Big Data Developer at IRi

June 1, 2013 - January 1, 2017

Implemented and maintained data pipelines to load dimensional data into Oracle data warehouse and data marts, and transactional/fact data in Hive on MapR Hadoop; delivered star schema data for modeling and reporting teams; developed weekly load schedules using Apache Airflow; optimized pipelines by automating data loading processes, including binary data validation and cross-node transfers; designed the Buyer-Graphics attribute framework and automated unit tests; supported data and application issues; performed backward compatibility testing after major releases.

Senior Technical Lead / Data Developer at HCL Technologies / TATA

October 1, 2006 - May 1, 2013

Led high-level solution design for moving data from Oracle to Hadoop; built data marts; developed materialized views, PL/SQL procedures, and Python scripts to support Hadoop ingestion via SQOOP; implemented CSV-based collateral data loads with SQL*Loader and automated Unix scripts; performed UAT and post-implementation support; introduced performance monitoring using dynaTrace to map Oracle bottlenecks to web transactions; delivered training and mentoring on Oracle SQL/PL-SQL.

Education

Bachelor of Engineering in Information Science and Engineering at Gogte Institute of Technology, Visweshwaraiah Technological University, Karnataka, India

January 11, 2030 - January 1, 2006

Qualifications

AWS Certified Cloud Practitioner

January 11, 2030 - January 8, 2026

Industry Experience

Education, Energy & Utilities, Transportation & Logistics, Retail, Financial Services, Professional Services, Software & Internet, Travel & Hospitality