Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

Data engineer with experience building scalable ETL/ELT pipelines, cloud data platforms, and data transformation workflows using Azure (ADF, Databricks, ADLS Gen2, Azure SQL, Synapse) and Apache Spark with Delta Lake. Skilled in advanced SQL, PySpark, performance-optimised Spark workloads, incremental data pipelines, and data validation/quality monitoring to improve data accuracy and reporting efficiency. Hands-on with Lakehouse architecture, OneLake/Fabric data engineering workflows (pipelines, notebooks, analytics), and automation for reliable daily data delivery. Also experienced with Generative AI for structured prompt design, prompt optimisation, and AI-assisted technical problem solving.…Data engineer with experience building scalable ETL/ELT pipelines, cloud data platforms, and data transformation workflows using Azure (ADF, Databricks, ADLS Gen2, Azure SQL, Synapse) and Apache Spark with Delta Lake. Skilled in advanced SQL, PySpark, performance-optimised Spark workloads, incremental data pipelines, and data validation/quality monitoring to improve data accuracy and reporting efficiency. Hands-on with Lakehouse architecture, OneLake/Fabric data engineering workflows (pipelines, notebooks, analytics), and automation for reliable daily data delivery. Also experienced with Generative AI for structured prompt design, prompt optimisation, and AI-assisted technical problem solving.

sri divya Lalam

Data Engineer, Software Engineer, Data Scientist, +2





Data engineer with experience building scalable ETL/ELT pipelines, cloud data platforms, and data transformation workflows using Azure (ADF, Databricks, ADLS Gen2, Azure SQL, Synapse) and Apache Spark with Delta Lake. Skilled in advanced SQL, PySpark, performance-optimised Spark workloads, incremental data pipelines, and data validation/quality monitoring to improve data accuracy and reporting efficiency. Hands-on with Lakehouse architecture, OneLake/Fabric data engineering workflows (pipelines, notebooks, analytics), and automation for reliable daily data delivery. Also experienced with Generative AI for structured prompt design, prompt optimisation, and AI-assisted technical problem solving.…Data engineer with experience building scalable ETL/ELT pipelines, cloud data platforms, and data transformation workflows using Azure (ADF, Databricks, ADLS Gen2, Azure SQL, Synapse) and Apache Spark with Delta Lake. Skilled in advanced SQL, PySpark, performance-optimised Spark workloads, incremental data pipelines, and data validation/quality monitoring to improve data accuracy and reporting efficiency. Hands-on with Lakehouse architecture, OneLake/Fabric data engineering workflows (pipelines, notebooks, analytics), and automation for reliable daily data delivery. Also experienced with Generative AI for structured prompt design, prompt optimisation, and AI-assisted technical problem solving.

Available to hire

Data engineer with experience building scalable ETL/ELT pipelines, cloud data platforms, and data transformation workflows using Azure (ADF, Databricks, ADLS Gen2, Azure SQL, Synapse) and Apache Spark with Delta Lake. Skilled in advanced SQL, PySpark, performance-optimised Spark workloads, incremental data pipelines, and data validation/quality monitoring to improve data accuracy and reporting efficiency.

Hands-on with Lakehouse architecture, OneLake/Fabric data engineering workflows (pipelines, notebooks, analytics), and automation for reliable daily data delivery. Also experienced with Generative AI for structured prompt design, prompt optimisation, and AI-assisted technical problem solving.

Skills

Experience Level

Expert

Expert

Expert

Expert

Intermediate

Intermediate

Work Experience

Data Engineer at IT – Nine Analytics

December 1, 2024 - Present

Designed and implemented SQL data schemas and optimised complex SQL queries using indexing and performance tuning techniques. Built analytics datasets for examination results and subscription analytics to support reporting and downstream analytics. Developed and optimised PySpark and SQL ETL/ELT transformation workloads on Azure, reducing execution time by 40% and improving efficiency. Built automated ETL/ELT workflows and implemented data validation and data quality monitoring, achieving approximately 99% data accuracy across reporting. Automated daily data pipelines to improve pipeline reliability and reduce operational effort. Developed supporting Python and SQL scripts for data transformations and applied modern cloud data engineering principles including orchestration and reliable analytics data delivery.

Data Engineer at Tech a India

April 1, 2020 - July 31, 2023

Developed and optimised Azure Data Factory pipelines to ingest data from on-premises databases into Azure Storage. Used ADF mapping and joins to perform transformations and improve efficiency. Implemented incremental data processing in SQL Server using Change Data Capture-style mechanisms, checkpointing, and dependency-based pipeline orchestration. Built ingestion pipelines writing records into ADLS Gen2, and developed data processing solutions using Spark for scalable analytics workloads. Improved performance through SQL query optimisation, indexing, partitioning, and Spark optimisation techniques. Applied data modelling concepts for analytics-ready datasets and collaborated with business and technical stakeholders to establish data quality controls across Azure data platforms.