Available to hire
Data Engineer and Data Analyst with 4+ years of experience building ETL pipelines and analytics-ready datasets using Python, SQL, PySpark, Databricks, Snowflake, dbt, and Apache Airflow. Specialised in data warehousing, data validation, and workflow automation across multi-cloud platforms.
Experienced in transforming large-scale data (millions of records, 200GB+ datasets), designing Bronze/Silver/Gold Delta Lake models, and delivering KPI reporting and semantic models in Power BI.
Skills
Experience Level
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Intermediate
Intermediate
Intermediate
Intermediate
Intermediate
Beginner
Beginner
Language
Work Experience
Data Consultant at Victoria Solutions UK
November 1, 2025 - PresentImplemented PySpark transformation pipelines in Databricks to process 7M+ records from cloud and API sources into curated Delta Lake datasets for reporting. Engineered 60+ DBT models and validation tests in Snowflake covering schema verification, freshness monitoring, reconciliation logic, and source consistency checks across enterprise reporting datasets. Integrated data from 10 external APIs, Azure Data Lake Storage, and AWS S3 into Microsoft Fabric OneLake, applying transformation and validation workflows for 12M+ transactional records. Built ingestion pipelines using Python and Databricks to process 5M+ records from multi-cloud sources, standardising schemas and delivering structured analytical datasets.
Data Engineer at Denny Enterprises UK
October 1, 2024 - August 1, 2025Consolidated datasets from 8 cloud and operational sources into a centralised Snowflake warehouse using Python, SQL, and Databricks, supporting analytics workflows handling 6M+ records across business functions. Deployed 25+ SQL transformation scripts and staging tables for financial and operational reporting pipelines, enabling structured aggregation, reconciliation, and validation of high-volume transactional datasets. Configured 12 orchestrated Airflow workflows with SLA tracking, dependency mapping, and failure recovery to support scheduled ingestion and transformation across multiple reporting systems. Developed semantic reporting models and Power BI datasets with calculated DAX measures, relationship mapping, and access controls, supporting KPI analysis for 4 internal departments.
Data Analyst at Trinity Technolabs
July 1, 2023 - August 1, 2024Produced PySpark ETL pipelines on Databricks to ingest and transform data from REST APIs, AWS S3, and Azure Data Lake Storage, processing 200GB+ datasets into structured analytical layers for reporting. Architected Delta Lake Bronze, Silver, and Gold layers with incremental loads, schema enforcement, and partition logic to support reliable historical tracking and reduce reprocessing. Delivered 40+ DBT models in Snowflake with schema validation, freshness checks, and transformation logic to standardise datasets across 6+ business domains. Orchestrated 10+ Airflow DAGs with task dependencies, scheduling rules, retry mechanisms, and failure monitoring, managing 3M+ records per pipeline cycle across ingestion workflows.
Data Engineer at Pravaha Software
November 1, 2022 - June 1, 2023Streamlined ERP data pipelines using Airflow orchestration, processing 4M+ records across finance, procurement, and operations datasets into structured reporting layers for downstream analysis. Authored 30+ SQL queries, views, and stored procedures to extract, transform, and consolidate data from multi-module ERP systems into consistent reporting formats for analytics. Applied DBT-based validation rules including schema checks, null checks, uniqueness constraints, and referential integrity tests across 20+ source tables to ensure dataset accuracy before warehouse loading.
Data Analyst at Trinity Technolabs
August 1, 2021 - October 1, 2022Managed 18 ETL pipelines using Python, SQL, and PySpark to process 300GB+ of structured and semi-structured datasets from REST APIs, AWS S3, and Azure Data Lake Storage into analytics-ready reporting layers. Developed 45 SQL transformation scripts and stored procedures within Snowflake and Databricks for data cleansing, reconciliation, aggregation, and validation workflows across operational systems. Designed Bronze, Silver, and Gold Delta Lake data models with incremental loading, schema evolution, and partition optimisation, supporting reporting access for 25+ internal stakeholders. Configured and monitored 14 Airflow DAGs with dependency management, scheduling policies, retry handling, and SLA tracking to automate ingestion workflows processing 5M+ records daily.
Education
MSc, Big Data Technology at Glasgow Caledonian University
September 1, 2024 - September 1, 2025MSc, Computer Science at Bharathiyar University
April 1, 2016 - May 1, 2018BSc, Computer Science at Bharathiyar University
June 1, 2013 - April 1, 2015MSc, Big Data Technology at Glasgow Caledonian University
September 1, 2024 - September 1, 2025MSc, Computer Science at Bharathiyar University
April 1, 2016 - May 1, 2018BSc, Computer Science at Bharathiyar University
June 1, 2013 - April 1, 2015MSc, Big Data Technology at Glasgow Caledonian University
January 1, 2023 - September 1, 2025MSc, Computer Science at Bharathiyar University
April 1, 2016 - May 1, 2018BSc, Computer Science at Bharathiyar University
June 1, 2013 - April 1, 2015Qualifications
Microsoft Certified: Fabric Analytics Engineer Associate
January 11, 2030 - August 20, 2026Microsoft Certified: Azure AI Fundamentals (AI-900)
January 11, 2030 - August 20, 2026AWS Certified Data Engineer
January 11, 2030 - August 20, 2026Oracle Database SQL Certification
January 11, 2030 - August 20, 2026Microsoft Fabric End-to-End Analytics – Udemy
January 11, 2030 - August 20, 2026Data Engineering and Cloud Data Pipelines – Udemy
January 11, 2030 - August 20, 2026REST API Development and Integration – Udemy
January 11, 2030 - August 20, 2026Microsoft Certified: Fabric Analytics Engineer Associate
January 11, 2030 - August 20, 2026Microsoft Certified: Azure AI Fundamentals (AI-900)
January 11, 2030 - August 20, 2026AWS Certified Data Engineer
January 11, 2030 - August 20, 2026Oracle Database SQL Certification
January 11, 2030 - August 20, 2026Microsoft Fabric End-to-End Analytics – Udemy
January 11, 2030 - August 20, 2026Data Engineering and Cloud Data Pipelines – Udemy
January 11, 2030 - August 20, 2026REST API Development and Integration – Udemy
January 11, 2030 - August 20, 2026Industry Experience
Professional Services, Computers & Electronics, Education, Financial Services
Skills
Experience Level
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Intermediate
Intermediate
Intermediate
Intermediate
Intermediate
Beginner
Beginner
Hire a Data Engineer
We have the best data engineer experts on Twine. Hire a data engineer in Ipswich today.