Valery Lishko

Developed ETL/ELT pipelines for banking data using Databricks (PySpark, SQL) and integrated with downstream data stores. Built and optimized data workflows in Azure Data Factory, enabling reliable ingestion, transformation, and scheduling for financial data.

Ingested data from Azure Blob Storage in multiple formats (Parquet, CSV). Built ETL/ELT pipelines with Databricks (PySpark, SQL) to clean, standardize, and preprocess data. Used dbt to model, transform, and document data in Snowflake, and automated dbt workflows for incremental updates. Designed Azure Data Factory pipelines for ingestion orchestration. Loaded processed data via Snowflake COPY commands and native connectors; optimized Databricks Spark jobs and implemented RBAC/data masking in Snowflake to enhance security. Improved ETL efficiency and reduced processing times with on-demand Snowflake scaling.

Ingested raw data from Woody_IO and Azure Blob Storage (JSON, Parquet, CSV) with efficient partitioning. Built ETL/ELT pipelines using Databricks (PySpark, SQL) to clean, transform, and enrich data before loading into Snowflake data marts. Implemented Azure Data Factory pipelines for ingestion orchestration. Used Snowflake COPY and native connectors to load data; optimized Spark jobs with tuning and parallel processing. Implemented data validation and cleansing routines with PySpark to ensure data integrity.