Data Engineer – Machine Learning at IBM Philippines

Collaborated with Data Scientists to analyze, retrieve, and transform source data as prerequisites for ML model implementation. Built and maintained machine learning pipelines using Airflow, Control-M, Python, and Snowflake to support ML workflows. Integrated machine learning models with relevant data sources, enabling real-time data delivery through APIs and SFTP. Migrated a Domino-based machine learning pipeline to a Kubernetes-based implementation, improving data processing efficiency and delivery time. Refactored and optimized complex SQL queries by applying query plan analysis and temporary tables, improving data extraction performance for machine learning workflows by over 75%.

Associate Data Engineer at Security Bank Corporation

Utilized PySpark and SQL to resolve data issues related to pipeline optimization and data quality, leading to enhanced performance and reliability of data workflows. Enhanced data pipeline using Python and Airflow, creating new DAG for a Datamart table to improve data consumption and reporting. Enhanced data pipeline using Python and Pyspark by adding additional columns to an existing table in the data lakehouse, improving data accuracy and usability for reporting and analytics. Collaborated within a team to streamline data ingestion processes by extracting data from Kafka Topic and efficiently loading it into S3 buckets, facilitating smooth data consumption for analysis. Utilized root cause analysis methodologies to address data issues in production environments, guaranteeing the integrity of data and the reliability of systems.