Baye Ndiaga Ndao

Created and maintained technical documentation. Designed and implemented a DBT unit testing framework to automate SQL validation within a GitHub Actions CI/CD pipeline, ensuring changes to dbt models are tested before merge or deployment. Implemented additional data quality checks using DBT test functions, Great Expectations, and DBT-expectations. Built a client-friendly UI to onboard new clients, automating daily and snapshot file validation with auto-feedback. Built and maintained reusable Python modules to construct Airflow DAGs. Led data pipeline development to improve data integration from diverse sources for new clients. Implemented a monitoring DAG to measure daily duration, resource usage, cost, and error rate of BigQuery jobs, table rows/volumes, and GCS metadata. Identified bottlenecks and optimized DAGs to reduce runtime. Maintained code quality and deprecated components. Significantly optimized SQL queries to enhance performance and minimize costs. Implemented a comprehensi

Implemented ETL processes with data transformation using Python, Airflow, DBT, Beam, BigQuery, and SQL. Explored technologies and developed solutions to continuously improve data quality, workflow reliability, and scalability. Demonstrated ownership mentality, curiosity, and rapid learning in a fully remote, international team. Implemented an alerting system to notify Slack immediately whenever a data pipeline task fails or data ingestion is delayed.

Implemented ETL processes and data transformations using Python, Airflow, DBT, Beam, BigQuery, and SQL; worked on improving data quality, reliability, and scalability; demonstrated ownership mentality and rapid learning; collaborated with a fully remote, international team; implemented Slack alerts for pipeline failures and ingestion delays.