Yan Cheng Liu
I am a Senior Data Engineer with 8+ years of experience designing and delivering end-to-end data solutions for real-time analytics, business intelligence, and AI applications.
About
I am a Senior Data Engineer with 8+ years of experience designing and delivering end-to-end data solutions for real-time analytics, business intelligence, and AI applications. I have a strong background in data pipelines, modeling, and platform architecture across AWS, Azure, and GCP environments. I thrive on transforming complex data into actionable insights and reliable reporting that empower smarter decisions.
I enjoy collaborating with product, data science, and UX teams to align data models with key KPIs and ship measurable business impact. I excel at building scalable data platforms, automating workflows, and delivering dashboards that drive engagement and revenue.
- 9 years' experience
- Joined January 2026
Industry experience
- Software & Internet
- Professional Services
- Media & Entertainment
More experts in San Francisco, United States
Work experience· 4 roles
Built the Host Profile Data Service for Airbnb Rooms using Scala, Kafka, and Flink, integrating host activity into Snowflake pipelines that powered the Host Passport feature and improved data freshness by ~23%. Designed the Atlas Schema and Databricks + Spark pipelines for Airbnb Services & Experiences data into a unified model, reducing query latency by ~20% and simplifying business reporting. Created Looker dashboards on top of dbt and Snowflake, providing actionable engagement and revenue metrics for product and business teams. Developed ML feature pipelines with Python, TensorFlow, and Vertex AI to enhance personalization and boost trip-planning engagement. Automated 200+ Airflow workflows in AWS with Terraform, maintaining 99% reliability and full observability via Prometheus and Grafana. Tuned Delta Lake performance on Databricks, reducing refresh times and compute costs through incremental processing and optimized partitioning. Collaborated with PMs, Data Scientists, and UX team
Rebuilt a large-scale A/B testing pipeline using Spark and Snowflake, improving data freshness by 16% and reliability across thousands of experiments. Modernized the Metric Monitor with new streaming and batch pipelines, reducing alert latency by 17% and improving anomaly detection accuracy. Optimized Spark + Presto jobs for experimentation analysis, improving compute efficiency and reducing job failures through query tuning and parallel execution. Built Airflow + Snowflake pipelines for experiment metrics, automating ingestion and reporting to serve product and analytics teams. Partnered with Product Science and Infrastructure teams to standardize experiment schemas and develop interactive dashboards in Scuba, improving data visibility and decision speed.
Built a new client-side logging pipeline for Instagram Stories, capturing high-volume engagement with improved throughput and data consistency. Designed and deployed an end-to-end data solution for Explore Brand Safety, integrating content integrity models with ad-delivery metrics for safer recommendations. Developed exposure-based A/B testing metrics for Instagram Ads, enabling fine-grained performance attribution and automated experiment analysis. Collaborated with cross-functional ML and Ads Engineering teams to align event schemas and experiment frameworks across Instagram and Facebook. Drove adoption of modern data modeling and monitoring standards, improving visibility and reliability of metrics powering Instagram's ad-ranking systems.
Education
Hire Yan Cheng Liu today
Post a job and invite Yan, or contact them directly. No hidden commission.








