Taleen Chen

Designed and iterated an LLM evaluation framework for a call centre AI agent using RAGAS and the Google Vertex AI Evaluation SDK. Defined custom metrics and prompt-based acceptance criteria, combined quantitative scoring with human feedback to drive model iteration and model selection. Improved retrieval accuracy by 20% and supported production deployment by delivering actionable evaluation insights. Assessed a bank’s RAG-based internal knowledge base to validate factual accuracy and retrieval reliability. Built test scenarios for stress testing, multi-hop reasoning, top-k truncation, and hyperparameter tuning to surface hallucinations and failure modes, and used LLM-as-judge scoring to detect hallucinations and assess response quality. Produced an analytical report that directly informed model tuning ahead of rollout. Prototyped interactive AI agents on GCP for automated Q&A and stakeholder response capture. Built the retrieval layer (embeddings in Datastore, responses stored in B

Enabled an AI decision engine by designing the confidence engine for rules-based eligibility decisioning. Owned the end-to-end data flow from source systems through the ML model into downstream platforms and supplied feature-level datasets for the AI/ML component. Established a Bronze/Silver/Curated Medallion architecture using Azure Synapse and Databricks, using a metadata-driven framework and control tables to manage storage paths and ingestion toggles for scalable processing. Implemented automated data validation and operational monitoring (uniqueness, null checks, deduplication) with centralized metric logging and integration with Azure Log Analytics for auditability. Standardized engineering practices by defining repository structures and notebook templates in Azure DevOps, including code organization and shared utility standards across DE and DS teams. Led technical discovery workshops to translate domain requirements, mapped legacy data relationships across distributed systems

Architected a centralized Marketing - Retail Data Model in Databricks (PySpark, Spark SQL), resolving cross-department metric discrepancies and improving processing efficiency by 30%. Built star-schema Power BI semantic layers and deployed Power BI pipelines for agile releases, automating daily KPI report distribution to regional managers and executives. Implemented Row-Level Security for regional data compliance and provided self-service analytics training to non-technical stakeholders. Mentored junior analysts on data modeling best practices.