I'm an AI quality and evaluation operations leader focused on transcript AI, enterprise GenAI safety, and search relevance for decision-grade research workflows. I design and scale evaluation programs that translate complex model behavior into actionable requirements for product and policy teams. I build rubrics, gold sets, calibration cadences, adjudication workflows, error taxonomies, and root-cause reporting that drive measurable product improvements. I champion transparent evidence-traceability and clear QA gates to ensure responsible AI outcomes across cross-functional teams.

Nicholas Yanes, PhD

I'm an AI quality and evaluation operations leader focused on transcript AI, enterprise GenAI safety, and search relevance for decision-grade research workflows. I design and scale evaluation programs that translate complex model behavior into actionable requirements for product and policy teams. I build rubrics, gold sets, calibration cadences, adjudication workflows, error taxonomies, and root-cause reporting that drive measurable product improvements. I champion transparent evidence-traceability and clear QA gates to ensure responsible AI outcomes across cross-functional teams.

Available to hire

I’m an AI quality and evaluation operations leader focused on transcript AI, enterprise GenAI safety, and search relevance for decision-grade research workflows. I design and scale evaluation programs that translate complex model behavior into actionable requirements for product and policy teams.

I build rubrics, gold sets, calibration cadences, adjudication workflows, error taxonomies, and root-cause reporting that drive measurable product improvements. I champion transparent evidence-traceability and clear QA gates to ensure responsible AI outcomes across cross-functional teams.

See more

Language

English
Fluent

Work Experience

AI Tutor (LLM Evaluator) at XAI
November 1, 2024 - January 1, 2026
Led human evaluation workstreams for LLM behavior, focusing on grounded summarization, omission detection, and rubric-based scoring. Evaluated transcript-style Q&A and long-form summaries for completeness, attribution integrity, and omission risk in compliance-sensitive workflows. Designed evidence-traceability checks to reduce unsupported claims and improve decision integrity in research outputs. Built scalable evaluation ops: rubrics, gold sets, calibration cadence, adjudication paths, and reviewer coaching to improve inter-rater consistency. Developed failure-mode taxonomies (hallucinations, omissions, misattribution) and delivered stakeholder readouts that drove quality improvements. Led vibe coding workshops to prototype transcript synthesis and QA templates, then translated prototypes into SOPs for repeatable execution.
AI Trust & Safety Quality Assessment Lead at Tech Mahindra (Google Contractor)
February 1, 2023 - November 1, 2024
Led and trained a 20-person global QA team evaluating generative-search and Gemini outputs; built onboarding modules and weekly calibrations to maintain consistent quality. Authored scoring rubrics, guidelines, and playbooks; ran adjudication loops with regional leads to resolve edge cases and reduce rework. Built reporting dashboards and weekly readouts for program leadership, tracking volume, error categories, and root-cause themes across 5,000+ evaluated model outputs and 1,000+ datasets. Partnered with engineering, product, and policy stakeholders to translate evaluator findings into actionable tickets and workflow changes; supported rapid iteration on quality and safety issues. Ran escalation pathways for high-risk content and policy edge cases; maintained defensible QA documentation suitable for audit and compliance review.
Mergers and Acquisitions Researcher / Writer at Corum Group, Ltd.
December 1, 2020 - August 1, 2023
Produced decision-ready briefs for deal teams by synthesizing customer signals, financial drivers, and competitive context. Checked AI/ML targets for data use, privacy, and IP risks; documented issues and mitigations for stakeholders.
Associate, Strategic Communications and Public Relations (Contractor) at WIT Strategy
April 1, 2022 - September 1, 2022
Developed thought-leadership content and outreach assets for MarTech and gaming clients (briefs, bylines, press releases, and LinkedIn content). Turned messy inputs into structured narratives and partner-ready materials under tight deadlines.
Project Manager, Co-Editor, Contributor at Hannibal for Dinner (Book Published by McFarland)
January 1, 2018 - February 1, 2021
Created publication proposal with market and demographic analysis, managed contracts, recruited and managed 16 contributors, copyedited contributed articles, and created publication roadmap.
Editor, Analyst, Journalist at Casual Games Association
July 1, 2012 - July 1, 2015
Wrote and edited articles for CGA magazine and website; recruited and monitored progress of contributors; managed writers and editors to maintain publication calendars.

Education

Ph.D. at The University of Iowa
January 11, 2030 - January 27, 2026
M.A. at Florida State University
January 11, 2030 - January 27, 2026
B.A. at Florida Atlantic University, Harriet L. Wilkes Honors College
January 11, 2030 - January 27, 2026

Qualifications

Data Security Posture Management (DSPM) Certification
January 1, 2026 - January 27, 2026
AI Security & Governance Certification
January 1, 2026 - January 27, 2026
NCMEC CONNECT certificates (issued 2025): Someone Disclosed to Me - Now What?; Sextortion; NCMEC Programs for Missing Children; Understanding Why Youth Run
January 1, 2025 - January 27, 2026
Cybersecurity Fundamentals (IBM SkillsBuild)
January 1, 2025 - January 27, 2026
Academic Applications of Artificial Intelligence (AAAI) Micro-credential, San Diego State University
January 1, 2025 - January 27, 2026
Google Tech Certifications: Attention Mechanism, Encoder-Decoder Architecture, Introduction to Image Generation, Introduction to Generative AI Studio, Create Image Captioning Models, Transformer and BERT Models, Generative AI Fundamentals, Introduction to
January 1, 2025 - January 27, 2026

Industry Experience

Media & Entertainment, Professional Services, Software & Internet, Education