AI Linguist and Prompt Engineering Specialist focused on editorial judgment, data annotation design, and prompt-driven workflows for generative AI systems. Experienced in evaluating model outputs, resolving ambiguity, and translating insights into clear guidelines and evaluation frameworks.\n\nCollaborative, analytical, and comfortable iterating quickly in ambiguous environments.

Avani Upadhyaya

AI Linguist and Prompt Engineering Specialist focused on editorial judgment, data annotation design, and prompt-driven workflows for generative AI systems. Experienced in evaluating model outputs, resolving ambiguity, and translating insights into clear guidelines and evaluation frameworks.\n\nCollaborative, analytical, and comfortable iterating quickly in ambiguous environments.

Available to hire

AI Linguist and Prompt Engineering Specialist focused on editorial judgment, data annotation design, and prompt-driven workflows for generative AI systems. Experienced in evaluating model outputs, resolving ambiguity, and translating insights into clear guidelines and evaluation frameworks.\n\nCollaborative, analytical, and comfortable iterating quickly in ambiguous environments.

See more

Language

English
Fluent

Work Experience

AI Experience & Quality Lead (Alexa+) at Amazon
November 1, 2018 - Present
Owned evaluation of large-scale generative and conversational AI systems serving millions of users, with a focus on response quality, consistency, ambiguity resolution, and editorial soundness across real-world workflows. Established and applied structured evaluation frameworks and human-in-the-loop review processes to assess model outputs, surface failure modes, and guide iteration and launch readiness decisions. Acted as a core partner to applied science, product management, and engineering teams, translating qualitative model behavior into clear evaluation criteria, guardrails, and prompt-level recommendations. Reviewed and validated changes across prompts, orchestration logic, and downstream dependencies to ensure predictable, high-quality AI behavior under production constraints. Evaluated model reliability across edge cases, fallback scenarios, and multimodal surfaces, generating insights that directly influenced prompt strategy, evaluation coverage, and go/no-go launch decisions
AI Linguist & Model Evaluation Specialist at Independent Contract with Clients - Google, Meta, LinkedIn
September 1, 2022 - Present
Led design and evaluation of natural language annotations, prompt frameworks, and multi-turn conversational behaviors for generative AI systems supporting editorial, knowledge, and learning-focused use cases. Authored and refined structured system prompts incorporating editorial intent, task decomposition, few-shot prompting techniques, and guardrails to simulate production-grade AI behavior and inform model iteration. Analyzed large-scale annotation results to assess data quality, consistency, and inter-annotator agreement, informing classifier evaluation and prompt iteration decisions. Served as an evaluation lead for generative AI output quality, consistency, ambiguity handling, and alignment with editorial standards, providing clear, actionable feedback to guide refinement and deployment readiness.
Operations Analyst (Part-time) at Target
May 1, 2018 - November 1, 2018
Built and automated operational dashboards and reports to support inventory planning and workforce scheduling, improving data accuracy and reducing manual effort. Partnered with operations stakeholders to analyze trends and deliver insights that improved planning efficiency and day-to-day decision-making.
Native Language Tutor (Part-time) at Framingham Public Schools
October 1, 2016 - March 1, 2017
Facilitated effective communication between students, educators, and parents with limited English proficiency to support academic engagement. Collaborated with educators to adapt communication strategies for diverse cultural backgrounds and classroom needs.
AI Linguist & Model Evaluation Specialist at Independent Contract with Clients (Google, Meta, LinkedIn)
September 1, 2022 - Present
Led design and evaluation of natural language annotations, prompt frameworks, and multi-turn conversational behaviors for generative AI systems supporting editorial, knowledge, and learning-focused use cases. Authored and refined structured system prompts incorporating editorial intent, task decomposition, few-shot prompting techniques, and guardrails to simulate production-grade AI behavior and inform model iteration. Analyzed large-scale annotation results to assess data quality, consistency, and inter-annotator agreement, informing classifier evaluation and prompt iteration decisions. Served as an evaluation lead for generative AI output quality, consistency, ambiguity handling, and alignment with editorial standards, providing clear, actionable feedback to guide refinement and deployment readiness. Developed and applied evaluation rubrics, side-by-side review methodologies, and golden response frameworks to assess generative AI quality, reliability, and classifier performance. Advi

Education

Bachelor of Engineering, Information Technology at University of Mumbai
January 11, 2030 - February 12, 2026
Diploma in Computer Engineering at University of Mumbai
January 11, 2030 - February 12, 2026
Bachelor of Engineering, Information Technology at University of Mumbai - India
January 11, 2030 - February 12, 2026
Diploma in Computer Engineering at University of Mumbai - India
January 11, 2030 - February 12, 2026

Qualifications

Add your qualifications or awards here.

Industry Experience

Software & Internet, Professional Services, Education, Media & Entertainment