Nesh Khazeni

Freelance AI/LLM evaluation across the LLM lifecycle: model evaluation & benchmarking (correctness, factuality, groundedness, safety, helpfulness), LLM-as-a-judge and rubric-based human evaluation. Evaluated Microsoft Copilot outputs using evaluation logs and response trajectories, surfacing failure patterns and regressions. Performed agent-trajectory analysis for multi-step reasoning and tool/function calling (including MCP replicas), labeling step-by-step correctness and pinpointing drift/loops/misuse. Produced/curated RLHF/DPO and SFT/instruction-tuning datasets with quality control and inter-annotator agreement. Built human-in-the-loop annotation workflows at scale and executed safety red teaming for prompt injection, jailbreaks, and policy compliance. Automated scoring/metric aggregation and reporting in Python, using Hugging Face and OpenAI/Azure OpenAI/Anthropic APIs, with experiment tracking in Weights & Biases/MLflow.

Built and operated energy-market data ETL pipelines ingesting and transforming power/gas market data from 20+ external providers and exchanges (EPEX SPOT, Nord Pool), landing curated datasets into Snowflake. Designed scheduled .NET worker/console jobs (C#, .NET 8) with retry/backoff, idempotency, and watermarking for safe reruns and correct reconciliation of late-arriving data. Implemented a parallel Python/Prefect extraction track for productionized, monitored workflows from notebook prototypes. Modeled Snowflake with staging/curated/consumption layers and tuned loads (clustering/SQL transformations) for performance and cost. Deployed and operated extraction services on Azure Service Fabric for resilient scheduling and self-healing behavior. Instrumented end-to-end data freshness monitoring with Datadog (metrics, logs, dashboards, alerts). Implemented data validation, reconciliation, and completeness checks, documented lineage and on-call runbooks, and integrated TSO/grid data with

Built internal AI-assisted tools used daily by hundreds of users across backend microservices, an AI/LLM integration layer, and a Blazor/Angular UI. Implemented a Retrieval-Augmented Generation service in C# wrapping Azure OpenAI (GPT-4o), exposing a Web API for knowledge-base querying and cutting average knowledge-search time by ~40%. Engineered prompt templates, function-calling schemas, token-usage controls, and NLP preprocessing so product owners could add document sources without code changes. Implemented ASP.NET Core microservices with clean architecture and async/await, with xUnit/NUnit/Moq tests and quality gates via SonarQube. Built Blazor components for admin workflows and reusable Angular modules for end-user chat. Led migration from on-prem Windows-hosted applications to GitHub Enterprise Cloud by rewriting Azure Pipelines as GitHub Actions, containerizing with Docker, and shipping through Azure DevOps and Kubernetes. Shared GitHub Copilot agent-mode workflows with the te