TradeGuard AI — LLM Evaluation Framework for Capital Markets Trade Surveillance

An open-source AI testing portfolio project that evaluates a RAG-based trade surveillance agent for detecting wash trading, layering, and spoofing in Capital Markets. Built end-to-end in Python with a fully automated CI/CD evaluation pipeline.
● Designed and implemented a Python-based LLM evaluation framework using DeepEval metrics (AnswerRelevancyMetric, FaithfulnessMetric, ContextualPrecisionMetric, GEval) to assess RAG pipeline quality for financial compliance use cases.
● Built RAGAS evaluation suite to measure retrieval augmentation quality across trade surveillance query scenarios including context recall, answer correctness, and context relevance.
● Integrated evaluation pipeline into GitHub Actions CI/CD; automated DeepEval and RAGAS test runs on each commit with threshold-based quality gates and custom HTML reporting.
● Stack: Python 3, pytest, DeepEval, RAGAS, Ollama (local), OpenAI GPT-4o, GitHub Actions | Domain: Capital Markets, AML, Wash Trading, Spoofing, MiFID II

Git Repo: https://www.twine.net/signin