Looks like you have JavaScript disabled. For the full Twine experience, you will need to re-enable it.

I’m Billy Macrae, an AI trainer and evaluator who designs complex, multi-step prompts to test frontier-level LLMs, including scenarios that require AI agents to operate across tools and platforms. I build and record demonstrations of task completion, use them to identify model failures and inconsistencies, and provide structured feedback to development teams in a sandboxed, realistic test environment.…I’m Billy Macrae, an AI trainer and evaluator who designs complex, multi-step prompts to test frontier-level LLMs, including scenarios that require AI agents to operate across tools and platforms. I build and record demonstrations of task completion, use them to identify model failures and inconsistencies, and provide structured feedback to development teams in a sandboxed, realistic test environment.

Billy Macrae

AI Trainer, AI Evaluator, Online Trainer, +2





I’m Billy Macrae, an AI trainer and evaluator who designs complex, multi-step prompts to test frontier-level LLMs, including scenarios that require AI agents to operate across tools and platforms. I build and record demonstrations of task completion, use them to identify model failures and inconsistencies, and provide structured feedback to development teams in a sandboxed, realistic test environment.…I’m Billy Macrae, an AI trainer and evaluator who designs complex, multi-step prompts to test frontier-level LLMs, including scenarios that require AI agents to operate across tools and platforms. I build and record demonstrations of task completion, use them to identify model failures and inconsistencies, and provide structured feedback to development teams in a sandboxed, realistic test environment.

Available to hire

I’m Billy Macrae, an AI trainer and evaluator who designs complex, multi-step prompts to test frontier-level LLMs, including scenarios that require AI agents to operate across tools and platforms. I build and record demonstrations of task completion, use them to identify model failures and inconsistencies, and provide structured feedback to development teams in a sandboxed, realistic test environment.

Skills

Experience Level

Expert

Expert

Expert

Expert

Expert

Intermediate

Intermediate

Work Experience

AI Voice contributor at Macgence

January 1, 2026 - Present

Contributed professionally recorded scripted speech used to create a realistic AI voice clone for automated answering services. Voice model licensed on an ongoing royalty basis.

Fleet AI — LLM Trainer / Evaluator at Fleet AI

January 1, 2026 - Present

Designed complex, multi-step prompts to test and challenge frontier-level LLMs, including tasks requiring AI agents to operate across multiple software applications such as Jira, financial platforms, and productivity tools. Created and recorded demonstrations of successful task completion to produce reference outcomes for evaluating model performance. Identified model failures, inconsistencies, and usability issues, and delivered structured feedback to the development team while collaborating with other trainers and freelancers via Slack. Worked within sandboxed software environments to develop realistic, challenging scenarios involving reasoning, application navigation, and accurate execution across multiple tools.

Professional freelance photographer (weddings, events, commercial branding, portraiture)

July 1, 2025 - Present

Specialised in wedding, event, commercial branding, and portrait photography. Completed paid voiceover work for podcasts and short films, pairing professional recording experience with strong verbal communication and script interpretation.

AI Voice contributor at Macgence

July 1, 2025 - Present

Provided paid voiceover work for phonecasts and short films, combining professional recording experience with strong verbal communication and script interpretation skills.

Systems Operations Manager at Mobilize Power Solutions

August 1, 2023 - July 1, 2025

Managed technical support and maintenance operations for B2B EV charging customers nationwide, handling a high-volume case load of support queries and in- and out-of-warranty maintenance projects. Designed and implemented internal backend processes using tools such as Miro and Microsoft PowerPoint, creating training materials and delivering onboarding sessions in person and virtually. Led end-to-end migration of company systems (CRM, project management, finance) to the Odoo ERP platform, including vendor research and selection, contract negotiation with a UK integrator, stakeholder coordination, and full process implementation.

Delivery Manager + Product Owner at Fuuse, Remote

August 1, 2022 - April 1, 2023

Acted as Product Owner for the BAU Scrum team, responsible for analysing and debugging the Fuuse CPMS platform. Worked closely with an Agile team of software developers (backend, frontend, and DevOps) to prioritise the case load effectively and deliver tasks in real time. Owned software bug resolution and ad hoc development requests, including white-labelling the platform for key account holders.

Team Leader, Network Assurance at Pod Point, London

November 1, 2020 - August 1, 2022

Started as a Technical Support Engineer supporting both B2B and B2C customers, then quickly promoted to B2C Team Leader. Managed an incoming case load of technical support requests, resolving issues efficiently and maintaining service quality. As Team Leader, facilitated daily stand-ups, performed quality assurance of team output, designed and improved systems, delivered training, and coordinated cross-functional communication with Product Support teams.

Digital Producer (Photos + VR) + Digital Surveyor at Pupil, London

August 1, 2017 - September 1, 2020

Worked at a spatial technology company producing photography, VR tours, and floor plans used as marketing materials on platforms such as Rightmove and Zoopla, and also used by larger organisations for surveying services. Delivered on-site services including photography, floor plan creation, and VR capture using technologies such as the Leic a BLK 360, Canon DSLR cameras, and mobile devices to capture and upload imagery via 4G. Progressed from digital surveying into the back office as a digital producer, then to team lead and subsequently QA supervision for stills and VR. Gained initial AI experience leading QA for an external image annotation team, reviewing and correcting labelled data, defining classification categories, and liaising with the external team to monitor progress for an AI training project focused on identifying architectural elements via bounding-box annotation tools.