AuraOne Human Data - Music Audio Expert Greek

Client: AuraOne Human Data

Location: Remote

Contract: undefined

Job Description

The Music Audio Expert - Greek is a remote evaluation role focused on reviewing Greek generalist evaluation prompts and responses in accordance with AuraOne’s quality rubric. Reviewers are responsible for comparing outputs, labeling edge cases, and providing structured feedback for the modeling team to enhance training.

Requirements

The requirements for the role include:

• Prior evaluation, annotation, or human-rater experience in Greek generalist evaluation or related content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning that identifies issues and the relevant rubric clause.
• Strong attention to detail, especially in identifying problematic prompts.
• Availability for at least 10 hours per week asynchronously.

Responsibilities

• Evaluate Greek generalist evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and determine the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and relay them to the program team for rubric updates.
• Maintain reviewer-quality scores through weekly calibration against gold-standard examples.
• Document recurring failure modes for the modeling team to address in future training sessions.

Example Tasks

• Compare two Greek generalist evaluation model responses and select the better response with rationale.
• Tag unsafe responses correctly with the appropriate policy category and severity.
• Audit a 50-row batch for rubric consistency and report findings to the program lead.
• Suggest clarifications to the rubric upon identifying recurring issues.

Nice to Have

• Background in linguistics, content moderation, or trust & safety review.
• Experience with inter-rater agreement metrics and calibration cycles.
• Domain expertise for identifying subject-matter errors that automated checks may miss.

Skills

• Model output evaluation
• Rubric-based annotation
• Severity tagging
• Inter-rater calibration
• Greek generalist evaluation

Work Model

This is a remote position available to US-eligible candidates. The role is for an independent specialist contractor and employment type is CONTRACTOR. Applicants must be authorized to work in the US.

Compensation

The offered compensation for this role ranges from $12 to $22 per hour.

Application Process

Interested candidates should apply through AuraOne’s specialist intake for specific routing and review. The final project scope, schedule, and contractor terms will be confirmed before placement.