HumanGrader class abstract
Anthropic Step 5 / 8: human graders are gold-standard for subjective dimensions, used both for direct scoring and for calibrating LLM judges.
Concrete implementations connect to an application-provided HumanReviewQueue. The default grade flow is:
- Push the trial to the queue (non-blocking).
- Poll for a verdict; if absent, return a pending
Score(value: null).
- Implemented types
Constructors
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- kind → GraderKind
-
Anthropic kind: code / model / human.
no setteroverride
- name → String
-
Stable name. Used as score key in reports.
no setterinherited
- passThreshold → double
-
If a Score.value meets or exceeds this threshold, Score.passed is
true. Defaults to1.0(binary) but graders may override to support partial credit thresholds.no setteroverride - queue → HumanReviewQueue
-
Where to enqueue trials and where to read back human verdicts.
no setter
- rubric → String?
-
Optional rubric prompt shown to human reviewers in the UI.
no setter
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
grade(
{required Trial trial, required Transcript transcript, required Outcome outcome, required EvalContext context, ReferenceSolution? referenceSolution}) → Future< Score> -
Compute a score for one trial.
override
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited