Grader class abstract
A grader scores some aspect of an agent's performance for one trial.
Implementations should inspect the artifact that contains the evidence for that aspect: Outcome for final environment state, Transcript for the agent's path, tool calls, messages, events, and metrics.
- Implementers
Constructors
- Grader()
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- kind → GraderKind
-
Anthropic kind: code / model / human.
no setter
- name → String
-
Stable name. Used as score key in reports.
no setter
- passThreshold → double
-
If a Score.value meets or exceeds this threshold, Score.passed is
true. Defaults to1.0(binary) but graders may override to support partial credit thresholds.no setter - runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
grade(
{required Trial trial, required Transcript transcript, required Outcome outcome, required EvalContext context, ReferenceSolution? referenceSolution}) → Future< Score> - Compute a score for one trial.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited