ScenarioRun class

Orchestrates the evaluation of a single iteration of a scenario.

Call evaluate once with the model response to score, then call dispose (or use a try/finally) to persist the ScenarioRunResult to the EvaluationResultStore.

Annotations
  • @Source(name: 'ScenarioRun.cs', namespace: 'Microsoft.Extensions.AI.Evaluation.Reporting', repository: 'dotnet/extensions', path: 'src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/')

Constructors

ScenarioRun(String scenarioName, String iterationName, String executionName, Iterable<Evaluator> evaluators, EvaluationResultStore resultStore, {ChatConfiguration? chatConfiguration, EvaluationMetricInterpretation? evaluationMetricInterpreter(EvaluationMetric)?, ChatDetails? chatDetails, Iterable<String>? tags})
Creates a ScenarioRun.

Properties

chatConfiguration → ChatConfiguration?
Optional ChatConfiguration used by AI-based evaluators.
final
executionName → String
The execution name shared across all runs in one evaluation batch.
final
hashCode → int
The hash code for this object.
no setterinherited
iterationName → String
The name of the iteration within the scenario.
final
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
scenarioName → String
The name of the scenario.
final

Methods

dispose() → Future<void>
Writes the ScenarioRunResult to the EvaluationResultStore.
evaluate(Iterable<ChatMessage> messages, ChatResponse modelResponse, {Iterable<EvaluationContext>? additionalContext, CancellationToken? cancellationToken}) → Future<EvaluationResult>
Evaluates modelResponse against all configured evaluators and returns the aggregated EvaluationResult.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited