testing library
Test helpers for the catalog the generator produced.
A generated CatalogItem is a contract with a model: these components
exist, they take these properties, and this is what the app does with them.
The schema half of that contract is checked at build time. This library
checks the other half — what the rendered component exposes to the person
using it — by recording the semantics of each item's example and failing
when they change.
The recorded file is the same shape A2UI's rendering cases are written in, so it doubles as the answer to "what can the model make this app announce", which is not visible from a Dart diff.
// test/genui_semantics_test.dart
import 'dart:io';
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:genui_gen/testing.dart';
import 'package:my_app/genui_catalog.g.dart';
void main() {
testWidgets('the catalog exposes what it did before', (tester) async {
final recorded = <String, List<GenUiSemanticNode>>{};
for (final item in genUiCatalog.items) {
final handle = tester.ensureSemantics();
await tester.pumpWidget(
MaterialApp(
home: Scaffold(
body: GenUiExampleSurface(catalog: genUiCatalog, item: item),
),
),
);
await tester.pumpAndSettle();
recorded[item.name] = genUiRenderedSemantics();
handle.dispose();
}
expect(
genUiSemanticsGolden(recorded, File('test/genui_semantics.json')),
isNull,
);
});
}
genUiSemanticsAudit reads the same recording and reports what a person using a screen reader could not work with: a control with no name, a component that reaches assistive technology as nothing at all, two buttons that announce themselves identically. genUiCatalogWeight answers a different question with the same catalog: how much of every prompt each component takes up.
genUiCatalogDiff answers the other half of the question, about the contract rather than the rendering: what changed for the model between two versions of the catalog, and whether a message composed against the old one can still be wrong.
Record the file the first time, and after a deliberate change, by setting
GENUI_UPDATE_GOLDENS=1:
GENUI_UPDATE_GOLDENS=1 flutter test test/genui_semantics_test.dart
Classes
- GenUiAuditFinding
- One thing the audit found.
- GenUiCatalogChange
- One difference between two catalogs.
- GenUiCatalogWeight
- How much of every prompt one component takes up.
- GenUiCoverage
- What a corpus of recorded sessions says the agent actually used.
- GenUiExampleSurface
- Renders one catalog item's generated example, through a real surface.
- GenUiFuzzFinding
- One thing a fuzz run found.
- GenUiSemanticNode
- One node of what a surface exposes to assistive technology, in the shape A2UI's rendering cases are written in.
Enums
- GenUiAuditRule
- Something a recorded catalog exposes that a person using assistive technology cannot work with.
- GenUiCatalogChangeKind
- What changed between two versions of a catalog, seen from where it matters: the model that composes against it.
- GenUiFuzzKind
- Why a case is being reported.
Functions
-
genUiCatalogDiff(
Map< String, Object?> before, Map<String, Object?> after) → List<GenUiCatalogChange> - Every difference between two catalog documents, in the order a reader wants them: breaking first, then by where.
-
genUiCatalogWeight(
Map< String, Object?> catalogJson) → GenUiCatalogWeight -
Measures what
catalogJsoncosts, component by component. -
genUiCoverage(
{required Catalog catalog, required Iterable< GenUiTrace> traces}) → GenUiCoverage -
Reads
tracesand reports what the agent did withcatalog. -
genUiFuzz(
{required Catalog catalog, required GenUiPump pump, Set< String> only = const <String>{}, Set<String> skip = const <String>{}, int maxCasesPerComponent = 80, bool reportEmpty = false}) → Future<List< GenUiFuzzFinding> > -
Renders everything
catalog's own examples allow, mutated, and reports what broke. -
genUiFuzzSummary(
List< GenUiFuzzFinding> findings) → String -
A table of
findings, one line per component, for a CI log. -
genUiRenderedSemantics(
) → List< GenUiSemanticNode> - The meaningful semantics of what is currently on screen.
-
genUiSemantics(
SemanticsNode root) → List< GenUiSemanticNode> -
The meaningful part of
root, in traversal order. -
genUiSemanticsAudit(
Map< String, List< recorded, {Set<GenUiSemanticNode> >String> allowEmpty = const <String>{}}) → List<GenUiAuditFinding> - Reads a recording made by genUiSemantics and reports what a person using a screen reader could not work with.
-
genUiSemanticsDiff(
List< GenUiSemanticNode> expected, List<GenUiSemanticNode> actual) → String? -
Describes how
actualdiffers fromexpected, ornullwhen it does not. -
genUiSemanticsGolden(
Map< String, List< recorded, File golden, {bool? update}) → String?GenUiSemanticNode> > -
Compares
recordedagainstgolden, and returns what differs, ornullwhen nothing does.