BenchmarkService class

Performance benchmark service

Constructors

BenchmarkService()

Properties

hashCode → int
The hash code for this object.
no setterinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited

Methods

noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited

Static Methods

aggregateRoundStats(List<DetailedBenchmarkStats> rounds) → DetailedBenchmarkStats
Aggregate multiple round stats by flattening all measured samples.
benchmark(String name, Future<void> fn(), {int iterations = 10, required BenchmarkCategory category}) → Future<BenchmarkResult>
Run multiple iterations and measure avg/min/max
benchmarkDetailed(Future<void> fn(), {int warmupIterations = 5, int measuredIterations = 30}) → Future<DetailedBenchmarkStats>
Run detailed benchmark with warmup and reproducible summary.
benchmarkEmbed(String text, {int iterations = 10}) → Future<BenchmarkResult>
Embedding generation benchmark (ONNX-powered)
benchmarkHybridSearch(String dbPath, String queryText, List<double> queryEmbedding, int docCount, {int iterations = 20}) → Future<BenchmarkResult>
Hybrid Search benchmark
benchmarkIngestFfiEntrypoints({int targetBytes = 1 * 1024 * 1024, int embeddingDim = 384, int maxChunkChars = 1500, int overlapChars = 100, int batchSize = 16, String? restoreDbPath, String? dbPathOverride}) → Future<IngestFfiEntrypointBenchResult>
Measure FFI text-byte traffic across the three IngestSession entrypoints on a deterministic document: the canonical prepareSourceIngestion (Dart String), prepareSourceIngestionFromUtf8 (bytes), and prepareSourceIngestionFromFile (path-only). Stub embeddings keep the measurement focused on FFI byte traffic.
benchmarkIngestFfiTraffic({int targetBytes = 1 * 1024 * 1024, int embeddingDim = 384, int maxChunkChars = 1500, int overlapChars = 100, int batchSize = 16, String? restoreDbPath, String? dbPathOverride}) → Future<IngestFfiBenchResult>
Measure FFI text-byte traffic for an addDocument ingest, comparing the legacy chain (addSourceInCollection + chunker + addChunks) against the IngestSession chain (prepareSourceIngestion + take_embedding_batch + commit_embeddings) on a deterministic document.
benchmarkIngestHeap({int targetBytes = 4 * 1024 * 1024, int embeddingDim = 384, int maxChunkChars = 1500, int overlapChars = 0, int batchSize = 16, String? restoreDbPath, String? dbPathOverride}) → Future<IngestHeapBenchResult>
Measure peak process RSS during each ingest entrypoint variant on the realistic caller pattern:
benchmarkIngestLatency({int targetBytes = 1 * 1024 * 1024, int embeddingDim = 384, int maxChunkChars = 1500, int overlapChars = 0, int batchSize = 16, int warmupRuns = 2, int measuredRuns = 5, String? restoreDbPath, String? dbPathOverride}) → Future<IngestLatencyBenchResult>
Measure wall-clock latency of the full prepare → drain → finalize → delete cycle across the three ingest entrypoint variants. Stub embeddings (zeroed Float32List) keep the measurement focused on the FFI + Rust pipeline cost rather than ONNX inference time.
benchmarkQueryPayload({int topK = 4, int adjacentChunks = 1, int previewMaxBytes = 24, int tokenBudget = 4000, String? restoreDbPath, String? dbPathOverride}) → Future<QueryPayloadBenchResult>
Measure current query/search payload shape without changing runtime behavior.
benchmarkScopedExactScan({int scopedChunkCount = 50, int distractorChunkCount = 10, int topK = 4, int chunkContentChars = 256, List<double> bm25Weights = const [0.0, 0.5], String? restoreDbPath, String? dbPathOverride}) → Future<ScopedExactScanBenchResult>
Measure the scoped exact-scan branch of searchMetaHybrid.
benchmarkSearch(String dbPath, List<double> queryEmbedding, int docCount, {int iterations = 20}) → Future<BenchmarkResult>
Search benchmark (Rust HNSW-powered)
benchmarkTokenize(String text, {int iterations = 50}) → Future<BenchmarkResult>
Tokenization benchmark (Rust-powered)
collectSamples(Future<void> fn(), {int warmupIterations = 5, int measuredIterations = 30}) → Future<List<double>>
Collect measured samples after warmup.
measureMs(Future<void> fn()) → Future<double>
Measure execution time of async code block
measureMsSync(void fn()) → double
Measure execution time of sync code block
runFullBenchmark({required String dbPath, dynamic onProgress(String)?}) → Future<List<BenchmarkResult>>
Run full benchmark suite
summarizeSamples(List<double> samplesMs, {required int warmupIterations}) → DetailedBenchmarkStats
Summarize measured samples with p50/p95/stddev.