webweavex 2.1.0

webweavex: ^2.1.0 copied to clipboard

Deterministic runtime cognition infrastructure
for humans and AI agents

Contents #

• Overview · Why WebWeaveX · Features
• Architecture · Installation · Quick Start
• Subsystems: Extraction · Runtime · Memory · Replay · Reconstruction · Workflows · Graph · Determinism
• API Reference · Examples · Performance
• Testing · Coverage · CI/CD · Pub.dev Release
• Contributing · OSS Governance · Security · Roadmap · Vision

Overview #

WebWeaveX is deterministic runtime cognition infrastructure for humans and AI agents to understand, continue, reconstruct, replay, and reason about authenticated operational software systems.

This dart branch is the native pub.flutter-io.cn implementation — not a scraper, not an LLM wrapper, not AGI hype. It is byte-for-byte parity-aligned with the Python (PyPI) and JavaScript (npm) implementations at the cryptographic and deterministic-serialization layers.

Spec: CANONICAL_RUNTIME_SPEC.md · Matrix: ECOSYSTEM_MATRIX.md

Humans and AI agents #

Why WebWeaveX #

Traditional tools capture HTML, not operational runtime state. WebWeaveX provides canonical serialization, Kaalka-sealed sessions, replay equivalence, and reconstruction identities so that how software runs — not just what HTML was returned — becomes a first-class, reproducible artifact.

What WebWeaveX is NOT #

Features #

• Deterministic core — normalizeRuntimeValue → stableSerialize → UTF-8 → Kaalka v5 → base64, byte-identical to Python and JavaScript (computeDeterministicHash produces matching hashes).
• Runtime graphs — sorted, fingerprinted node/edge graphs (buildRuntimeGraph, graphFingerprint).
• Runtime memory fabric — build, query, search, and lineage-track operational memory.
• Replay equivalence — prove two runtime envelopes are operationally identical.
• Runtime reconstruction — rebuild deterministic runtime identities from extraction envelopes.
• Authenticated continuation — Kaalka-encrypted session save/load (you supply credentials).
• 12 runtime-cognition families — causality, semantic, synchronization, evolution, workflows, execution, memory-runtime, reconstruction-runtime, persistence, connectors, query, kernel/IR.
• Cross-language parity vectors — 11/11 core + ~145 runtime-API hash vectors under validation/.

Architecture #

Input → Canonical pipeline → Graph + Memory → Replay check → Reconstruction
              ↓
     Normalization + Kaalka v5 (pub.flutter-io.cn kaalka)

Layered source layout (lib/src/):

Installation #

dart pub add webweavex

or add to pubspec.yaml:

dependencies:
  webweavex: ^2.1.0
  kaalka: ^5.0.0

Requires Dart SDK >=3.3.0 <4.0.0.

Quick Start #

import 'package:webweavex/webweavex.dart';

Future<void> main() async {
  final hash = computeDeterministicHash({'status': 'ok'});
  final pipeline = await runCanonicalPipeline({
    'url': 'https://example.com',
    'sourceType': 'web',
  });
  print('$hash ${pipeline['bounded']}');
}

Core Capabilities #

WebWeaveX exposes one deterministic engine across six cognition domains. Every output is a bounded, hashable, replayable IR.

Common Workflows #

import 'package:webweavex/webweavex.dart';

Future<void> main() async {
  // Extract structured content
  final content = await extractWeb('https://example.com');

  // Analyze documents
  final docs = queryDocuments(text: '...document text...');

  // Analyze repositories
  final repo = queryRepository(source: 'my-project');

  // Query semantic IR
  final semantics = querySemantics('entities', content);

  // Runtime reasoning
  final pipeline = await runCanonicalPipeline({
    'url': 'https://example.com',
    'sourceType': 'web',
  });
  print(pipeline['pipeline_hash']);

  // Application cognition
  final app = runApplicationCognition('https://app.example.com', '<html>...</html>');
}

Supported Platforms #

Versioning #

WebWeaveX follows Semantic Versioning — MAJOR.MINOR.PATCH. The version is synchronized across all three implementations: pub.flutter-io.cn, PyPI, and npm share the same 2.1.0, so a given version number denotes the same certified deterministic contract in every language. MAJOR marks a breaking change, MINOR adds backward-compatible capability, PATCH is a fix. The crypto substrate pin (kaalka 5.0.0) is independent of the package version.

Extraction systems #

WebWeaveX models extraction as a bounded, deterministic operation over provided or fetched input — never an unbounded crawl. The browser layer (extractWeb, renderPage, captureRuntime) operates over a bounded HTTP surface; live-browser-only capabilities (infinite scroll, DevTools frames) are documented as platform-deferred in API_REFERENCE.md.

final result = await extractWeb('https://example.com');
print(result['kind']);          // extraction kind
print(result['deterministic_hash']);

Runtime systems #

final graph = buildRuntimeGraph({'session': {'authenticated': true}});
final runtime = {'unified_runtime_graph': graph.toJson()};

Runtime-cognition families each expose run_*, save_*, load_*, and replay_* entry points with proven cross-language hash parity (causality, semantic, synchronization, evolution, workflows, execution).

Memory systems #

Canonical, Python-aligned API:

final memory = buildRuntimeMemory(
  runtimeHistory: [{'tick': 1, 'kind': 'workflow'}],
  lineage: [{'id': 'a'}],
  semanticRelations: [{'from': 'a', 'to': 'b'}],
);
final found = queryRuntimeMemory(memory, 'semantic', 'a'); // {results, count, ...}
final lineage = buildMemoryLineage(memory);

Graph-based memory fabric (Dart-native helper):

final graph = buildRuntimeGraph({'session': {'authenticated': true}});
final fabric = buildRuntimeMemoryFabric(graph);
final slice = queryRuntimeMemoryFabric(fabric, 'graph');

Replay systems #

final report = validateReplayEquivalence(envelope, envelopeClone);
print(report['equivalent']); // true when checks pass

Checks: graph hash, global fingerprint, browser identity, DOM hash (when present), memory stable hash (when present).

Reconstruction systems #

final rebuilt = reconstructRuntime(extraction: envelope);
print(rebuilt['runtime_id']);

Workflows #

final plan = buildWorkflowPlan({'objective': 'extract-and-verify'});
final run = runAutonomousWorkflow(plan);
final replay = replayWorkflowRuntime(run);

Graph intelligence #

Runtime graphs are deterministically sorted and fingerprinted so identical operational state yields identical identity:

final graph = buildRuntimeGraph({'agent_step': 'observe'});
final fp = graphFingerprint(graph);
final replayed = replayRuntimeGraph(graph.toJson());

Deterministic systems #

normalizeRuntimeValue → stableSerialize → UTF-8 → deriveKaalkaTimeKey → kaalka._proc → base64

11/11 core vectors match the JavaScript reference:

dart run validation/validate_parity.dart   # crossLangMatch: true

API Reference #

The public barrel (package:webweavex/webweavex.dart) re-exports 53 family modules (~372 public functions, 7 classes). Grouped by family:

Full per-API parity classification (Complete / Partial / Deferred): PUBLIC_API_MATRIX.md · API_REFERENCE.md.

Examples #

Runnable programs live in example/. AI-agent continuity pattern:

final graph = buildRuntimeGraph({'agent_step': 'observe'});
final fabric = buildRuntimeMemoryFabric(graph);
final agentView = queryRuntimeMemoryFabric(fabric, 'graph');
final continuity = encryptValue({'checkpoint': agentView}, 'agent-session-key');

Authenticated continuation (you supply authorized credentials — no bypass tooling):

saveAuthenticatedRuntime('./session.json', {'cookies': []}, 'your-key');
final result = await extractWeb('https://example.com',
    authenticated: true, sessionPath: './session.json', encryptionKey: 'your-key');

Performance #

WebWeaveX is CPU-bound deterministic serialization + hashing; there is no network in the core path. Typical operations (graph build, fingerprint, hash, replay-equivalence) complete in sub-millisecond to low-millisecond time on commodity hardware. The full 1,583-test suite runs in ~54 s including coverage instrumentation. No allocation-heavy hot loops; List.sort uses index-tiebreak comparators to match Python's stable sorted without extra passes.

Testing #

dart test

1,583 tests across crypto, determinism, graph, replay, memory, reconstruction, kernel, browser, connectors, selector-healing, interaction-replay, and the 12 ported runtime families (test/parity/, test/engines/). See TEST_INVENTORY.md and TEST_VALIDATION_REPORT.md.

Coverage #

dart test --coverage=coverage
dart pub global run coverage:format_coverage --lcov --in=coverage \
  --out=coverage/lcov.info --report-on=lib --packages=.dart_tool/package_config.json

97.26% line coverage (6394/6574). One file (normalization.dart, 85.71%) carries a single unreachable Node-fallback line. Details: COVERAGE_VALIDATION_REPORT.md.

CI/CD #

GitHub Actions (.github/workflows/dart.yml, .github/workflows/ci.yml) gate every push:

See CI_VALIDATION_REPORT.md.

Pub.dev Release #

Release readiness: RELEASE_READINESS_REPORT.md.

Contributing #

Contributions are welcome. Before opening a PR, run the full gate sequence:

dart format --set-exit-if-changed .
dart analyze
dart test
dart run validation/validate_parity.dart
dart pub publish --dry-run

Any new public API must ship with a cross-language hash-parity vector and test — parity is proven, never assumed. See CONTRIBUTING.md and CODE_OF_CONDUCT.md.

OSS Governance #

Security #

Authorized session material only — no credential cracking, no CAPTCHA/auth bypass. Report vulnerabilities per SECURITY.md.

Roadmap #

See ROADMAP.md. Near-term: widen bounded extraction parity, expand examples and benchmarks, deepen the semantic/query sub-path coverage toward Complete.

Vision #

WebWeaveX aims to make operational runtime cognition a portable, deterministic, cross-language substrate — so that any human or AI agent, in Python, JavaScript, or Dart, can capture, replay, reconstruct, and reason about authenticated software systems with identical, verifiable results. Determinism is the contract; parity across languages is the proof.

Cross-language parity & certification #

Three implementations — Python (canonical, PyPI), JavaScript (npm), Dart (pub.flutter-io.cn) — byte-identical on the certified surface. Every claim is regenerated by execution; nothing passes on the strength of a report:

The full model, reproduction commands, and current verdict: CERTIFICATION.md and final_certification.json. Per-API status: API_REFERENCE.md (generated from PARITY_MANIFEST.json).

AI-agent usage #

WebWeaveX is built to be operated by AI agents as much as by humans: every output is a bounded, deterministic, evidence-carrying IR that an agent can hash, diff, replay, and reason over without screenshots or DOM diffing. Agents contributing to the codebase should start at AI_AGENT_GUIDE.md — architecture map, determinism rules, the cross-language pitfalls catalogue, and the certification workflow.

Limitations #

• Network/live-browser APIs are Partial by design — the extract* / crawl* family and five bounded APIs have certified deterministic cores; their live side effects are out of certification scope.
• Platform-bound APIs are Deferred — live-page capture and OS-coupled native cognition (extract_native, run_native_cognition) branch on the host OS even in Python and cannot be made cross-platform-deterministic.
• Valid-Python AST enrichment is Python-only — the certified JS/Dart contract takes the parse-error fallback for parse_ast on valid Python (see ARCHITECTURE.md, "The AST contract").

License #

Apache 2.0 — LICENSE

WebWeaveX is deterministic runtime cognition infrastructure — not a disposable scraper.