cera_ffi

Dart bindings for the Cera on-device inference engine. Runs GGUF language models locally through dart:ffi.

The 0.6.2 API includes chat lifecycle, streaming, checkpoint validation, schema and audio corrections. See the 0.6 API guide for contracts and compatibility limits, and Releases for published builds.

Building a Flutter app? Use cera_ffi_flutter instead. It depends on this package and re-exports it, so the API is identical, and it adds the per-platform build wiring that ships the native library with your app. This package on its own does not carry a native library.

This one exists for Dart without Flutter: a CLI, a server, a test. It is also where the bindings live so that they can be resolved without Flutter, since a package declaring flutter.plugin.platforms is one dart pub get refuses.

Native ChatSession.recoveryStatus() and Session.recoveryStatus() expose usability and the retained recovery report, including typed rewind/reset errors. Read it after a failed ingestion or append call; Reset requires context replay and Unusable requires a successful reset or recreation. Legacy sendMessage is deprecated in favor of ChatSession (session.intoChat()).

ChatSession provides reactive streaming through standard Dart Stream<String>:

final chat = session.intoChat();
chat.ingest(chatMessageUser('Hello!'));

// Stream text tokens reactively
await for (final chunk in chat.stream(opts)) {
  stdout.write(chunk);
}

// Start a separate turn from a clean conversation for this JSON example.
chat.reset();
chat.ingest(chatMessageUser('Return the answer as JSON.'));
await for (final chunk in chat.streamJson(opts, schemaJson)) {
  stdout.write(chunk);
}

Both calls emit text fragments, not individual tokens. A stream ending normally does not guarantee SessionPhase.turnComplete: a token limit can leave SessionPhase.interrupted, while no-progress calls can preserve promptReady for retry. Check chat.phase() before ingesting the next user turn; reset or replace messages after interruption. Let an in-flight native operation finish before reusing its handle; diagnostic queries can return Busy. JSON streaming enforces only the supported subset, and a token limit can still truncate the JSON value.

See the recovery contract and the conversational chat example.

Explicit loading in this checkout

The regenerated native bindings also expose ModelLoader, ModelSource and GenerativeModel. The complete raw completion example loads a local GGUF and closes every native handle. With a matching library built from this branch, run from cera_ffi/:

CERA_FFI_LIB=/absolute/path/to/libcera_ffi.dylib \
  dart run example/explicit_loading.dart /absolute/path/to/model.gguf "The capital of France is"

This synchronous path belongs on a worker in UI applications. Use matching bindings and native libraries from this checkout when testing 0.6.2; a local version bump does not publish a package. The portable async Cera facade retains its existing API. The native loader's web stubs do not load models in a browser.

Native CPU Session and Chat provide exportCheckpoint / importCheckpoint and file save/load methods. Native Metal/wgpu checkpoints are rejected; the separate browser WebGPU API owns its own device snapshot path. Older f16/TurboQuant snapshots need recreation after the compression fingerprint change. See checkpoint compatibility.

Install

dart pub add cera_ffi

Then point it at a cera-ffi shared library, since nothing bundles one for you:

# built from the repo: cargo build -p cera-ffi --release --features ffi-buffer
export CERA_FFI_LIB=/path/to/libcera_ffi.dylib

Without CERA_FFI_LIB the loader falls back to the platform's normal search path (libcera_ffi.so, libcera_ffi.dylib, cera_ffi.dll).

The ffi-buffer cargo feature is not optional. UniFFI emits the uniffi_ffibuffer_* trampolines the Dart bindings call only under that feature, and a library built without it loads cleanly and then fails at the first call with a dlsym error naming a symbol you never wrote.

Use

Cera is the portable API: one asynchronous surface that runs on every target, web included.

import 'dart:io';

import 'package:cera_ffi/cera_ffi.dart';

Future<void> main() async {
  final cera = await Cera.openPath('/path/to/model.gguf');

  final prompt = await cera.applyChatTemplate(
    [const CeraMessage.user('Why is the sky blue?')],
  );
  await for (final piece in cera.generate(prompt, maxTokens: 128)) {
    stdout.write(piece);   // fragments, not tokens: just append them
  }

  await cera.close();
  exit(0);   // see "Platform support": a NativeCallable keeps the isolate alive
}

Downloading a published model

Cera.listBundles() returns the models published on LiquidAI/LeapBundles as <name, quants> pairs, which is what a picker needs, and Cera.openBundle downloads one and opens it. Both work on every target, and the same catalog and parser back cera list-bundles, so a menu shows what the CLI shows.

final bundles = await Cera.listBundles();
for (final b in bundles) {
  print('${b.displayName}: ${b.quants.join(" ")}');
}

final cera = await Cera.openBundle(
  bundles.first.name,     // the full id, not displayName
  bundles.first.quants.first,
  onProgress: (p) => print('${p.url} ${p.fraction ?? "?"}'),
);

Prefer this over fetching a .gguf yourself and calling openBytes. A bundle's manifest names every file it needs and states its modality, so a vision or audio model arrives complete rather than being guessed at from the arguments. It is also the cheaper path in memory, and on the web dramatically so: the weights go from the cache straight into the engine, one copy of the model instead of two, which is also what keeps it clear of the roughly 2 GiB ceiling on a single contiguous JavaScript allocation that openBytes runs into.

Downloads are cached, so re-opening the same pair is offline and instant. storeDir chooses where, and is required on Android and iOS, where an app may write only inside its own container: pass a path from path_provider. On desktop it defaults to $HOME/.cache/cera, which the cera CLI also uses, so the two share downloads.

The generated bindings underneath cover the whole engine (LoRA, vision, audio, embeddings, grammars, tool calling, KV compression). Reach for them when you need something Cera does not expose, and accept that they are native-only: dart:ffi is what they are, and the web does not have it. Most of their surface is synchronous, though fromPathAsync, fromBytesAsync and generateStreamingAsync are what Cera is built on.

import 'package:cera_ffi/cera_ffi.dart';

void main() {
  final engine = CeraEngine.fromPath(
    '/path/to/model.gguf',
    const EngineConfig(
      contextSize: 2048,
      backend: BackendPreference.auto,
      bundleRepo: null,
    ),
  );

  final session = engine.newSession(const SessionConfig());
  session.appendTokens(engine.encodeText('Why is the sky blue?'));
  final out = session.generate(const GenerateOpts(maxTokens: 128));

  print(engine.decodeTokens(out.tokens));
}

generate returns token IDs plus a summary; decodeTokens turns them back into text. For streaming and the async variants that keep the isolate responsive, see example/.

For multi-turn conversational chat with delta-only prefill and automatic template handling, use ChatSession via session.intoChat():

// Discard the preceding raw completion before starting a new conversation.
session.reset();
final chat = session.intoChat();
chat.ingest(chatMessageUser('Why is the sky blue?'));
final reply = chat.complete(const GenerateOpts(maxTokens: 128));
print(reply.text);

See example/chat.dart for the complete multi-turn conversational chat workflow.

Native Metal/wgpu models permit one live session per loaded model. Close a raw session before creating another on that model; reset and cancellation retain ownership. CPU sessions can share a model. The portable Cera adapter handles replacement when applying a seed to an empty conversation and keeps the current session for continuation.

Cera.transcribe preserves that conversation. When its GPU context is occupied, the core lazily loads and caches a separate model/context from retained weights, without reopening the original files or copying conversation KV. This costs extra model memory and first-use setup. See the GPU ownership guide and native consumer for runnable fixtures and seeded generation, transcription, recovery and continuation checks.

Voice Activity Detection (Silero VAD v5)

cera_ffi provides native bindings for the Silero VAD v5 speech activity detection engine:

import 'package:cera_ffi/cera_ffi.dart';

void main() {
  final vad = FfiSileroVad.fromFile('/path/to/silero_vad.gguf');
  final iterator = FfiVadIterator.create(
    FfiVadSampleRate.rate16kHz,
    sileroVadDefaultConfig(),
  );

  // Process 512-sample (32ms) audio chunks (PCM samples in [-1.0, 1.0])
  final chunk = List<double>.filled(512, 0.0);
  final event = iterator.processChunk(vad, chunk);
  if (event is FfiVadEventSpeechStart) {
    print('Speech started at ${event.sample} samples (${event.ms} ms)');
  } else if (event is FfiVadEventSpeechEnd) {
    print('Speech segment: ${event.startMs} to ${event.endMs} ms');
  }

  // Or extract speech timestamps for an entire audio buffer
  final timestamps = vad.getSpeechTimestamps(
    chunk,
    FfiVadSampleRate.rate16kHz,
    sileroVadDefaultConfig(),
  );
  for (final ts in timestamps) {
    print('Speech segment: ${ts.startMs} to ${ts.endMs} ms');
  }
  iterator.close();
  vad.close();
}

Platform support

Cera runs everywhere: Android, iOS, macOS, Linux, Windows, and the web.

The generated bindings run everywhere dart:ffi does, which is everywhere except the web.

That split is not a gap someone forgot to close. A browser runs the engine in a Web Worker; postMessage is asynchronous and a worker offers no synchronous escape hatch, so a synchronous engine.generate(...) is not implementable there at any price. Cera is the same engine behind an API whose shape a browser can satisfy: a Rust async runtime on native, a Web Worker on web.

On the web

Inference runs on WebGPU when the browser has it and falls back to a wasm CPU build when it does not: ~58 tok/s against ~1.4 tok/s, measured on the same machine and model. Install the runtime into your app's web/ directory once with dart run cera_ffi:install_web, then use Cera.openBundle (preferred, see above) or Cera.openBytes for a model the user supplies; there is no filesystem for openPath. Downloads are cached in the origin's private filesystem (OPFS), so a second visit reopens the same model without the network. No COOP/COEP headers are needed; nothing here uses threads. See the cera_ffi_flutter README for the full setup and for what is narrower there.

The generated bindings are still a stub on the web, and have to be: an unconditional dart:ffi import fails the entire build rather than one branch, so they are exported conditionally with a generated no-FFI branch. It is produced by the same just dart-bindings run, from the same interface, and CI compiles a web app against it, so it cannot drift. Data types are real there (EngineConfig, GenerateOpts, the error hierarchy and the enums construct and compare normally) while engine entry points throw UnsupportedError naming themselves.

Exiting a Dart script

Cera.generate registers a callback interface, and the vtable behind it holds static NativeCallables for the process's lifetime. A live NativeCallable keeps its isolate alive, so a plain Dart CLI has to call exit() even after close(). Flutter apps are running anyway and never notice.

Bindings

lib/src/generated/cera_ffi.dart is generated from the compiled cera-ffi cdylib by a vendored uniffi-bindgen-dart, then run through a deterministic patch tool. The same run emits lib/src/generated/cera_ffi_web.dart, the no-FFI stub lib/cera_ffi.dart falls back to on the web. Both are committed, so the package works out of the box. Regenerate from the repo root after any change to the Rust FFI surface:

just dart-bindings         # regenerate + patch
just dart-bindings-check   # verify nothing drifted, then analyze

Drift is not cosmetic: UniFFI checksums every method at construction, so bindings that lag the Rust side make the engine throw before the first call. The stub can drift too, in its own way, so the generator has tests asserting the two files expose the same members and CI compiles a web app against the stub.

License

Apache-2.0 OR MIT, matching the rest of the workspace.