dart_vector_embedding 0.1.1 copy "dart_vector_embedding: ^0.1.1" to clipboard
dart_vector_embedding: ^0.1.1 copied to clipboard

Published 2 months ago
SDKDartFlutter
PlatformLinuxmacOSWindows
→

Metadata

Cross-platform Dart Native GGUF embeddings via llama.cpp with shared per-model concurrency and native async callbacks.

More...

0.1.1 #

  • Include the vendored llama.cpp src/models sources required by the native CMake build; local GGUF test models remain excluded from the package.

0.1.0 #

  • Cross-platform Dart Native embedding package backed by vendored llama.cpp sources.
  • Shared native llama_model + llama_context per model path within one process.
  • Async-only Dart API for open, embed, embedBatch, and close.
  • Native worker-thread request queue with callback-based async completion for safe multi-isolate access.
  • GPU offload enabled when the host/backend supports it.
  • Bounded per-model request queues with VeQueueFullException backpressure.
  • Safe last-handle shutdown while native work is still queued.
  • Auto-detected CUDA and Vulkan backends with CPU fallback.
  • Portable CPU builds with bounded text and batch request sizes.
  • Reentrant native callback shutdown protection and full-context micro-batching.
  • Embedding latency benchmark and documented test-model setup.

← Metadata

0
likes
160
points
29
downloads

Documentation

API reference

Publisher

unverified uploader

Weekly Downloads

Metadata

Cross-platform Dart Native GGUF embeddings via llama.cpp with shared per-model concurrency and native async callbacks.

Repository (GitHub)
View/report issues

Topics

#ffi #embeddings #gguf #llama-cpp

License

MIT (license)

Dependencies

code_assets, ffi, hooks, native_toolchain_c

More

Packages that depend on dart_vector_embedding

Back