hf_tokenizers 1.2.2 
hf_tokenizers: ^1.2.2 copied to clipboard
Published 8 hours ago • 
developeryusuf.com
developeryusuf.comHuggingFace tokenizers for Dart over FFI. Load any tokenizer.json and get byte-exact BPE, WordPiece, and Unigram encoding, backed by the Rust crate.
23 versions
Stable versions of hf_tokenizers
| Version | Min Dart SDK | Uploaded | Documentation | Archive | |
|---|---|---|---|---|---|
| 1.2.2 | 3.10 | 8 hours ago |  |  | |
| 1.2.1 | 3.10 | 24 days ago | | | |
| 1.2.0 | 3.10 | 24 days ago | | ||
| 1.1.0 | 3.10 | 44 days ago | | ||
| 1.0.6 | 3.10 | 44 days ago | | ||
| 1.0.5 | 3.10 | 44 days ago | | ||
| 1.0.4 | 3.10 | 44 days ago | | ||
| 1.0.3 | 3.10 | 44 days ago | | ||
| 1.0.2 | 3.10 | 45 days ago | | ||
| 1.0.1 | 3.10 | 46 days ago | | ||
| 1.0.0 | 3.10 | 60 days ago | | ||
| 0.5.1 | 3.10 | 2 months ago | | | |
| 0.5.0 | 3.10 | 2 months ago | | ||
| 0.4.5 | 3.10 | 2 months ago | | ||
| 0.4.4 | 3.10 | 2 months ago | | ||
| 0.4.3 | 3.10 | 2 months ago | | ||
| 0.4.2 | 3.10 | 2 months ago | | ||
| 0.4.1 | 3.10 | 2 months ago | | ||
| 0.4.0 | 3.10 | 2 months ago | | ||
| 0.3.0 | 3.10 | 2 months ago | | ||
| 0.2.0 | 3.10 | 2 months ago | | ||
| 0.1.1 | 3.10 | 2 months ago | | ||
| 0.1.0 | 3.10 | 2 months ago | |
Metadata
1
likes
160
points
207
downloads
Documentation
Publisher
Weekly Downloads
Metadata
HuggingFace tokenizers for Dart over FFI. Load any tokenizer.json and get byte-exact BPE, WordPiece, and Unigram encoding, backed by the Rust crate.
Repository (GitHub)
View/report issues
Topics
License
MIT (license)