Platformsfold toggle (up/down arrow)

SDKsfold toggle (up/down arrow)

Licensefold toggle (up/down arrow)

Advancedfold toggle (up/down arrow)

Results 3 packages
Sort by

tiktoken_tokenizer_gpt4o_o1copy "tiktoken_tokenizer_gpt4o_o1: ^1.2.1" to clipboard
tiktoken_tokenizer_gpt4o_o1: ^1.2.1 copied to clipboard

2
likes
140
points
220
downloads
OpenAI's Tiktoken tokenizer for models: GPT-4, GPT-4o, GPT-4o-mini, o1, o1-mini, and o1-preview.#tokenizer#openai#gpt#gpt-4o#tiktoken

v 1.2.1 (6 months ago)verified publisherglasberg.devBSD-2-Clause

tokenizer_moqasemcopy "tokenizer_moqasem: ^1.0.1" to clipboard
tokenizer_moqasem: ^1.0.1 copied to clipboard

recently created package Added 15 days ago
1
likes
145
points
137
downloads
Intelligent tokenization plugin with advanced Arabic support, fuzzy matching, phonetic analysis, and AI-powered text processing. Built with 95% AI assistance.#nlp#tokenizer#arabic#text-analysis#fuzzy-matching

v 1.0.1 (15 days ago)MIT

tiny_segmenter_dartcopy "tiny_segmenter_dart: ^1.0.1" to clipboard
tiny_segmenter_dart: ^1.0.1 copied to clipboard

3
likes
125
points
38
downloads
A compact Japanese text tokenizer for Dart. TinySegmenter is a Japanese word segmentation library based on the original JavaScript implementation by Taku Kudo.#japanese#text-processing#tokenizer#nlp#segmentation

v 1.0.1 (4 months ago)verified publisheriori.devunknown

Check our help page for details on search expressions and result ranking.