dart_sentencepiece_tokenizer 1.1.0
dart_sentencepiece_tokenizer: ^1.1.0 copied to clipboard
A lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
We analyzed this package 7 hours ago, and awarded it 160 pub points (of a possible 160):
10/10 points: Provide a valid pubspec.yaml
5/5 points: Provide a valid README.md
5/5 points: Provide a valid CHANGELOG.md
10/10 points: Use an OSI-approved license
Detected license: MIT.
10/10 points: 20% or more of the public API has dartdoc comments
35 out of 107 API elements (32.7 %) have documentation comments.
Some symbols that are missing documentation: dart_sentencepiece_tokenizer.Encoding, dart_sentencepiece_tokenizer.Encoding.Encoding.empty, dart_sentencepiece_tokenizer.Encoding.Encoding.new, dart_sentencepiece_tokenizer.Encoding.attentionMask, dart_sentencepiece_tokenizer.Encoding.charToToken.
10/10 points: Package has an example
20/20 points: Supports 5 of 6 possible platforms (iOS, Android, Web, Windows, macOS, Linux)
-
✓ Android
-
✓ iOS
-
✓ Windows
-
✓ Linux
-
✓ macOS
These platforms are not supported:
Package not compatible with platform Web
Because:
package:dart_sentencepiece_tokenizer/dart_sentencepiece_tokenizer.dartthat imports:package:dart_sentencepiece_tokenizer/src/sentencepiece/sentencepiece_tokenizer.dartthat imports:package:dart_sentencepiece_tokenizer/src/sentencepiece/model/sentencepiece_model.dartthat imports:dart:io
50/50 points: code has no errors, warnings, lints, or formatting issues
10/10 points: All of the package dependencies are supported in the latest version
No dependencies.
To reproduce run dart pub outdated --no-dev-dependencies --up-to-date --no-dependency-overrides.
10/10 points: Package supports latest stable Dart and Flutter SDKs
20/20 points: Compatible with dependency constraint lower bounds
pub downgrade does not expose any static analysis error.
Analyzed with Pana 0.23.5, Dart 3.11.0-200.1.beta.
Check the analysis log for details.