yake

Pure-Dart port of YAKE! (Yet Another Keyword Extractor) — the unsupervised, single-document keyword extraction algorithm from Campos et al., "YAKE! Keyword extraction from single documents using multiple local features" (Information Sciences 509, 2020). Reference implementation: LIAAD/yake.

Why this algorithm for Dart/Flutter apps:

• Single document — no corpus, no IDF tables, works on one text.
• Unsupervised — no model download, no network, no training data.
• Language-independent — the only language-specific input is a stopword list you supply.
• Fast and tiny — a few hundred lines, dart:math is the only import. Runs fine on-device.

Usage

import 'package:yake/yake.dart';

final yake = Yake(stopwords: englishStopwords);
for (final k in yake.extract(
    'The payment gateway migration is blocked on the Stripe webhook '
    'signature. Marco said the payment gateway migration slips to next '
    'sprint unless the webhook fix lands.',
    top: 5)) {
  print('${k.score.toStringAsFixed(4)}  ${k.keyword} (x${k.count})');
}
// 0.0xxx  Stripe webhook signature (x1)
// 0.0xxx  payment gateway migration (x2)
// ...

Scores are lower-is-better, per the paper. count is how many times that exact candidate occurred — a useful topicality signal on short informal text.

Accented languages

Pass a folding function and a stoplist folded the same way, so "Được" / "duoc" / "ĐƯỢC" are one term. The bundled foldDiacritics covers Latin-1 accents and the full Vietnamese alphabet:

final yake = Yake(
  stopwords: myFoldedVietnameseStopwords,
  fold: foldDiacritics,
);

Tuning

Yake(
  stopwords: englishStopwords,
  maxNgram: 3,        // longest phrase, in words (default 3)
  windowSize: 1,      // co-occurrence window (default 1, per the paper)
  dedupThreshold: 0.9 // Levenshtein similarity above which candidates merge
)

Faithfulness and deviations

All five term features (casing, position, frequency, relatedness, sentence spread), the candidate scoring formula and Levenshtein dedup follow the paper. Two deliberate deviations:

1. Identifier compounds tokenize whole — "GC-1042", "v2.1" stay single terms; real-world text is full of identifiers punctuation splitting would shed.
2. Stopword matching runs through the caller's fold (case folding by default), enabling accent-insensitive stoplists.

Known bias worth understanding before use on very short texts: YAKE's position feature favours terms of the first sentence, so on one-line messages a bland opening bigram can outrank a distinctive name later in the line. On multi-sentence text it behaves as published.

License

MIT © Golden Owl Asia