extract static method

List<String> extract(
  1. String text
)

Extract meaningful keywords from text.

Lowercases, splits on non-alphanumeric chars, removes stopwords and tokens shorter than 3 characters. Returns unique tokens.

Implementation

static List<String> extract(String text) {
  final tokens = text.toLowerCase().split(_nonAlphaNum);
  final seen = <String>{};
  final result = <String>[];
  for (final token in tokens) {
    if (token.length < 3) continue;
    if (_stopwords.contains(token)) continue;
    if (seen.add(token)) result.add(token);
  }
  return result;
}