parseDuckDuckGoResults function

List<WebSearchSource> parseDuckDuckGoResults(
  1. String html, {
  2. int? limit,
})

Walks the DDG results page and pulls out result blocks in document order (omp's parseHtmlResults, rebuilt on the forgiving scanHtml tokenizer so minor markup rot — attribute order, quote style, extra classes, snippet element variants — does not break parsing).

Each result lives in a <div class="… result …"> container with an <a class="… result__a …"> title link and an optional result__snippet element. Sponsored rows, missing snippets, and the pagination row are tolerated; duplicate URLs are dropped.

Implementation

List<WebSearchSource> parseDuckDuckGoResults(String html, {int? limit}) {
  final tokens = scanHtml(html).toList(growable: false);
  final results = <WebSearchSource>[];
  final seen = <String>{};

  for (var i = 0; i < tokens.length; i++) {
    final token = tokens[i];
    if (token is! HtmlTag || !_isResultContainer(token)) continue;

    final end = _subtreeEnd(tokens, i);
    final source = _parseResultBlock(tokens, i + 1, end);
    i = end;
    if (source == null || !seen.add(source.url)) continue;
    results.add(source);
    if (_resultLimitReached(results, limit)) return results;
  }
  return results;
}