token_text_controller #

A token-aware TextEditingController for Flutter. It detects and styles semantic text while preserving a normal editable string, cursor, selection, clipboard behavior, and IME composition.

Use it with any TextField, TextFormField, or EditableText. No custom input widget is required.

Features #

• Every built-in detector enabled by default, with fine-grained activation.
• Reusable token groups for messaging, social, contact, commerce, structured data, suggestions, links, and Markdown.
• Context-aware styles derived from the current ColorScheme.
• A dedicated optional TextStyle for every built-in token kind.
• Regex, function, trigger, and fully custom token detectors.
• Priority-based overlap resolution.
• Unicode grapheme handling for joined, modified, keycap, and flag emoji.
• Current-token queries for mention, hashtag, command, and cashtag suggestions.
• Token list, newly detected token, and active query callbacks.
• Semantic token insertion with arbitrary application metadata.
• A reusable TokenText renderer for non-editable content.
• Correct composing-range rendering for international keyboards and IMEs.

Installation #

For local development:

dependencies:
  token_text_controller:
    path: ../token_text_controller

After publication to pub.flutter-io.cn:

dependencies:
  token_text_controller: ^0.2.0

Minimal usage #

final controller = TokenTextController();

TextField(
  controller: controller,
  decoration: const InputDecoration(hintText: 'Message'),
)

By default, every built-in token kind is active, including links, contact values, social tokens, structured values, lightweight Markdown, commands, and emoji. controller.text remains an ordinary string.

Styling #

Every built-in token accepts a complete Flutter TextStyle. Unspecified styles fall back to theme-aware defaults.

Emoji use a stable 22 font size by default in both mixed text and emoji-only values. Set emoji or emojiOnly when the product needs a different size.

final controller = TokenTextController(
  styles: TokenTextStyles(
    base: TextStyle(fontSize: 16),
    url: TextStyle(decorationThickness: 2),
    email: TextStyle(fontWeight: FontWeight.w500),
    mention: TextStyle(
      color: Colors.blue,
      fontWeight: FontWeight.w700,
    ),
    hashtag: TextStyle(color: Colors.green),
    command: TextStyle(color: Colors.deepPurple),
    emoji: TextStyle(fontSize: 22),
    emojiOnly: TextStyle(fontSize: 30),
  ),
);

TokenTextStyles also provides styles for phone numbers, references, dates, times, currencies, percentages, UUIDs, IP addresses, variables, Markdown links, inline code, bold, italic, strikethrough, spoilers, and custom kinds.

Use TokenTextStyles.unstyled() when no built-in visual treatment is desired. For per-value or metadata-driven styling, provide a resolver:

TokenTextController(
  styleResolver: (details) {
    if (details.token.data == 'verified') {
      return details.suggestedStyle.copyWith(
        decoration: TextDecoration.underline,
      );
    }
    return details.suggestedStyle;
  },
)

Suggestions and active queries #

onActiveTokenChanged and activeQueryListenable report the trigger token at the caret, including incomplete values such as @lu, #flu, or /he.

final controller = TokenTextController(
  onActiveTokenChanged: (query) {
    if (query?.kind == TokenKind.mention) {
      loadProfiles(query!.query);
    }
  },
);

A suggestion can replace the complete active range and retain domain data:

controller.replaceActiveQuery(
  '@Luis',
  kind: TokenKind.mention,
  data: {'profileId': 'profile-123'},
);

The inserted token appears in controller.semanticTokens and remains linked to its metadata until an edit intersects that token.

Serialization #

Persist plain text and resolved semantic metadata together:

final json = controller.document.toJson();
final document = TokenTextDocument.fromJson(json);
final restoredController = TokenTextController.fromDocument(document);

Use encodeData and decodeData when token metadata needs conversion to a JSON-compatible representation.

Token events #

TokenTextController(
  onTokensChanged: (tokens) {
    // Current non-overlapping token snapshot.
  },
  onTokenDetected: (token) {
    // A token has newly appeared in the editable value.
  },
  onActiveTokenChanged: (query) {
    // The token query at the current collapsed selection changed.
  },
)

The same state is available through tokensListenable and activeQueryListenable for widget composition without callbacks.

Active tokens and groups #

Use activeTokens to enable only the token kinds needed by a field. The default is TokenGroups.all.

final controller = TokenTextController(
  activeTokens: TokenGroups.chat,
);

Groups are intentionally scoped to a concrete kind of field:

Groups are regular constant lists, so they can be used directly or combined:

final controller = TokenTextController(
  activeTokens: TokenGroups.combine(
    [TokenGroups.social, TokenGroups.markdown],
    include: const [TokenKind.reference],
    exclude: const [TokenKind.cashtag],
  ),
);

Activation can change while the field is alive. Existing text, styles, token events, and the active suggestion query are recalculated immediately.

controller.activeTokens = TokenGroups.suggestions;
controller.activateTokens(const [TokenKind.emoji]);
controller.deactivateTokens(const [TokenKind.cashtag]);
controller.activateAllTokens();

availableTokens reports the kinds backed by the controller's detector catalog, while isTokenActive(kind) checks the current selection.

Detector presets #

TokenTextController(detectors: TokenDetectors.social());
TokenTextController(detectors: TokenDetectors.chat());
TokenTextController(detectors: TokenDetectors.messaging());
TokenTextController(detectors: TokenDetectors.assistantChat());
TokenTextController(detectors: TokenDetectors.commerce());
TokenTextController(detectors: TokenDetectors.structured());
TokenTextController(detectors: TokenDetectors.markdown());
TokenTextController(detectors: TokenDetectors.all());

Use detectors to replace or customize the detector catalog. All supplied detectors are active unless activeTokens is also provided. TokenDetectors also exposes forKind and forKinds for building a catalog from token kinds.

Custom tokens #

Token kinds are value objects rather than a closed enum:

const orderKind = TokenKind('order');

final controller = TokenTextController(
  detectors: [
    ...TokenDetectors.defaults(),
    RegexTokenDetector(
      kind: orderKind,
      id: 'order',
      priority: 150,
      pattern: RegExp(r'ORD-\d+'),
    ),
  ],
  styles: TokenTextStyles(
    custom: {
      orderKind: TextStyle(
        color: Colors.orange,
        fontWeight: FontWeight.bold,
      ),
    },
  ),
);

Implement TokenDetector directly or use FunctionTokenDetector when a regular expression is not sufficient. Higher priorities win when ranges overlap.

Rendering outside a field #

TokenText(
  message,
  activeTokens: TokenGroups.chat,
  styles: controller.styles,
  semanticTokens: controller.semanticTokens,
)

This package performs token-aware styling and semantic range management. It is not a document editor and does not alter the source string with hidden markup.