topK property

int? topK
final

Optional. The maximum number of tokens to consider when sampling.

The model uses combined Top-k and nucleus sampling.

Top-k sampling considers the set of top_k most probable tokens.

Implementation

final int? topK;