topK property

int? topK
final

Optional. The maximum number of tokens to consider when sampling.

The model uses combined Top-k and nucleus sampling.

Top-k sampling considers the set of top_k most probable tokens. Defaults to 40.

Note: The default value varies by model, see the Model.top_k attribute of the Model returned the getModel function.

Implementation

final int? topK;