The TopK property returns or changes the top-k sampling
value used when generating responses from the current model. Top-k
sampling limits token selection to the most likely candidate tokens
during text generation.
When a top-k value is specified, the model considers only the
most probable k tokens at each step of generation before
selecting the next token. Lower values generally produce more
deterministic and focused output, while higher values allow more
variation and diversity in generated responses.
A value of zero typically indicates that top-k sampling is
disabled or that the provider should use its default behavior. The
exact interpretation of this value may vary between providers and
models.
Support for top-k sampling varies significantly between providers
and models. Many providers either ignore this setting entirely or use
provider-specific defaults that cannot be overridden. Some providers
may also restrict the valid range of values or silently clamp values
to supported limits.
In most cases, applications should adjust only the
Temperature property and leave advanced sampling parameters
such as TopK and TopP at their default values unless
the application has specific requirements and understands how those
settings affect model behavior.
Changing the value of this property only affects subsequent
requests and does not modify existing messages or responses stored in
the conversation history.