The LlmGetModelContextSize function returns the maximum number of
tokens that can be processed by the specified model in a single request.
This value is commonly referred to as the model's context size or context
window.
The context size includes both the input provided to the model and the
output generated in response. This means that the total number of tokens in
a request, including the prompt, conversation history, and generated
response, must not exceed this limit.
If the combined number of tokens exceeds the model's context size, the
request may be rejected by the provider or the response may be truncated.
Applications should take this limit into account when constructing prompts
or maintaining long conversation histories.
In general, larger context sizes allow a model to retain more information
from previous interactions, but may also increase latency and cost depending
on the provider and model being used.
The context size returned by this function should be treated as an
approximate limit rather than an absolute value. For some providers,
particularly locally hosted models, the effective context size may be
configured or adjusted outside of the application. In these cases, the
reported value serves as a guideline and may not reflect the exact limit
enforced by the model at runtime.