The LlmSetHistorySize function controls the number of messages
retained in the conversation history for a client session. This history
is used to provide context for subsequent requests to the model.
When the number of messages exceeds the specified limit, the oldest
messages are removed automatically. This ensures that the total number
of stored messages does not exceed the configured maximum.
Reducing the history size can help limit memory usage and reduce the
amount of context sent with each request, which may improve performance
and reduce token usage. However, limiting the history size may also
reduce the model's ability to maintain context across multiple
interactions.
This setting works in conjunction with token limits imposed by the
selected model. Even if the maximum number of messages has not been
reached, older messages may be excluded from a request if including
them would exceed the model's context window.
If nMaxMessages has a value of zero, conversation
history is effectively disabled. In this mode, only the current
message and response are retained for the duration of the request,
and no prior messages are included as context for subsequent
requests. Disabling conversation history may reduce the model's ability to
maintain context across multiple interactions and can impact the
quality or relevance of responses. Functions such as
LlmGetFirstMessage and LlmGetNextMessage will fail with
the last error code set to ST_ERROR_NO_MESSAGE_HISTORY.
Disabling conversation history does not provide additional security or privacy.
It only affects whether previous messages are retained and automatically
included in subsequent requests. The content of each request is still
transmitted to the service provider and may be processed or stored in
accordance with that provider's policies. Applications that require
secure handling of sensitive information should avoid including
confidential data in requests and review the data handling practices
of the selected service provider.
Even when message history is disabled, the model may still produce
responses that appear contextually relevant within a single interaction.
This is because the current request can include sufficient context for
the model to generate an appropriate response. However, without retaining
previous messages, the model does not have access to prior exchanges in
the conversation, and continuity across multiple requests may be limited.
In most cases, retaining a moderate history size provides better
results for conversational applications.
The default history size is 100 message exchanges.