LLM_TOKEN

LLM_TOKEN_USAGE

The LLM_TOKEN_USAGE structure contains information about token usage for a client session.

typedef struct _LLM_TOKEN_USAGE
{
    DWORD   dwSize;
    DWORD   dwProviderId;
    DWORD   dwReserved;
    LONG    nMessageCount;
    LONG    nInputTokens;
    LONG    nOutputTokens;
    LONG    nContextUsed;
    LONG    nContextSize;
    TCHAR   szModelName[LLM_MODEL_NAMELEN];
} LLM_TOKEN_USAGE, *LPLLM_TOKEN_USAGE;

Members

dwSize

The size of this structure, in bytes. This member must be set to sizeof(LLM_TOKEN_USAGE) before the structure is used.

dwProviderId

The provider identifier. This member should be one of the LLM_PROVIDER values. Refer to the providers page for a list of supported providers and their default models.

dwReserved

A reserved value for future use. This member will always be zero.

nMessageCount

The number of messages currently stored in the session history.

nInputTokens

The number of input tokens associated with system and user messages in the current session.

nOutputTokens

The number of output tokens generated in model responses for the current session.

nContextUsed

The total number of tokens currently used in the active chat context, including input messages and message history.

nContextSize

The maximum number of tokens supported by the current model for the active context. A value of zero indicates that this limit is not available.

szModelName

A null-terminated string which specifies the name of the model associated with the current session. This value may be empty if no model name is available.

Remarks

This structure is used to return token usage information for an active client session. It provides a summary of how many messages are currently in the session history, how many tokens have been used for input and output, and how much of the available context window is currently in use.

The nInputTokens member reflects tokens associated with user and system input, while nOutputTokens reflects tokens generated by the model in its responses. The nContextUsed member represents the total token usage currently contributing to the active chat context, including prior message history that may be sent with each request.

The nContextSize member indicates the maximum context size supported by the current model, when that information is available. Applications may compare nContextUsed with nContextSize to estimate how much of the available context window has been consumed.

This structure is intended for informational and diagnostic use. Token counts are provider-dependent and may represent estimates rather than exact values, depending on the model or service in use. Applications should not rely on these values to enforce strict limits, as the provider ultimately determines whether a request exceeds its supported context or token constraints.

Applications should set the dwSize member before passing this structure to a function. Unless otherwise documented, reserved members should be set to zero. String values are stored directly in the fixed-length buffers contained within the structure and may be read safely by the application.

Requirements

Minimum Desktop Platform: Windows 7 Service Pack 1
Minimum Server Platform: Windows Server 2008 R2 Service Pack 1
Header File: cstools12.h
Unicode: Implemented as Unicode and ANSI versions

Members

Remarks

Requirements

See Also