This structure is used with the LlmConnectEx
function to provide extended configuration information when creating a
session. It allows an application to customize connection behavior,
sampling parameters, model selection, authentication, and other optional
settings without requiring a large number of function parameters.
Applications that only need to specify the provider, base URL, model
name, and API key may use the simpler LlmConnect function
instead.
The dwProviderId member should normally match the
provider specified in the LlmConnectEx function call. A value of
LLM_PROVIDER_NONE may be used when the application relies
on the function parameter to select the provider. If conflicting
provider values are specified, the function will fail.
In most cases, applications only need to initialize
dwSize, dwProviderId, and the specific members they wish
to override. All unused numeric members should be set to zero, and all
unused string members should be set to NULL. The library will apply
provider-specific defaults when optional values are not specified.
The lpszApiKey member is a string value that specifies the API key
used to authenticate with the provider. The API key may be provided
using a literal string, an environment variable, or a value stored in
the Windows Credential Manager. Using an environment variable or
secure credential storage is recommended for production applications
to avoid exposing sensitive information in source code or application
binaries. To specify an environment variable, surround the name in
percent (%) symbols, such as "%OPENAI_API_KEY%".
To specify a Windows Credential, surround it in braces, such as
"{MyAppName/OpenAI}".
For production applications, it is recommended that you avoid using
literal API key strings and instead use an environment variable or a
value stored in the Windows Credential Manager. If you must provide
the API key as a literal string, you should take additional steps to
protect it, such as storing the key in an encrypted form and limiting
access to authorized users or processes. Avoid embedding API keys
directly in application source code, configuration files, installers
or other content that could be exposed through version control
systems, logs or public repositories.
The dwReasoningLevel member provides a provider-neutral hint
that controls how much internal reasoning a model should use when generating
a response. Higher levels may improve the quality of results for more
complex tasks, but can also increase response time and token usage.
Not all models support reasoning controls, and the exact behavior is
provider-specific. In most cases, it is recommended that you set this
value to LLM_REASONING_DEFAULT, allowing the provider or model to determine
the appropriate level automatically.
The sampling-related members fTemperature,
fTopP, and nTopK control how responses are generated. In
most applications, adjusting the temperature alone is sufficient, while
Top-P and Top-K are intended for more advanced tuning scenarios. If
these values are not specified, the provider or model defaults are used.
Not all providers or models support sampling parameters such as temperature,
top-p or top-k. The library will only include those values when they
are supported by the selected provider and model. For models that do
not support these parameters, the session values are ignored.
The connection-related members lpszBaseUrl, nTimeout,
and nIdleTime control how requests are sent and how long an
existing HTTP connection may be reused. When the LLM_OPTION_KEEPALIVE
option is enabled, connections may be reused across multiple requests,
subject to the idle timeout period.
The nCacheTime member controls how long the library
may cache provider-specific information, such as model lists. This helps
reduce unnecessary network requests when applications repeatedly query
provider capabilities.
The lpszAppName member specifies the application name
associated with requests sent by the control. This value is included
as part of the default user-agent string and may be used for
diagnostic and identification purposes by the provider. A recommended
format is to specify both the application name and version number
using a convention similar to standard user-agent strings, for
example: "MyApplication/1.0".
The lpszUserId member is a user identifier associated with
requests sent to the current provider. Some providers allow
applications to specify a user identifier as part of the request
payload for diagnostics, analytics, rate limiting, or request
tracking. Some providers may ignore the value or use it only for
internal monitoring and logging purposes. Applications should avoid
including sensitive or personally identifiable information in the user
identifier value unless required by the provider.
The lpszOrgId member specifies an optional organization,
tenant or account identifier used by some providers when processing
requests. This is primarily intended for enterprise or multi-tenant
environments where a provider account may be associated with multiple
organizations or projects. Most applications will not need to set this
value unless specifically required by the provider. If the current
provider does not support organization identifiers, the value is
ignored.
The lpszProjectId member specifies an optional project
identifier used by some providers when processing requests. This is
primarily intended for providers which support project-based
authentication, billing or resource management. Most applications will
not need to set this value unless specifically required by the
provider or account configuration. If the current provider does not
support project identifiers, the value is ignored.
The lpszSystemPrompt member may be used to define a
session-level system prompt that establishes the behavior of the
model. A system prompt provides instructions that are included with
each request sent to the model. It can be used to define the expected
role, tone, response style, formatting rules, or other
application-specific guidance that should apply to the session. For
example, an application might use a system prompt to request short
responses, avoid Markdown formatting, return structured text, or
answer only within the context of a specific feature or workflow.
The string members in this structure are supplied by the
application and are treated as input values when the session is created.
Applications should ensure that any referenced strings remain valid
until a connection is established. The library makes an internal copy of
these values, so the application does not need to maintain them after
the connection completes.