The NormalizeText method processes the input string, Although removing unnecessary whitespace may slightly reduce
token usage, the primary benefit is producing cleaner and more consistent
input for the model to process.
Line break characters are normalized to use Windows end-of-line conventions.
Isolated carriage return or line feed characters, Unicode line and paragraph
separators, and form feed characters are converted to CRLF sequences.
Multiple consecutive line breaks are reduced to improve consistency.
Normalization can be useful when comparing prompts, checking for empty or
whitespace-only input, or applying simple validation rules before a request
is submitted. It does not attempt to determine whether the text is safe,
appropriate, or semantically valid. Applications should perform any additional
validation required for their own use case.
Using this method is optional. Prompt text does not need to be normalized
before it is sent to a provider, and using this method is not a required
sanitization step. If preserving the exact formatting of the original text
is important, such as indentation, tables, or code samples, the application
should send the original text instead.