Unicode

Unicode is a multi-language character set designed to encompass virtually all of the characters used with computers today. Unicode characters are represented by a 16-bit value, and differ from other character sets in two important ways. First, unlike the traditional single-byte (ANSI) character sets, Unicode is capable of representing significantly more characters in a variety of languages. Second, unlike multi-byte character sets (where some characters may be one byte in length, while others may be two bytes), the characters are fixed-width, which makes them easier to work with.

Whenever a string is assigned to a property value or passed to a method, that string is in Unicode. If necessary, the control will automatically convert that string to ANSI and it does not require any additional programming on the part of the developer. This is all largely transparent when using the components in high-level languages like Visual Basic. However, in Visual C++ and other languages that deal with COM objects on a lower level, it is important to understand that string values must be passed as BSTRs, which are Unicode strings.

The issue that most commonly confronts developers with regards to how strings are handled by the SocketTools components are with regards to the Read and Write methods. These methods are used to send and receive data over the network, and accept several different types of data. Typically, the data is exchanged as either a string of text characters, or as an array of bytes. Consider the following code:

Dim strMessage As String
Dim strBuffer As String
Dim cbBuffer As Long

Do

  cbBuffer = SocketWrench1.Read(strBuffer, 1024)
  If cbBuffer > 0 Then strMessage = strMessage + strBuffer

Loop Until cbBuffer < 1

In this case, the program expects to receive data from the server which is textual, and it will be stored in the string strMessage. What happens internally is that the data received from the server is automatically converted from an array of bytes into a string by the control. This is done because the control knows that the strBuffer argument is typed as a String, which means it is Unicode. However, what if the data being returned by the server contains binary data or is already Unicode text? In this case, the data may end up being corrupted because of the conversion performed by the control. To prevent this, the solution is to read the data into an array of bytes rather than a string. For example:

Dim byteMessage() As Byte
Dim byteBuffer(1024) As Byte
Dim cbMessage As Long
Dim cbBuffer As Long

Do
  cbBuffer = SocketWrench1.Read(byteBuffer, 1024)

  If cbBuffer > 0 Then
    ReDim Preserve byteMessage(cbMessage + cbBuffer) As Byte

    For nIndex = 0 To cbBuffer - 1
      byteMessage(cbMessage + nIndex) = byteBuffer(nIndex)
    Next
    cbMessage = cbMessage + cbBuffer
  End If

Loop Until cbBuffer < 1

In this case, because the data is being read into a byte array, not a string, then no Unicode conversion is performed and the data is returned exactly as it was sent. Note that Visual Basic also supports the ability to explicitly convert between Unicode strings and byte arrays using the StrConv function. For more information, refer to the language reference and online help in Visual Basic.

If you are reading UTF-8 encoded text into a byte array, either from a file or over a network connection, the FileEncoder control provides two helper methods: StrToUtf8 and Utf8ToStr. These methods make it easier to convert between UTF-8 stored in a byte array and the native Unicode String type in languages such as Visual Basic. It is important to note that the StrConv function in Visual Basic does not correctly handle UTF-8 encoded text unless it contains only ASCII characters.

If your application needs to display Unicode text in Visual Basic — for example, an email message written in Greek, Japanese, or another non-Latin script — you will encounter limitations with the standard UI controls provided by the Visual Basic runtime. Although Visual Basic internally supports Unicode strings, the default controls such as Label and TextBox do not render Unicode characters correctly. Instead, they attempt to convert the text to ANSI using the system's active code page, which is based on the current locale settings. If the Unicode string contains characters that are not supported by the active code page, such as Greek letter ά, they are typically replaced with a question mark (?).

To display Unicode text correctly in Visual Basic 6.0 or other legacy development environments, regardless of the user's system locale, you should use Unicode-aware components such as the TextBox control from the Microsoft Forms 2.0 Object Library, or call Windows API functions that support wide-character (UTF-16) text output directly.

The Microsoft Forms 2.0 Object Library (FM20.DLL) includes controls such as a Unicode-aware TextBox that can correctly display international text, including characters from non-Latin scripts. These controls are commonly used in Visual Basic 6.0 applications as a workaround for the limitations of the standard UI controls. In practice, they work well and integrate smoothly with VB forms, supporting full Unicode input and display. However, there are important considerations to be aware of: this library is officially licensed for use only with Microsoft Office, and it is not redistributable as part of a standalone application. While the controls may be present on systems that have Office installed, deploying applications that rely on FM20.DLL to users without Office can result in missing or non-functional controls, and may raise legal or support concerns. If you choose to use Forms 2.0 controls, ensure that your deployment environment includes Office, or consider providing an alternate interface or fallback behavior if the controls are unavailable.

For advanced scenarios, developers familiar with the Windows API may choose to display Unicode text using functions such as DrawTextW or SetWindowTextW with custom controls or owner-drawn interfaces. However, this approach is significantly more complex and typically unnecessary for most applications.