Wireshark-dev: Re: [Wireshark-dev] Display of UTF-8 Characters
From: Guy Harris <gharris@xxxxxxxxx>
Date: Sat, 12 Dec 2020 11:32:28 -0800
On Dec 12, 2020, at 10:43 AM, jayrturner99@xxxxxxxxx wrote:

> I create a GString str = “A{Dagger}B{Sigma}C”; (i.e. “\x41\xE2\x80\xA0\x42\xCE\xA3\x43” where \xE2\x80\xA0 is Dagger and \xCE\xA3 is Sigma).
> The Dagger is the correct UTF-8 code (https://www.fileformat.info/info/unicode/char/2020/index.htm)
> and the Sigma is the correct UTF-8 code (https://www.fileformat.info/info/unicode/char/03a3/index.htm).
>  
> I use col_append_lstr(pinfo->cinfo, COL_INFO, str, COL_ADD_LSTR_TERMINATOR);
> The display is “A{Dagger}B{Sigma}C” where the {Dagger} and {Sigma} are the correct visual single characters.
>  
> I use proto_string_add_string(…, str);

Presumably you mean proto_tree_add_string() - there is no proto_string_add_string() routine in Wireshark.

> The display is “A{Dagger}B{black-diamond-with-question-mark}{black-diamond-with-question-mark}C” where the {black-diamond-with-question-mark} is the visual single character of a black diamond with a question mark (and it is displayed twice).

So, as a test, I added to a dissector (the DHCP dissector, because a small capture file I often use as a test file has DHCP traffic in it), the statement

	proto_tree_add_string(tree, hf_dhcp_random_crap, tvb, 0, 8, "\x41\xE2\x80\xA0\x42\xCE\xA3\x43");

added a definition of "hf_dhcp_random_crap" before that:

	static int hf_dhcp_random_crap = -1;

and added a definition of that field to the hf[] array later:

	{ &hf_dhcp_random_crap,
	 { "Random crap", "dhcp.random_crap",
	   FT_STRING, BASE_NONE, NULL, 0x0,
	   NULL, HFILL }},

and compiled it on my Mac and tested it; it seemed to work, displaying the dagger and sigma correctly, when I did that to a recently-updated 3.2 branch, a recently-updated 3.4 branch, and a recently-updated trunk tree.

What was the definition of the field you used in the proto_tree_add_string() call?
 
> Environment:
> Windows 10 Enterprise (10.0.18363) x64
> Microsoft Visual Studio Community 2019 Version 16.7.1
> QT v5.15.0 using msvc2019_64
> Wireshark 3.3.0 with customer dissector

3.3.0 isn't an official release, so we don't provide support for it.  Do you mean 3.4.0?