Wireshark · Wireshark-dev: Re: [Wireshark-dev] Wrongly escaped UTF-8 characters in JSON values ( epan/print.c )

[Dario Lombardo <lomato@xxxxxxxxx>]

Ok, got it. Again, the best chances not to get this lost is to file a bug on bugzilla.

On Fri, Jul 6, 2018 at 1:47 PM Andrea Lo Pumo <alopumo@xxxxxxxxx> wrote:

From: Dario Lombardo

What do you mean by "I do not know the Wireshark code"? What did you patch? Do you mean you don't know the submission procedure instead?

I mean I do not know the full implications of changing the code as I did. It worked for me because I am just interested in gsm_sms.sms_text, however, before accepting this patch someone with better understanding of the Wireshark code should think if it is ok.

What did you patch?

 

print_escaped_bare() of epan/print.c

2018-07-05 16:01 GMT+02:00 Andrea Lo Pumo <alopumo@xxxxxxxxx>:

I am using "tshark -T json -V -r file.pcap" and specifically I am looking for the gsm_sms.sms_text field.

I get this output:

"gsm_sms.sms_text": "Ok per\u00c3\u00b2 non piove"

Instead, using "tshark -V -r file.pcap" I get:

SMS text: Ok però non piove

(There is an accent in the "o" of "però")

The problem is that the \uXXYY syntax is UTF-16 (see [1]), while "ò" is UTF-8 and its bytes are c3 b2. Wireshark writes c3 b2 as they were UTF-16.

I solved the problem by changing print_escaped_bare() of epan/print.c as follow:

substitute

        default:
            if (g_ascii_isprint(*p))
                fputc(*p, fh);
            else {
                g_snprintf(temp_str, sizeof(temp_str), "\\u00%02x", (guint8)*p);
                fputs(temp_str, fh);
            }

with

        default:

            fputc(*p, fh);

I do not know the Wireshark code, so I am not submitting a patch. This, however, should work because JSON supports UTF-8 (see again [1]).

[1] From the JSON page on Wikipedia: JSON exchange in an open ecosystem must be encoded in UTF-8. However, if escaped, those characters must be written using UTF-16 surrogate pairs, a detail missed by some JSON parsers.

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe

--

Naima is online.