Wireshark-dev: Re: [Wireshark-dev] hf_http_response_code in packet-http.c
Date Index · Thread Index · Other Months · All Mailing Lists
Date Prev · Date Next · Thread Prev · Thread Next
From: Pascal Quantin <pascal.quantin@xxxxxxxxx>
Date: Thu, 13 Jul 2017 22:16:01 +0200


Le 13 juil. 2017 22:00, "Sultan, Hassan via Wireshark-dev" <wireshark-dev@xxxxxxxxxxxxx> a écrit :


> -----Original Message-----
> From: Wireshark-dev [mailto:wireshark-dev-bounces@wireshark.org] On Behalf
> Of Erik de Jong
> Sent: Thursday, July 13, 2017 11:58 AM
> To: Developer support list for Wireshark <wireshark-dev@xxxxxxxxxxxxx>
> Subject: Re: [Wireshark-dev] hf_http_response_code in packet-http.c
>
>
>
> On Thu, Jul 13, 2017 at 8:47 PM, Sultan, Hassan via Wireshark-dev <wireshark-
> dev@xxxxxxxxxxxxx <mailto:wireshark-dev@wireshark.org> > wrote:
>
>
>
>
>       > -----Original Message-----
>       > From: Pascal Quantin [mailto:pascal.quantin@gmail.com
> <mailto:pascal.quantin@gmail.com> ]
>       > Sent: Thursday, July 13, 2017 11:26 AM
>       > To: Sultan, Hassan <sultah@xxxxxxxxxx
> <mailto:sultah@xxxxxxxxxx> >
>       > Cc: Developer support list for Wireshark <wireshark-
> dev@xxxxxxxxxxxxx <mailto:wireshark-dev@wireshark.org> >
>       > Subject: RE: [Wireshark-dev] hf_http_response_code in packet-http.c
>       >
>       >
>       >
>       > Le 13 juil. 2017 19:54, "Sultan, Hassan" <sultah@xxxxxxxxxx
> <mailto:sultah@xxxxxxxxxx>
>       > <mailto:sultah@xxxxxxxxxx <mailto:sultah@xxxxxxxxxx> > > a
> écrit :
>       >
>       >
>       >
>       >
>       >       > -----Original Message-----
>       >       > From: Pascal Quantin [mailto:pascal.quantin@gmail.com
> <mailto:pascal.quantin@gmail.com>
>       > <mailto:pascal.quantin@gmail.com
> <mailto:pascal.quantin@gmail.com> > ]
>       >       > Sent: Thursday, July 13, 2017 10:39 AM
>       >       > To: Developer support list for Wireshark <wireshark-
>       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>  <mailto:wireshark-
> dev@xxxxxxxxxxxxx <mailto:wireshark-dev@wireshark.org> > >
>       >
>       >       > Cc: Sultan, Hassan <sultah@xxxxxxxxxx
> <mailto:sultah@xxxxxxxxxx>
>       > <mailto:sultah@xxxxxxxxxx <mailto:sultah@xxxxxxxxxx> > >
>       >       > Subject: Re: [Wireshark-dev] hf_http_response_code in packet-
> http.c
>       >       >
>       >
>       >       > Hi Hassan,
>       >       >
>       >       >
>       >       > 2017-07-13 19:25 GMT+02:00 Sultan, Hassan via Wireshark-dev
>       > <wireshark-
>       >
>       >       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>
> <mailto:dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>  <mailto:wireshark-
> dev@xxxxxxxxxxxxx <mailto:wireshark-dev@wireshark.org> > > >:
>       >
>       >       >
>       >       >
>       >       >
>       >       >
>       >       >       > -----Original Message-----
>       >       >       > From: Wireshark-dev [mailto:wireshark-dev-
> <mailto:wireshark-dev->
>       > bounces@xxxxxxxxxxxxx <mailto:bounces@xxxxxxxxxxxxx>
> <mailto:wireshark-dev-bounces@wireshark.org <mailto:wireshark-dev-
> bounces@xxxxxxxxxxxxx> >
>       >       > <mailto:wireshark-dev-bounces@wireshark.org
> <mailto:wireshark-dev-bounces@wireshark.org>  <mailto:wireshark-
> <mailto:wireshark->
>       > dev-bounces@xxxxxxxxxxxxx <mailto:dev-bounces@wireshark.org> >
> > ] On Behalf
>       >       >       > Of Erik de Jong
>       >       >       > Sent: Wednesday, July 12, 2017 10:12 PM
>       >       >       > To: Developer support list for Wireshark <wireshark-
>       >
>       >       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>
> <mailto:dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>  <mailto:wireshark-
> dev@xxxxxxxxxxxxx <mailto:wireshark-dev@wireshark.org> > > >
>       >       >       > Subject: Re: [Wireshark-dev] hf_http_response_code in
> packet-
>       > http.c
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       > On Thu, Jul 13, 2017 at 1:12 AM, Sultan, Hassan via
> Wireshark-
>       > dev
>       >       > <wireshark-
>       >
>       >       >       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>
> <mailto:dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx> >
>       > <mailto:dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>
> <mailto:dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx> > >
> <mailto:wireshark- <mailto:wireshark->
>       > <mailto:wireshark- <mailto:wireshark-> >
>
>       >
>       >       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>
> <mailto:dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx> >  <mailto:wireshark-
> <mailto:wireshark->
>       > dev@xxxxxxxxxxxxx <mailto:dev@xxxxxxxxxxxxx>  <mailto:wireshark-
> dev@xxxxxxxxxxxxx <mailto:wireshark-dev@wireshark.org> > > > > wrote:
>       >       >       >
>       >       >       >
>       >       >       >       Hi,
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       I am starting to learn the Wireshark code base, and one
> thing
>       >       > puzzles
>       >       >       > me…
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       Why is hf_http_response_code defined as a FT_UINT16
> with
>       >       > BASE_DEC
>       >       >       > rather than an FT_STRING ?
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >       >       It’s a text field… not an integer.
>       >       >       >
>       >       >       >
>       >       >       > Presenting it as a number allows for filtering like:
>       >       >       > http.response.code > 200
>       >       >       >
>       >       >       > Which would not be possible when presented as a string.
>       >       >
>       >       >       Thanks for the info, but in that case would it not be more
>       > appropriate to
>       >       > have the normal field as an FT_STRING and add a generated field
> as
>       > FT_UINT16
>       >       > ? My understanding of generated fields is that this is their
> purpose :
>       > represent
>       >       > data that doesn't exactly correspond to the packet data.
>       >       >       We could still keep the field named as is today (hence
> ensuring all
>       >       > existing filters still work), but simply make it a generated field,
> and add
>       > an
>       >       > FT_STRING to represent the actual data as it is in the packet.
>       >       >
>       >       >       Thoughts ?
>       >       >
>       >       >
>       >       >
>       >       >
>       >       > Does having the field as it is designed today generating you any
> issue?
>       > For fields
>       >       > having numerical values only, it makes much more sense to have
> the
>       > digits
>       >       > directly instead of the string and that's already what we are doing
> in
>       > numerous
>       >       > places. And personally I do not see any valid reason to change
> that.
>       > Your
>       >       > suggestion implies that many fields would start to be duplicated
>       > without any
>       >       > added value.
>       >
>       >
>       >       My reason is that anyone wanting to build automation based on
>       > Wireshark's parsing will not be able to rely on Wireshark's reporting of
> field type
>       > & format to interpret the packet data.
>       >       The automation would walk through parsed fields and be told that
> the
>       > http response code is a 3-bytes UINT16 with value say 200, so bytes 00
> 00 c8,
>       > which is wrong. Right now the way these fields are setup works for
> human
>       > interpretation, by people looking at the rendered value on a screen,
> but it is far
>       > from ideal for automation.
>       >
>       >
>       > Thanks fact that HTTP is text based does not change the fact that the
> code is
>       > supposed to be a number. So I'm not sure to get your point.
>       > Moreover tons of people use tshark for automation and I do not
> remember
>       > someone complaining about the current output. So maybe I simply did
> not
>       > understand your use case.
>       >
>       >
>       >       One could argue it's the main purpose of Wireshark, but I think it
>       > needlessly prevent a whole class of use-cases from being achieved
> with
>       > Wireshark's technology.
>       >
>       >
>       >
>       > Could you elaborate? What use case do you have in mind exactly?
>
>
>       Imagine automation that attempts to perform modification on packets
> using the parsed information provided by Wireshark, or somehow goes and
> directly reads fields in the packet data based on what Wireshark's parsed
> information says (type/offset/length/...). Wireshark reporting the field as
> FT_UINT16 instead of say a UTF-8 FT_STRING throws off the automation as it
> would either attempt to write an FT_UINT16 at the offset of the field, writing
> the wrong format, or be confused by seeing a 3 bytes FT_UINT16 and give up.
>
>
>
>
> But in that case it's more like you're doing a part of the parsing in another
> application, why not implement a full parser in that application? The whole point
> of an analyzer is to make sense of the data passed into it for the user. And what
> about protocols that could be send with different encodings (BER, XER, etc)
> there a field type should still be the same - eg UINT16 - but the representation
> on the wire is totally different.

The beauty of this is specifically that the other app does not need to do any parsing at all. It fully relies on Wireshark's parsing engine to get the tree of fields, the offset/length/type/name/encoding/... of each field and instead of having to write custom code for each protocol (you know, a parser), the app simply has to implement generic handling for FT_STRING, FT_UINT16, etc... and the various encodings. So for your BER encoded integer, having an FT_UINT16 with an ENC_BER encoding flag for example and a size of 4 (type+length+2 bytes for most short ints) would convey all the info needed to handle the field.

That would require having an encoder able to understand Wireshark output syntax and the structure of the protocol you are using to perform the encoding (text based, TLV, CSN1, ASN1, etc): all the encoding rules related to the protocol you are playing with. Having a decoder / parser is far from giving you an encoder.

As Eric explained, having the HTTP response code as a number simply gives you much more advantages (filtering, comparisons, inequalities...) than having it in text. So this is clearly not a bug as it was done on purpose.


Will such a thing work for absolutely every single protocol out there ? Maybe not, but it works for the vast majority of them.

I have some doubts regarding this statement, for the reason I gave above.
But as I'm eager to learn, if you can come up with a PoC I would be very interested to look at it :)

I did a very similar thing on top of Network Monitor in a previous life at Microsoft (I had the code available being internal to the company) and it was a huge success, helping many people in various ways.

So what kind of encoding did it support?


>       How about having an #ifdef for such things so that Wireshark's use-case
> today does not have to suffer the addition of generated fields, and such
> automation cases could still get full fidelity parsing by building Wireshark's
> engine with the specific #define ? Today they have to build directly against
> Wireshark's code anyway so that shouldn't be a problem.
>
>
>       __________________________________________________________
> _________________
>       Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx
> <mailto:wireshark-dev@wireshark.org> >
>       Archives:    https://www.wireshark.org/lists/wireshark-dev
> <https://www.wireshark.org/lists/wireshark-dev>
>       Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-
> dev <https://www.wireshark.org/mailman/options/wireshark-dev>
>                    mailto:wireshark-dev-request@wireshark.org <mailto:wireshark-
> dev-request@xxxxxxxxxxxxx> ?subject=unsubscribe
>

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@wireshark.org?subject=unsubscribe