Wireshark · Ethereal-dev: Re: [ethereal-dev] Suggestions

[Ashok Narayanan <ashokn@xxxxxxxxx>]

 > I am currently working on changing the way that ethereal decodes
 > protocols for the GUI tree. Instead of having the dissectors place
 > strings onto a tree via proto_tree_add_item, I'm using a tree structure
 > (that is, GNode* from glib-1.2.x) to maintain a list of field values.
 > The dissectors will use proto_tree_add_item(), along with a couple new
 > functions, to store information about what they found while dissecting.
 > 
 > Then a new function, proto_tree_draw(), will take the information in the
 > logical tree and create the GUI tree. The print functions will do the
 > same; they'll read the logical tree and print text or postscript.
 > 
 > The reason I'm doing is this is to put display filters into ethereal.
 > Instead of creating a BPF filter for some or most fields, thereby
 > bloating wiretap or libpcap, we can make a filter for every single field
 > by putting it in ethereal. This makes the display filters much much more
 > powerful. You'll be able to filter on every field that is in the protocol
 > tree.
 > 
 > In order for filtering to be possible, I have to strongly type the
 > information that is being put into the logical tree. As such, the
 > dissectors register both their protocols and their fields to my new
 > protocol-handling code when ethereal starts up.

There's another way to do this, which may or may not be
better. Basically, each protocol implements it's own filtering
mechanism. It has it's own dialog box for filtering where you can
enter filtering info. For example, you can activate an OSPF filter by
clicking on some menu item which opens an OSPF filter dialog box,
where you can then add things like "show me all messages relating to
area 10". This information is stored by the OSPF filtering module in
some internal format. 

Now, when a packet is read into Ethereal, it basically looks to see if 
any filters are active. If a filter is active, it simply calls the
filtering function associated with that filter, passing it the
packet. The function looks at the packet, does it's thing, and returns 
TRUE or FALSE depending on whether the packet passes the filters or
not. 

The advantage of this technique is that it is somewhat distributed and 
robust. You don't need to do this two-phase processing with the
trees. Filters can be added or extended, and be made arbitrarily
detailed. Further, they don't depend on tree fields. For example, one
could think of a filter like "show me all XXXX packets which contain
this IP address somewhere in them". If this sort of filter was useful
to protocol XXXX, the filter manager for XXXX would have it (but
others wouldn't).

The disadvantage is, of course, that this stuff will have to be
written for each protocol. 

 > The result of this is that I now have a list of protocols. :)
 > 
 > I can easily put a new field in my protocol structure, one to store
 > color information.
 > 
 > > 2) Protocol filtering
 > > 
 > > - The wiretap filtering grammar looks to be an excellent improvement
 > > on BPF grammar, since it's extendable. I'll help out there if help is
 > > needed.
 > 
 > Here is what I'm inclined to do in the short term. Use libpcap for
 > capturing, including capture filters, use wiretap for reading many file
 > formats, and use ethereal for display filtering. I really want to get
 > display filters working well. Yes, there are problems with libpcap and I
 > want to address those in wiretap sometime, but at least for my needs,
 > I'd like to get display filters working well on all the file formats
 > supported by wiretap. And I think the best way is to move display
 > filters into ethereal.

Well, the benefit of a grammar is that filtering rules can be made
arbitrarily complex. But I think you are dead on when you say that we
need display filters in ethereal. I feel that capture filtering can be 
quite basic (even just protocol number is fine), and the real smarts
can be in display filtering. The BPF one-size-fits-all approach is bad.

Don't throw away the grammar idea just yet :-)

 > Wiretap still needs filters, but it has become a hard problem for me
 > because I'm not experienced in compiler writing, and what wiretap needs
 > is a good BPF compiler. I've been reading up on compiler writing, but
 > I'm not there yet.

I've some experience in compilers (though it was some time back) 
and can put in some effort here. However, I'd say that we should just
implement very basic capture filter predicates. That's enough for
this.

Are you planning to include both libpcap and wiretap in the same
image. This may be a good idea. You can use libpcap only for
capturing, and use wiretap for everything else. You then don't need to 
add any filtering to wiretap at all. Just pull it all out.

 > > - We need GUI extensions for filtering (this is not news to you, I'm
 > > sure).
 > 
 > yes!
 > 
 >  
 > > - A useful filtering scheme would be to filter on all packets in which
 > > a particular set of bytes match a particular pattern. Even more useful
 > > would be if you could select these. For example, you click on a
 > > packet, expand it, click on a protocol, expand it, click on an object
 > > within that protocol and select it (so it gets highlighted in the hex
 > > dump). Now, you go to the menu and say "Find all packets in which
 > > the selected bytes have the same value". Voila, it finds all those
 > > packets. This is a really useful technique and folks in my group use it
 > > all the time in Netmon (though they have to do a little more work
 > > there to get this functionality). I'm halfway through writing it; but
 > > if somebody has an equivalent or better solution, I'm all ears.
 > 
 > You mean using an offset from the beginning of a protocol layer?
 > Like,
 > 	tcp[5:3] = 01:02:03
 > 
 > Or even with a field?
 > 
 > 	tr.dst[0:3] = 00:00:f6  (madge token-ring cards!)

Exactly. The only thing would be that the user doesn't have to enter
this info in, but can extract it from an existing packet. I'll be
sending in a diff later today for this. Our devtest people say this
feature is a winner for them. 
 
 > > - There's also a set of filtering constraints which cannot be
 > > expressed in byte-offset primitives, but which are protocol
 > > specific. For example, in RSVP (my protocol of choice :-), a useful
 > > filter would be to find all WF-style reservation messages. Now, what
 > > defines a "WF-style" message cannot be expressed as an absolute byte
 > > offset. The message has a number of objects, one of which is a style
 > > object. So you need to parse the message to make this determination.
 > > 
 > > Where I'm getting with this is, we can make arbitrarily complex
 > > filters by implementing an extension to the filtering mechanism. We
 > > create new predicates (e.g. rsvp.resv.style) in the filter grammar,
 > > and associate a function with this predicate. If this predicate is
 > > found in the filter string, we call the function associated with the
 > > subject of this predicate, for every applicable packet in the packet
 > > stream. The function attempts to evaluate the predicate against the
 > > packet, and returns TRUE or FALSE. If the function returns FALSE, the
 > > packet did not match the predicate and should be thrown away. If the
 > > function returns TRUE, the packet matches the predicate and expression
 > > evaluation should continue on this packet. There are a couple of
 > > tricks you can use to optimize this so it's not so expensive. For
 > > starters, you could call all relevant protocols at the start of every
 > > packet so they can pre-parse and cache relevant information. Then the
 > > predicate evaluation function would be quite fast.
 > > 
 > > Thoughts?
 > > -Ashok
 > 
 > In my scheme, the rsvp protocol dissector would attach fields to the
 > logical tree. You can duplicate as many field types as you want.
 > Basically, anything you can decode via C can be added to the tree. And
 > then we can filter on it.

But, of course, you can't add filtering support for anything which is
not explicitly in the tree.

 > I have just finished getting the ethernet decoder to work with my new
 > scheme. I'll post a patch to ethereal-dev so people can see my work in
 > progress.

I'd be really interested in seeing this.

-Ashok

-- 
--- Ashok Narayanan ----------------------------------------
IOS Network Protocols, Cisco Systems
250 Apollo Drive, Chelmsford, MA 01824
Ph: 978-244-8387