On Sun, Jan 30, 2000 at 11:15:14AM -0800, Dragos Ruiu wrote:
> I once implemented some state engines on a protocol processor
> that added states to the frame storage. When searching though large files
> that were big, memory accesses to state info produced a lot of paging
> and slowed down bulk packet processing as we fetched around for states.
Yes, big captures on Ethereal tend to consume lots of memory; after
reading one of those, de-iconifying a Netscape window tends to cause the
disk to make lots of noise....
> I might also suggest storing the state information in a file format with the
> capture to avoid having to do states/reassembly on large files needlessly.
Presumably meaning "store the state information in a separate file from
the capture", unless you mean introducing a *new* capture file format -
existing capture file formats have no provision for additional
information of that sort.
> (I tend to have these massive capture files that stress out memory and take
> forever to load.)
One other reason why big capture files take a long time to load is that
we have to dissect every packet to the extent that we can generate its
packet-list line, as the CList widget requires that the contents of all
the columns be supplied at the time an entry in the list is created.
Microsoft Network Monitor reads big capture files much faster (and reads
them faster than it filters them); from that, from the calls to a
dissector that the SMS SDK documentation says are made, and from the
fact that MFC's equivalent widget apparently supports "lazy evaluation"
of the column data, where a routine is called (and presumably handed
some indication of which row is being displayed) when a row is to be
displayed, with that routine supplying the column data, I suspect NetMon
does as little work as possible when reading a file, and does a full
packet dissection only when a packet is to be displayed, printed, or
filtered.
Given a CList replacement that supports that (and, unless the GTK+ folk
manage to replace the CList widget with a newer shinier better widget,
there are other reasons why we might want to do a replacement that, in
addition to doing that form of lazy evaluation, also allows columns to
be added, deleted, and moved - and doesn't have the annoying quadratic
construction behavior that prompted us to put a patched version into the
Ethereal source), we could probably do something similar for Ethereal.
When reading the capture, enough work would be done to associate with
each frame all the information needed to dissect that frame *in addition
to* the raw contents of the frame, that being the sort of information
Richard wants to attach to the packet.
