Wireshark · Ethereal-users: RE: [Ethereal-users] Truly infinite capture

["McNutt, Justin M." <McNuttJ@xxxxxxxxxxxx>]

Yes, you do have to know what you're looking for, but that's a restriction I
don't mind imposing upon those who are asking for the statistics.  I know
beforehand that certain things are interesting.  Aside from that, they have
to tell me what they want to know before I can get the data.  <shrug>

The problem I have so far with ntop is that aside from some minor stability
problems, it has some rather egregious dissection problems.  I've found
three rather serious misinterpretations already, and have had no luck in
getting them resolved.  (Yes, I subscribed to the mailing list, and posted
there.)

I'm trying to figure out the code that classifies a packet as "IPX" so I can
fix it, but with limited free time, it's slow going.  :-(

--J

> -----Original Message-----
> From: Visser, Martin (SNO) [mailto:Martin.Visser@xxxxxxxxxx]
> Sent: Wednesday, December 06, 2000 2:30 PM
> Cc: 'ethereal-users@xxxxxxxxxxxx'
> Subject: RE: [Ethereal-users] Truly infinite capture
> 
> 
> As mentioned earlier this is basically what ntop does. I 
> guess you might
> want to look at adding finer grained protocol stats to do 
> this. I will be
> adding stats analysis as part of the graphical enhancements I'm doing
> <http://opensource.compaq.com/sourceforge/project/?group_id=16>
> 
> At the moment I'm just working on using the "colorization" 
> display filters
> to separate stats out.
> 
> I guess the problem is that if you use the ntop model, you 
> need to know what
> you are looking for up front, because you throw the raw data 
> away. Ethereal
> however allows you to analyse things iteratively, as it 
> retains all the raw
> data. 
> 
> Martin
> 
> Martin Visser
> Technology Consultant - Compaq Global Services
> 
> Compaq Computer Australia
> 410 Concord Road
> Rhodes, Sydney NSW 2138
> Australia
> 
> Phone: +61-2-9022-5630
> Mobile: +61-411-254-513
> Fax:+61-2-9022-7001
> Email:martin.visser@xxxxxxxxxx
> 
> 
> -----Original Message-----
> From: McNutt, Justin M. [mailto:McNuttJ@xxxxxxxxxxxx]
> Sent: Monday, 4 December 2000 4:14 AM
> To: 'Guy Harris'
> Cc: 'ethereal-users@xxxxxxxxxxxx'
> Subject: RE: [Ethereal-users] Truly infinite capture
> 
> 
> > I.e., you want all of Ethereal except for the part that dissects
> > packets? :-)
> 
> Not exactly.  I want all of Ethereal, but once it has 
> dissected a packet, I
> want it to keep the analysis in a counter and throw away the 
> actual packet.
> Therefore, the capture can run almost infinitely (to the limit of the
> counter, which would need to be a 32-bit counter in my case).
> 
> The reason is this:  I have a fast, highly-utilized connection (the
> connection from MU to the Internet).  It puts about 2.7 
> million packets
> through this link every five minutes.  Now there's NO WAY I'm 
> going to be
> able to capture that much data, but if I could get a 
> breakdown of what was
> IN those packets at some level, that would be incredibly 
> useful.  E.g.:
> 
> ICMP: 10,514 (0.1%)
> HTTP: 1,302,351 (55.2%)
> DNS: 25,124 (0.2%)
> Other TCP: 401,415 (4.2%)
> 
> ...etc.
> 
> In this case, I don't want to know what the URL was, or what 
> name was looked
> up, or even the src/dst IP addresses.  I just want to know 
> what *kind* of
> packets went through the link and how many.  If the majority 
> is HTTP, I can
> justify buying a Web caching server and setting up a 
> transparent proxy.
> However, if it's all FTP, I need to find out where it's 
> coming from (usually
> somebody's hacked RedHat 6.1 vanilla install).
> 
> > That might be a useful program, but I'm not sure it needs to 
> > be part of
> > Ethereal; the statistics shown in the capture window are - quite
> > intentionally - fairly limited (the idea is that you don't 
> want to do
> > any dissection past the transport layer while you're 
> > capturing, as doing
> > more dissection work to figure out the protocols used *above* the
> > transport layer could significantly increase the CPU 
> requirements), so
> > it's not clear how much of Ethereal's code would actually be useful.
> 
> Just to the TCP/UDP port number, which implicitly gives you 
> the higher-level
> protocol (port 80 = HTTP, port 25 = SMTP).  In this case, we 
> can trust the
> port number since the number of things masquerading on a port 
> are going to
> be the insignificant minority of traffic (and could be listed as a
> limitation of the stats-only mode).
> 
> > A more detailed breakdown, showing how much of the traffic 
> > was HTTP and
> > how much was NFS and how much was SMTP and so on, might be 
> useful, and
> > (especially if it does *all* the analysis that Ethereal does, 
> > including
> > the heuristic tests, watching traffic for protocol A to see if it's
> > saying that a subsequent connection will be using protocol B, 
> > etc.), and
> > *would* use the Ethereal dissection code, but I'm not certain 
> > you'd want
> > that by default during a regular Ethereal capture.
> 
> No, you certainly wouldn't want it by default.  It would have to be a
> checkbox that said something like "Statistics Only", and a tab in the
> preferences window that allowed you to pick and choose the 
> protocols you
> wanted to watch for.  Then if a user selects a layer 7 
> protocol, a warning
> could come up when you click OK that says, "You're asking for 
> it, buddy."
> 
> After that, Ethereal should forgo the heuristics and connection-level
> analysis that it usually does and simply take each packet and 
> say, "Which
> counters need to be incremented?", increment them, and then 
> toss the packet
> into the bit bucket.
> 
> Of course, the stats window would need to persist after you 
> click Stop so
> you could write down/store your totals.  Click once to Stop, 
> change Stop
> button to Close.  Click again to destroy the window.  In 
> fact, that might be
> useful for every capture (perhaps as a selectable option... 
> "Stats window
> persists" (checkbox)).
> 
> Personally, I'd go GET a machine with more CPU to throw at 
> this if that were
> the problem, but that's just me.  I have the luxury of a 1GHz 
> P-III.  Would
> that be enough horsepower to try this out, perhaps on a 
> slower connection
> first, and then on the real McCoy?
> 
> --J
> 
> P.S.  I'd be happy to help write this code myself, but I'll 
> need a little
> help getting started.  It's been years since I wrote C 
> (although I've been
> writing a lot of Perl lately, which helps).
> 
> _______________________________________________
> Ethereal-users mailing list
> Ethereal-users@xxxxxxxxxxxx
> http://www.ethereal.com/mailman/listinfo/ethereal-users
> 
> _______________________________________________
> Ethereal-users mailing list
> Ethereal-users@xxxxxxxxxxxx
> http://www.ethereal.com/mailman/listinfo/ethereal-users
>