Wireshark · Wireshark-users: Re: [Wireshark-users] Using Tshark to Total Packet Sizes -> helpneeded

["Phillips, Christopher M" <cmphil@xxxxxxxxxxx>]

Initially I want a total for every 60 seconds.  I can then use that data to generate other running totals for what ever period I decide later.
I empty the hash after processing the 60 second capture file once I have used the data.  I did not want to post all the code as I thought would be too long but assume a my %ip_data_download_totals = (); outside of the pipe within the pipe after the download section there is a similar bit for upload etc.

I don't care about accuracy, bigint just lets me store a big number of bytes greater rather than 2^32 which I can then convert to MB/GB etc later.

Is it possible to -z to do what I want without running multiple tshark on the capture file?
Ie total bytes sent per IP in each direction for a set period
If so a general example would be welcome as I can't really see anything similar on the man page regarding this.  Closest I saw was 
tshark -r /sa_cap/psize_00003_20090615173333.cap -z io,stat,60,SUM\(frame.pkt_len\)frame.pkt_len

Chris

-----Original Message-----
From: wireshark-users-bounces@xxxxxxxxxxxxx [mailto:wireshark-users-bounces@xxxxxxxxxxxxx] On Behalf Of Martin Visser
Sent: 16 June 2009 12:45
To: Community support list for Wireshark
Subject: Re: [Wireshark-users] Using Tshark to Total Packet Sizes -> helpneeded

Just checking, but your code is producing a running total every 60
seconds (rather than a total for each 60 secs) so it will always get
bigger - is this what you intended?

Also you expressed concern about using Big Ints. Why not use floats -
do you really need the precision?

Finally (and I'm not knocking using Perl) it just that it might be
more efficient to use the builtin "-z conv,type[,filter]" statistics
capability builtin to tshark.

Regards, Martin

MartinVisser99@xxxxxxxxx



On Tue, Jun 16, 2009 at 8:57 PM, Phillips, Christopher
M<cmphil@xxxxxxxxxxx> wrote:
> Hello,
>
>
>
> I was hoping to get some advice from knowledgeable wireshark/tshark gurus
> out there.
>
>
>
> I have a tshark process capturing to file, changing to the next file every
> 60 seconds from a network card is receiving mirrored traffic from my switch
> (not used for management by host OS).
>
> I have written a script that takes the pipe from another tshark process
> which takes a completed 60 second capture file then displays the srcip dstip
> and Size (packet size I assume in bytes) loops around and adds up packet
> sizes per ip for both download and upload.
>
> By download and upload I mean I have a known ip subnet so download is
> unknownip -> knownipsubnet and upload the other way around.
>
> The script works fine the problem I am having is the figures I am getting.
>
> Each 60 second period the total of the packet sizes per ip are increasing by
> roughly 50% where I expect them to stay roughly the same with a bit of
> wiggle in either direction.
>
> Looking at the raw data from tshark packet sizes the max Size is 1514.
>
> I have manually added up the packet sizes to eliminate errors in my script.
>
>
>
> The ip range I interested in are all on 10Meg Full Duplex links but I within
> a couple of minutes they exceed their theoretical max.
>
>
>
> I am obviously confused about how tshark handles it packets perhaps repeats
> are being displayed?
>
> Can someone give me definition what Size %L actually is.
>
> Either way I'm stuck please help.
>
>
>
> This is my first time posting to this list so please forgive any newbie
> mistakes.
>
>
>
> Regards
>
>
>
> Chris
>
>
>
>
>
> Details of options, scripts etc used
>
> #####################################
>
>
>
> Host OS:
>
> Linux Ubuntu
>
>
>
> Network Card used for capture:
>
> Intel Corporation 82540EM Gigabit Ethernet Controller
>
>
>
> Traffic Source:
>
> Port mirror of a 1GB uplink from a router with traffic level generally
> around 350Mb/s
>
>
>
> Contents of .wireshark->pref:
>
> cp.check_checksum: FALSE
>
> tcp.desegment_tcp_streams: FALSE
>
> tcp.analyze_sequence_numbers: FALSE
>
> tcp.desegment_tcp_streams: FALSE
>
> ip.defragment: FALSE
>
> dns.desegment_dns_messages: FALSE
>
>
>
> Capture tshark process run with following options (I fork this off in perl):
>
> where $capture_duration is 60.
>
> tshark -i $capture_interface -b duration:$capture_duration -s 64 -f 'not
> broadcast and not multicast and not icmp' -n -w
> $path_capture_files$path_capture_prefix`;
>
>
>
>
>
> The tshark I use to display the capture file and pipe into my script:
>
> open (TSHARK_PIPE,"/usr/bin/tshark -n -r $file_capture -o
> column.format:\"\"Source\",\"\%s\",\"Destination\",\"\%d\",\"Size\",\"\%L\"\"
> not ipv6 |");
>
>
>
> I did try using the line below but it ran 60% slower than the above line
>
> Obviously speed is factor as my script must finish processing a capture file
> in less time than the capture duration time.
>
> open (TSHARK_PIPE,"/usr/bin/tshark -n -r $file_capture -T text -T fields -E
> separator=# -e ip.src -e ip.dst -e frame.pkt_len |");
>
>
>
> Small bit of my script to get clear idea of what I'm doing (IP address
> ranges have been changed to protect the innocent^^:
>
>
>
> while (<TSHARK_PIPE>) {
>
>                                 chomp;
>
>                                 my ($ipsrc,$spacer,$ipdst,$size) = split '
> ';
>
>                                 if (defined $size) {
>
>                                         # Does packet belong to DOWNLOAD ie.
> ipdst = 111.111.0.0/16
>
>                                         if ($ipdst =~
> /^111\.111\.\d+\.\d+$/) {
>
>                                                 if (exists
> $ip_data_download_totals{$ipdst}) {
>
>                                                         # Add the size of
> the packet on
>
>
> $ip_data_download_totals{$ipdst} += $size;
>
>                                                 } else {
>
>
>                                             $ip_data_download_totals{$ipdst}
> = $size;
>
>                                                 }
>
>
>
> I then take those totals for a 60 second period and add them onto another
> total for my longer period 10mins or so.
>
> I need to do it this way as larger total requires Bigints for sizes in bytes
> of that period and bigints are Slooow to process.
>
>
>
> Example of 60second totals for unknownip->single known ip in bytes:
>
> 112263545
>
> 164527439
>
> 229262902
>
> 278524189
>
> 332487210
>
>
>
>
>
> ___________________________________________________________________________
> Sent via:    Wireshark-users mailing list <wireshark-users@xxxxxxxxxxxxx>
> Archives:    http://www.wireshark.org/lists/wireshark-users
> Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
>             mailto:wireshark-users-request@xxxxxxxxxxxxx?subject=unsubscribe
>
___________________________________________________________________________
Sent via:    Wireshark-users mailing list <wireshark-users@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-users
Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
             mailto:wireshark-users-request@xxxxxxxxxxxxx?subject=unsubscribe