Wireshark · Wireshark-users: Re: [Wireshark-users] Network spikes..

[andreas@xxxxxxxxx]

On Fri, 24 Feb 2017, Stephan Viljoen wrote:

> I was wondering whether there's a Network guru around who's brain I can pick.
>
> I'm running a small ISP and I'm using custom build Linux routers to
> get the job done but I've started noticing some oddities on the
> network for the past week. We have around 650Mbp/s of total Internet
> Capacity but usually averages between 450 to 550Mbp/s during the day
> but I started seeing network spikes every 10 to 20 seconds pushing my
> usage on my core firewall to over 1GB but only for a few seconds after
> which it drops back down to normal.
>
> The other strange part is , I'm only seeing the bandwidth spikes on my
> core router between my inner (eth5) and outer (eth0) Interface. These
> traffic spikes isn't visible on my edge router where my upstream
> providers flow into. I'm also not seeing these spikes on any of my
> MRTG graphs. Perhaps I'm missing something here but in my
> understanding my core Firewalls traffic should match the traffic on my
> edge firewall right ?
>
> So in short , my outgoing flow on eth5  which is essentially incoming
> traffic for my customers will spike to 1Gb but the increase traffic
> isn't visible on my edge router .
>
> My outer (eth0) Interface plugs into my bandwidth manager which in
> turn plugs into my edge routers inner (eno2) where all our upstream
> providers are flowing into (eno1) . I've done a few packet captures
> with tcpdump and imported it into Wireshark but I'm not really 100%
> what to look for.
>
> Any ideas on what might be causing these spikes would be greatly appreciated. .
>
> I used Nload on each interface to get some realtime statistics.
>
> The below values were recorded on my Core firewall.
> Eth0 (Outer Interface , goes to Bandwidth manager which goes to my
> edge firewall (eno2)
> Nload stats on interface Eth0
> Curr: 301.95 MBit/s
> Avg: 431.78 MBit/s
> Min: 0.00 Bit/s
> Max: 1.43 GBit/s
> Ttl: 787808.69 GByte
>
>
> Eth5 (Inner Interface , plugs into a switch which feeds my customers)
> Nload stats on interface eth5
> Curr: 385.77 MBit/s
> Avg: 399.40 MBit/s
> Min: 262.46 MBit/s
> Max: 1.29 GBit/s
> Ttl: 27408.75 GByte
>
> And these values were recorded on my Inner interface (eno2) of my edge router.
>
> eno1 WAN (This is my Wan Interface to the outside world)
> Edge Firewall / router (Centos 7);
> eno2 LAN (This interface plugs into my bandwidth manager which in turn
> plugs into my Core firewalls outer interface(eth0)
>
> Nload stats on interface eno2
> Curr: 336.97 MBit/s
> Avg: 381.66 MBit/s
> Min: 309.01 MBit/s
> Max: 548.93 MBit/s
> Ttl: 504389.72 GByte

Did you try breaking down the traffic mix? E.g. how much tcp, udp, icmp, 
other. Then which types of icmp, what dest ports tcp/udp. When a curious 
traffic spike occurs, check in which section is falls then dig deeper; 
e.g. if tcp/https is the culprit, what are the most common src and dest 
ips and does this change when a spike occurs.

Knowing what your typical traffic mix is can be handy in many situations 
where you need to ask yourself "is this normal?".

Regards
Andreas