Wireshark · Ethereal-dev: Re: [Ethereal-dev] Heuristic decode of RTP packets

[Guy Harris <gharris@xxxxxxxxx>]

On Wed, Jun 30, 2004 at 11:59:45PM -0700, Guy Harris wrote:
> On Wed, Jun 30, 2004 at 07:04:45PM +0200, Ruud Linders wrote:
> > Here is an updated patch, it is against last night's (2004-06-30) tar file.
> 
> Checked in.

Here's some mail from Ethereal lists discussing heuristics for
recognizing RTP, some of which mention additional tests that *might*
strengthen the heuristics enough to make them the default (or perhaps
they're strong enough now).

RTCP is also mentioned.

From: andreas.sikkema@xxxxxxxxxxx
To: ethereal-dev@xxxxxxxxxxxx
Subject: [Ethereal-dev] Re: [Ethereal-users] How to make ethereal recognise RTP,RTCP,SCTP over UDP
Date: Thu, 27 Jun 2002 09:22:34 +0200

> RTP and RTCP don't have standard port numbers, and nobody's figured out
> any heuristic to guess whether a UDP packet is an RTP or RTCP packet or
> not; 

Hmm, I was just thinking, 
- Check if the version is in the correct range (0-3 or something)
- Check if the length of the received UDP data is the same as the 
  length the RTP/RTCP message says it is
- For RTP you could also check if the mediatype is in the correct range.
- For RTCP you could check if walking the SSRC parts ends up at the end 
  of the message (or at the start of padding)
- Check if the padding stuff also matches? This one I'm not sure about.

This is all from the head, no RFC on hand at the moment, so....

But I don't have enough experience with heuristics to be able to predict 
how precise this is.

-- 
Andreas Sikkema
andreas.sikkema@xxxxxxxxxxx
"While you're waiting, read the free novel we sent you. 
 It's a Spanish story about a guy named `Manual'" - Dilbert



_______________________________________________
Ethereal-dev mailing list
Ethereal-dev@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-dev

From: Feyaerts Johan <Johan.Feyaerts@xxxxxxxxxxxxxxx>
To: "'andreas.sikkema@xxxxxxxxxxx'" <andreas.sikkema@xxxxxxxxxxx>
Cc: ethereal-dev@xxxxxxxxxxxx
Subject: RE: [Ethereal-dev] Re: [Ethereal-users] How to make ethereal reco
	gnise RTP,RTCP,SCTP over UDP
Date: Thu, 27 Jun 2002 09:30:36 +0200

I think the RFC also say somewhere that RTP runs on an even and RTCP on an
odd port number one higher than the RTP stream. (most implementations follow
this standard)
Checking sequence numbers that increase may also help but you'd have to look
at more than 1 packet then. 


It is possible to detect RTP/RTCP by means of heurisics. The Terayon
Highlink router (former radwiz) does it via some proprietary algorithm in
real time. So I think it should be possible to do a similar thing  offline
in ethereal.

_______________________________________________
Ethereal-dev mailing list
Ethereal-dev@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-dev

Subject: RE: [Ethereal-dev] Re: [Ethereal-users] How to make ethereal reco
	gnise RTP,RTCP,SCTP over UDP
From: Ed Warnicke <hagbard@xxxxxxxxxxxxxxxxxxx>
To: Feyaerts Johan <Johan.Feyaerts@xxxxxxxxxxxxxxx>
Cc: "'andreas.sikkema@xxxxxxxxxxx'" <andreas.sikkema@xxxxxxxxxxx>,
   ethereal-dev@xxxxxxxxxxxx
Date: 27 Jun 2002 09:02:48 -0400

You can't count on the RTP odd port RTCP even port as part of a=20
heuristic.  There are a LOT of RTP implementations that have decided
that RTCP isn't really necessary ( I happen to believe this means their=20
broken, but it gets done a lot ).  Also I seem to recall discussion=20
floating around about allowing the specification of port for RTCP=20
which are not the next port after the RTP traffic. =20

This Terayon Highlink router... are you sure it is recognizing the=20
RTP via a heuristic and not just watching the signalling traffic ( H323,
MGCP, SIP, RTSP etc ) to determine where to look?

Ed
On Thu, 2002-06-27 at 03:30, Feyaerts Johan wrote:
>
>
> I think the RFC also say somewhere that RTP runs on an even and RTCP on an
> odd port number one higher than the RTP stream. (most implementations follow
> this standard)
> Checking sequence numbers that increase may also help but you'd have to look
> at more than 1 packet then.
>
>
> It is possible to detect RTP/RTCP by means of heurisics. The Terayon
> Highlink router (former radwiz) does it via some proprietary algorithm in
> real time. So I think it should be possible to do a similar thing offline
> in ethereal.

_______________________________________________
Ethereal-dev mailing list
Ethereal-dev@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-dev

From: Feyaerts Johan <Johan.Feyaerts@xxxxxxxxxxxxxxx>
To: "'Ed Warnicke'" <hagbard@xxxxxxxxxxxxxxxxxxx>
Cc: "'ethereal-dev@xxxxxxxxxxxx'" <ethereal-dev@xxxxxxxxxxxx>
Subject: RE: [Ethereal-dev] Re: [Ethereal-users] How to make ethereal reco
	 gnise RTP,RTCP,SCTP over UDP
Date: Fri, 28 Jun 2002 09:28:12 +0200

I'm almost sure the router works via pure RTP heuristics because they
claimed independence of the signaling protocol.
I don't know what heuristic they used but it worked quite well when I did
some tests in the lab. They certainly do not look deep into the signaling
messages like e.g. a packeteer box does. 

Looking at the standard H.323 and SIP ports for presence of signalling
messages may also be used as an additional check for detecting RTP, but I
don't think that's really necessary.

> You can't count on the RTP odd port RTCP even port as part of a 
> heuristic.  There are a LOT of RTP implementations that have decided
> that RTCP isn't really necessary ( I happen to believe this 
> means their 
> broken, but it gets done a lot ). 
If they claim standard compliancy they can only use even ports on RTP. I
know some of the old radvision stacks were not compliant. Probably  you're
right that RTCP cannot be used because the standards give a lot of freedom
to when to send RTCP to the implementors. Some may indeed not even bother
about sending RTCP. If you want to use the presence of RCTP packets for the
heuristic it probably would be good to have a switch to disable this check.
Same probably applies for  even RTP port numbers. 
> Also I seem to recall discussion 
> floating around about allowing the specification of port for RTCP 
> which are not the next port after the RTP traffic.  
> 

> 
> 

_______________________________________________
Ethereal-dev mailing list
Ethereal-dev@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-dev

From: "Martin Regner" <martin.regner@xxxxxxxxx>
To: "Ubaldi Fabio" <ubaldi@xxxxxxxxxx>
Cc: <ethereal-users@xxxxxxxxxxxx>
Subject: Re: [Ethereal-users] RTP implementation by Ethereal
Date: Mon, 31 Mar 2003 20:15:20 +0200

Ubaldi Fabio wrote:
<I am using Ethereal (version 0.9.9) to capture packets from a video streaming transmission on the
<client side. The streaming server is the Darwin one, and the player is QuickTime.

<I'd like to know your opinion about the fact that Ethereal cannot display RTP/RTSP packets
< (embedded in UDP packets), even if such protocols are enabled on the server side and inside Ethereal.


http://www.ethereal.com/faq.html#q5.21

Ethereal should recognize RTSP packets if they are sent on tcp port 554.
It seems that Ethereal doesn't support RTSP over UDP.

Ethereal may decode RTP/RTCP packets automatically if it sees the RTSP/SDP messages
(or H.323 signalling in case of H.323 traffic). 

Otherwise you normally have to select one of those packages and use the menu item
"Decode As..." and select RTP as protocol (or RTCP).

I'm a bit tired on this, so I have written a plugin that does some heuristic checks on UDP
packets and calls the RTP/RTCP dissectors for packets that meets certain heuristics:
-Only packets with version bits equal to RTP version 2 are considered as possible RTP/RTCP packets.
-It is configurable if only RTP packets with CSRC counter value 0 shall be considered or if other values
 also should be considered.
-It is configurable if only packets without Extension Header shall be considered or if also packets
with Extension header should be considered.
-It is configurable if only RTP packets on even port numbers should be considered, and
odd port numbers for RTCP.
-Source and destination port values should be in certain ranges (configurable from preferences).
-RTP Payload should have certain values (configurable as a string "0;8;31;34;").
-The payload size should have certain normal values for e.g. G.711, GSM and G.723.1 
(this check is possible to turn-off).
-It is also possible to configure a list of ip-address that will need to match source and/or destination.
-For RTCP packets there are some more heuristic checks that can be done.
-The huristic dissection is possible to turn off with a preference setting.

I will send the source code to the list when I have added some more checks, cleaned the code up a bit
and tested it more. 
Is there someone that has some more ideas of how to do heuristic checks on certain payloads or similar?


<I have just written an e-mail few seconds ago about RTP support by Ethereal.
<I forgot an important detail: I am doing streaming using MPEG-4 coding, and 
<RTP version is done according to RFC 3016.

The MPEG-4 payload will however not be decoded. There is no MPEG4 dissector in Ethereal.

MPEG-4 is a dynamic payload so it's probably not possible to determine that it is MPEG-4
by just looking at the RTP packets. It could however be possible to determine this from the
RTSP/SDP messages, but currently this information is not passed to the RTP dissector (but I guess
it could be done in some way).

_______________________________________________
Ethereal-users mailing list
Ethereal-users@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-users

From: Guy Harris <guy@xxxxxxxxxxxx>
Subject: Re: [Ethereal-users] How do I decode RTP ?
Date: Thu, 28 Aug 2003 11:21:11 -0700
To: Chet Seligman <cseligman@xxxxxxxxxxxxx>
Cc: ethereal-users@xxxxxxxxxxxx

On Thursday, August 28, 2003, at 8:13 AM, Chet Seligman wrote:

> I have captured a few VoIP calls and the decode only shows them to be 
> UDP.
> I have done a "decode as" and selected a pair of ports as RTP (they 
> probably
> are but I'm not certain) and I get a decode.
>
> Is this the proper methodology?
>
> Is there a way to get ethereal to recognize which packets are RTP and
> without me knowing what ports are involved?

	http://www.ethereal.com/faq.html#q5.26

If somebody can come up with a "good" heuristic to implement 2) from 
the list in that FAQ, where "good" means "recognizes as many RTP 
packets as possible as being RTP *AND* mis-recognizes as *FEW* 
*NON*-RTP packets as possible as being RTP even though they aren't", we 
could add that to the RTP dissector.

Unfortunately, the "(they probably are but I'm not certain)" suggests 
that it wasn't obvious to you, from looking at the packet, whether it 
was RTP; if it's hard for a human to figure that out, it might be hard 
for a computer to do it as well....

There don't seem to be a lot of fields in the RTP fixed header that 
would always have the same value, or a value within a given range.  
There's the "V" field, although that's only 2 bits, so other packets 
might well have binary 10 in those two bits, and there's the "PT" 
field, for which we could require that it be a *known* payload type.  
Ethereal heuristics have to look at the packet in isolation (a 
cross-packet heuristic would be much more complicated - we'd have to 
implement some sort of "look-ahead", but, in the general case, I'm not 
sure you can implement a form of look-ahead that could recognize 
packets as being UDP and thus potential RTP packets without having to 
dissect those packets, in which case we'd have to worry about that 
dissection affecting packet state), so the sequence number doesn't seem 
to be usable.

_______________________________________________
Ethereal-users mailing list
Ethereal-users@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-users

From: "Chet Seligman" <cseligman@xxxxxxxxxxxxx>
To: "Guy Harris" <guy@xxxxxxxxxxxx>
Subject: RE: [Ethereal-users] How do I decode RTP ?
Date: Thu, 28 Aug 2003 11:50:15 -0700
Cc: ethereal-users@xxxxxxxxxxxx

Every RTP packet should have a payload type that lists the codec used.
In my case it would be G.729A or G.711, there are 3 or 4 other common ones

After I guessed which ports were used by RTP, I examined the decodes and saw
the G.729 payload - that's how I knew I was correct. What I didn't know was
whether I got all the packets. Not getting all packet makes jitter
measurements not terribly meaningful.

You speak of a "look-ahead" - could that be where the payload type is used?
BTW: Agilent Advisor and Radcom can identify RTP & RTCP without human help -
I don't know how they do it.

From: Guy Harris <guy@xxxxxxxxxxxx>
Subject: Re: [Ethereal-users] How do I decode RTP ?
Date: Thu, 28 Aug 2003 12:14:34 -0700
To: Chet Seligman <cseligman@xxxxxxxxxxxxx>
Cc: ethereal-users@xxxxxxxxxxxx

On Thursday, August 28, 2003, at 11:50 AM, Chet Seligman wrote:

> Every RTP packet should have a payload type that lists the codec used.

Yes, that's what I said:

> There's the "V" field, although that's only 2 bits, so other packets
> might well have binary 10 in those two bits, and there's the "PT"
> field, for which we could require that it be a *known* payload type.

Perhaps checking for "V" being 2, and the payload type being one of the 
values in the "rtp_payload_type_vals[]" table, would be a 
sufficiently-strong heuristic, accepting few, if any, non-RTP packets.  
(It means that we wouldn't recognize packets with a payload type we 
don't yet know about, but that's life.)

> After I guessed which ports were used by RTP, I examined the decodes 
> and saw
> the G.729 payload - that's how I knew I was correct. What I didn't 
> know was
> whether I got all the packets. Not getting all packet makes jitter
> measurements not terribly meaningful.

There's not much Ethereal can do about that, other than recognizing 
sequence number gaps.

> You speak of a "look-ahead" - could that be where the payload type is 
> used?

"Could" in what sense?  It's not where we use it, as we don't have any 
heuristics, much less ones involving look-ahead.  I don't know whether 
any of the dissectors that recognize RTP and RTCP without human help 
*AND* without seeing session setup packets (which, for at least some 
protocols, Ethereal dissects *and* uses to identify subsequent packets 
as RTP) use look-ahead or not.

The payload type could be used without look-ahead if you just check for 
a known value.  It could also be used *if* look-ahead were done, by 
checking to make sure it's the same in the later packets.  Note, 
however, that using the sequence number in a look-ahead scheme would 
require a "window" so that we don't get confused by lost packets, i.e. 
look for the N+1'st packet in that UDP stream having a sequence number 
no more than W greater than the Nth packet.

However, as far as I know, doing a look-ahead scheme would be very 
difficult, perhaps impossible; I don't plan to spend any time looking 
into it (I don't have the time to do so).

> BTW: Agilent Advisor and Radcom can identify RTP & RTCP without human 
> help -
> I don't know how they do it.

I'm aware that several analyzers do it.  That doesn't mean it's 
necessarily easy to figure out a good way to do it - maybe they do 
something simple that's easy to figure out, maybe they do something 
simple that was *hard* to figure out but easy to implement, and maybe 
they do something that's hard to implement.

_______________________________________________
Ethereal-users mailing list
Ethereal-users@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-users

From: Guy Harris <guy@xxxxxxxxxxxx>
Subject: Re: [Ethereal-users] How do I decode RTP ?
Date: Thu, 28 Aug 2003 11:21:11 -0700
To: Chet Seligman <cseligman@xxxxxxxxxxxxx>
Cc: ethereal-users@xxxxxxxxxxxx

On Thursday, August 28, 2003, at 8:13 AM, Chet Seligman wrote:

> I have captured a few VoIP calls and the decode only shows them to be 
> UDP.
> I have done a "decode as" and selected a pair of ports as RTP (they 
> probably
> are but I'm not certain) and I get a decode.
>
> Is this the proper methodology?
>
> Is there a way to get ethereal to recognize which packets are RTP and
> without me knowing what ports are involved?

	http://www.ethereal.com/faq.html#q5.26

If somebody can come up with a "good" heuristic to implement 2) from 
the list in that FAQ, where "good" means "recognizes as many RTP 
packets as possible as being RTP *AND* mis-recognizes as *FEW* 
*NON*-RTP packets as possible as being RTP even though they aren't", we 
could add that to the RTP dissector.

Unfortunately, the "(they probably are but I'm not certain)" suggests 
that it wasn't obvious to you, from looking at the packet, whether it 
was RTP; if it's hard for a human to figure that out, it might be hard 
for a computer to do it as well....

There don't seem to be a lot of fields in the RTP fixed header that 
would always have the same value, or a value within a given range.  
There's the "V" field, although that's only 2 bits, so other packets 
might well have binary 10 in those two bits, and there's the "PT" 
field, for which we could require that it be a *known* payload type.  
Ethereal heuristics have to look at the packet in isolation (a 
cross-packet heuristic would be much more complicated - we'd have to 
implement some sort of "look-ahead", but, in the general case, I'm not 
sure you can implement a form of look-ahead that could recognize 
packets as being UDP and thus potential RTP packets without having to 
dissect those packets, in which case we'd have to worry about that 
dissection affecting packet state), so the sequence number doesn't seem 
to be usable.

_______________________________________________
Ethereal-users mailing list
Ethereal-users@xxxxxxxxxxxx
http://www.ethereal.com/mailman/listinfo/ethereal-users