Wireshark · Wireshark-dev: [Wireshark-dev] ssl_private_decrypt fails to decrypt pre master secret (Linux only)

[Daniel Kabs <dkabs@xxxxxxxxxxx>]

Hello developers,

I've installed the pre-compiled version of wireshark 0.99.6 on Windows and 
compiled wireshark 0.99.6 on Linux (Debian "Etch" release 4.0).

Then I tried to decrypt the sample capture of an SSL connection provided 
in the Wireshark wiki:

  http://wiki.wireshark.org/SSL

SSL decryption works fine on Windows, but fails on Linux.

I ran a "diff" on both debug outputs to see where the first relevant 
differences occur:

The SSL debug output on both machines show that the private key file was 
loaded successfully.

Then the debug output continues with only minor differences in pointer 
addresses in the "association find" lines. I reckon this is owed to the 
different operating systems.

The output continues without relevant differences. Even the "pre master 
encrypted[128]" in frame #8 is the same.

After that, major differences show up!

On Windows:
  pcry_private_decrypt: stripping 79 bytes, decr_len 127
  decypted_unstrip_pre_master[127]:
  02 c8 3b d5 a5 24 3c 40 c7 6e 95 b9 46 da b2 79 
  b1 06 ec 61 2d f7 f5 4a b7 62 b6 33 4b b3 05 ef 
  90 14 59 72 08 d5 34 88 41 cc a6 96 f4 dd 97 9a 
  dc 3a 6e 92 1f 3a e4 6b 5b fb 3f ee 46 59 62 f3 
  f3 06 0f d1 1f f4 9d b2 29 08 c6 01 f5 c3 00 03 
  00 ff 84 56 6d a0 fb cc fd c6 c8 20 d5 f0 65 18 
  87 b0 44 45 9c e3 92 f0 4d 32 cd 41 85 10 24 cb 
  7a b3 01 36 3d 93 27 12 a4 7e 00 29 96 59 d8 
  pre master secret[48]:
  03 00 ff 84 56 6d a0 fb cc fd c6 c8 20 d5 f0 65 
  18 87 b0 44 45 9c e3 92 f0 4d 32 cd 41 85 10 24 
  cb 7a b3 01 36 3d 93 27 12 a4 7e 00 29 96 59 d8 

On Linux:
  pcry_private_decrypt: stripping 0 bytes, decr_len 128
  decypted_unstrip_pre_master[128]:
  6c 44 b7 6e 8a 8c c1 a6 51 1d fd ff 3e 2f 50 80 
  f3 90 27 86 fd 11 48 d6 9e 36 ed 46 98 30 7a 89 
  da c1 4b 56 b4 a7 08 06 98 f4 53 37 b1 9a 19 55 
  df b1 e1 c4 f2 8a 75 b0 47 51 7c 48 79 3f 6a 45 
  8a 8c d8 bd ec 92 df d9 7b d4 68 2b 44 b3 f1 90 
  38 39 cc fb ca c7 4d ce cb 70 22 62 64 50 d4 0a 
  4f 77 42 4b 4c 08 0f fa 6e 87 85 65 21 89 70 37 
  25 bc e0 48 20 0f 77 e1 0c 13 4d 7c 3e 86 5d f7 
  ssl_decrypt_pre_master_secret wrong pre_master_secret lenght (128,
  expected 48)
  dissect_ssl3_handshake can't decrypt pre master secret

With my limited SSL knowledge, I interpret these lines as follows:
The pre master secret has been decrypted. It contains padding data that 
needs to be discarded. The padding data ends with a byte containing zero. 
The padding is searched and stripped off from the decrypted data. 

On Windows, the padding is always 79 bytes and the pre master secret can 
be extracted successfully.

On Linux the decrypted data contains not padding at all. Even worse, if I 
restart wireshark and reload the capture data, 
the "decypted_unstrip_pre_master[128]:" is *always* different.

So I guess decryption in libgcrypt[1] is defective on my computer. I 
already downloaded the libgcrypt sources, compiled and ran the tests but 
to no avail.

Any ideas what else I can do about it? 

Cheers
Daniel

[1] libgcrypt version 1.2.3 api-version 1