Wireshark · Wireshark-dev: Re: [Wireshark-dev] Windows build setup - Concept required

[Joerg Mayer <jmayer@xxxxxxxxx>]

Before replying to some of the points, I want to clarify something:
This is about problems I have encountered in a current setup (no "old"
libs package, current source, clean out of tree setup and build).
Also, when I mentioned setup-macosx.sh I didn't take into account that
some people who are not interested in OS X may not know what it does
(or at least not all the details):
- At the beginning it has a section that defines the package versions
  (similar to config.nmake but only once, as there is no need to differentiate
  between 32bit and 64bit).
- These source(!) packages are downloaded, unpacked, patches from the
  Wireshark sources (macosx-support-lib-patches/) are applied if necessary,
  they are compiled and then installed into a common tree.

On Thu, Dec 05, 2013 at 10:20:58PM +0000, Graham Bloice wrote:
> On 5 December 2013 19:59, Gerald Combs <gerald@xxxxxxxxxxxxx> wrote:
> 
> > On 12/4/13 12:27 PM, Joerg Mayer wrote:
> > > as Graham and I are working on getting the Windows build process to
> > > a) work at all and b) be on par with the current nmake build process
> > > we currently rely on the setup infrastructure of the nmake build. I
> > > really don't like porting the "nmake -f Makefile.nmake setup" to cmake.
> > > Not because it is hard to do but because the current setup has various
> > > shortcomings
> >
> > > 1) zlib is installed as source only
> > > 2) portaudio is installed as source only
> >
> > I know there are historical reasons for this but are they still valid?
> > Zlib packages are available from the OpenSUSE Build Service (linked
> > against msvcrt.dll) and Nuget (linked against multiple runtimes).
> > Portaudio is also available from OBS.

My complaint is a different one: Why don't we compile these two packages at
setup time?
At least for zlib it seems simple enough if you have cmake installed (see
README.cmake).

> I could try them out.
> 
> >
> > > 3) Every package is installed into its own subdirectory, sometimes with
> > >    its own structure
> >
> > CoApp has a standardized directory layout for libraries and include
> > files called "Pivots":
> >
> > http://coapp.org/reference/autopackage-ref.html#Pivots
> >
> > "Pivot" is a nicer name than "deep explodey directory tree" but it seems
> > sensible and worth adopting IMHO.
> >
> 
> I still haven't looked into CoApp.  How does it play with Chocolatey?

I have no problem with individual install dirs for each package and arch,
as long as all packages follow the same schema.

> > > 4) glib2 contains zlib headers that break windows builds
> > > 5) glib3 contains zlib headers that break windows builds
> >
> > This should be reported as a bug at
> > https://build.opensuse.org/package/show/windows:mingw:win64/mingw64-zlib
> > I'm sure patches are welcome.
> >
> 
> I shall try to do this.

My complaint is not about these problems but about the fact that the install
process doesn't fix it. Because of this we have to work around it in the
toplevel CMakeLists.txt

> > > 6) krb5 contains includes that export krb5-build internal flags and
> > >    thus cause warnings during compiles
> >
> > Is there a better library we can use for Kerberos? MIT, Secure
> > Endpoints, and Heimdal all seem to provide packages that
> > almost-but-not-quite meet our needs. I tried cross-compiling MIT
> > Kerberos and Heimdal using MingW in the past but didn't have much luck.

My complaint is not about these problems but about the fact that the install
process doesn't fix it - instead we work around this in the source
(packet-kerberos.c)

> > > 6) Except for gtk3 no packages provide compile (includes, cflags) or
> > >    linking (libs, ldflags) information.
> > > 7) glib3 contains pkg-config files, but they contain a wrong paths
> > >    and unuseable compiler (gcc) flags
> >
> > The existence of the .pc files depends on my packaging scripts and the
> > OBS .spec files for OBS-sourced packages. Adding them shouldn't be too
> > difficult. The tricky part is getting them to point to the correct
> > location on Windows. I'm not sure if we can use or modify the stock .pc
> > files or if we'd have to create our own.
> >
> 
> My experiments with pkg-config from the gtk2 bundle and from pkg-configlite
> and the gtk3 bundle .pc files shows that "it just works".  The bundle .pc
> files have odd *nixy type paths in them but pkg-config and CMake sort all
> that out (with my modified FindGTK3.cmake)

And with the version number I could now identify that ancient ;-) install
and reproduce it. They probably do something similar to what I do in cmake.
I'Ve checked in a fix to no longer convert windows paths.

> > > 8) The current setup process does not install QT
> >
> > I've been hesitant to switch this on since it's such a large download
> > and I'm not sure which "Qt" should be installed. The Qt project provides
> > official 32- and 64-bit installers for VS 2012 and a 32-bit installer
> > for VS 2010. We provide 32- and 64-bit packages for VS 2010. We sign the
> > EXEs and DLLs in our packages but the Qt project doesn't. I don't know
> > if anyone provides VS 2013 packages.
> >
> 
> Folks only have to download it as often as it changes! Or is the issue
> server resources?

Maybe we don't have to provide it, but we should check for it (env path,
maybe registry) and if we can't identify a matching Qt install we should
issue a message on which version to install and where to find it.

> > > 9) To build qtshark without wireshark still requires the installation
> > >    of gtk2 or gtk3 for glib, gmodule, gthread
> >
> > I think we should switch from the separate wireshark-gtk2 and
> > wireshark-qt-release directories to a common deployment directory, e.g.
> > "wireshark-deploy" or "wireshark-stage". This would presumably help to
> > unify our build targets.

Is this referring to the nightly builds? (I never looked at how they are
done in detail).

> Ack
> 
> 
> > (I'd also like to get rid of ui/qt/QtShark.pro at some point in favor of
> > CMake. Qt Creator supports CMake projects well enough, and having to
> > maintain QtShark.pro for different platforms is a pain.)

I won't mind :-)

> > > 10) The setup process does not allow for the simultanous installation
> > >    of gtk2 and gtk3
> >
> > Does GTK3 work well enough on Windows to drop GTK2? This would simplify
> > things quite a bit.
> >
> 
> It's OK for me but there are some folks complaining about it from the cheap
> seats :-)

Old marketing technique: Switch to GTK3 first and then people will love the
QT version even more ;-)

> > > 11) The installation of some build tools (python, cmake, cygwin-stuff
> > like
> > >     cat, bash) might be automated - depending on the setup script
> > language
> > >     maybe not all of them.
> > >
> > > So maybe something more similar to the macosx setup is wanted. Not maybe
> > > the compile-it-yourself approach but an installation into a standard
> > > directory structure.
> > >
> > > So what I'd like to have is a script (.bat or maybe Powershell) that
> > works
> > > similar to the macosx-setup.sh script:
> > >
> > > - Contain a list of packages and their versions
> > > - Download missing packages
> > > - Download missing tools
> > > - Install not-yet-installed packages (includes, libs) into a standard
> > >   directory structure
> > >
> > > Feedback, ideas, details anyone?
> >
> > Sounds good to me. I like Graham's suggestion of using Chocolatey/Nuget
> > since they're currently the best option for package management under
> > Windows. I'd be willing to convert our current Windows libraries to
> > Nuget packages, and have them conform to Coapp's directory layout[1].
> >
> > If we could find or create Chocolatey packages for Bison and Flex that
> > would go a long way toward dropping the requirement for Cygwin.

Aren't there perl or python versions of these tools yet ;-)

> The GNUWin32 project provides Bison and Flex and sometime in the past in my
> "Removing the Cygwin dependency" experiments I did get the GNUWin32
> binaries to do the job.  I'll have a hunt for that VM, hopefully I still
> have it somewhere.
>   I
> 
> >
> > [1] The end goal being that someone else take over maintenance of the
> > packages. Some men dream of the heavens. I dream of no longer having to
> > create third-party Windows development libraries.
> >
> >
> That would be nice, but if we are signing packages it might continue for
> some time yet.

I'll be offline for the next 3 days (long weekend), so no comments until monday
from me.

Thanks for all the comments

 Jörg
-- 
Joerg Mayer                                           <jmayer@xxxxxxxxx>
We are stuck with technology when what we really want is just stuff that
works. Some say that should read Microsoft instead of technology.