1
0
mirror of https://github.com/qTox/qTox.git synced 2024-03-22 14:00:36 +08:00
qTox/windows/cross-compile/build.sh
Mick Sayson 8abe8320d2 chore(CI): Use docker for CI scripts
Motiviation:
* Reproducing issues in CI is currently difficult
* Predicting issues in CI is currently difficult if you are not on
  ubuntu 18.04
* Reproducing issues submitted from other distros is currently done by
  creating a VM of that distro and building qtox for it locally
* Documentation for how to build on different distros is out of date
* Issues on non-ubuntu distributions are not caught by CI
* Cross compiling for windows locally is not trivial
* Iterating when working with custom build scripts is slow, scripts
  don't necessarily support re-running without re-starting the docker
  container and re-building qtox again
* Updating dependencies is a pain

Changes:
* docker-compose file has been added to the root of our repo.
  After `docker compose run --rm ubuntu` (or other supported distros),
  you are ready to compile and run qtox
* Dependencies are owned by dependency install scripts in buildscripts/.
  This allows us to use the same exact dependencies in our
  OSX/windows/linux scripts
* New docker images have been added for a variety of distributions.
  These are now run in CI in a variety of configurations
  * Docker images are cached in CI so rebuild time for the majority of
    jobs is quite quick
* Build scripts have been trimmed to leverage state of docker
  containers.
  * Windows build script no longer installs anything, dependencies are
    now managed by the windows_builder docker images
  * Build scripts should now be easily re-runnable. Usage is now `docker
    compose run --rm <image>` and then run the scripts
* All artifacts are now uploaded to github after build, this means we
  can take an appimage/flatpak/exe/dmg for any given PR and try it out
  without having to build it ourselves

Notes:
* Docker image size is quite important. We have a maximum of 5GB cache
  space on github actions. The majority of the linux distro docker
  images cache at ~300-400MB, which gives us room to test ~6 distros
  after accounting for the sizes of flatpak/windows docker images
* Docker layer ordering is relatively intentional. Approximate order
  should be that large dependencies that change infrequently should be
  farther up. This lowers the amount of rebuilding we have to do when
  dependencies are updated
* download_xxx.sh scripts are the cleanest way I could find to implement
  a shared dependency map between osx scripts and docker containers.
  Although it would be nice to have a single dependency mapping file,
  splitting it into individual scripts allows us to only rebuild some
  docker layers when dependencies are updated.
* Github actions are split between docker image building and docker
  image use. This allows us to re-use the same docker images for
  multiple jobs, but only build it once
  * Unfortunately I could not find a way to de-duplicate the stitching
    between jobs, so we have a lot of copy pasta in that area
2021-12-19 14:56:05 -08:00

217 lines
6.3 KiB
Bash
Executable File

#!/usr/bin/env bash
# MIT License
#
# Copyright (c) 2017-2021 Maxim Biro <nurupo.contributions@gmail.com>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# Known issues:
# - Doesn't build qTox updater, because it wasn't ported to cmake yet and
# because it requires static Qt, which means we'd need to build Qt twice, and
# building Qt takes really long time.
set -euo pipefail
while (( $# > 0 )); do
case $1 in
--src-dir) QTOX_SRC_DIR=$2; shift 2 ;;
--arch) ARCH=$2; shift 2 ;;
--build-type) BUILD_TYPE=$2; shift 2;;
*) "Unexpected argument $1"; exit 1 ;;
esac
done
# Common directory paths
QTOX_PREFIX_DIR="$(realpath install-prefix)"
readonly QTOX_PREFIX_DIR
QTOX_PACKAGE_DIR="$(realpath package-prefix)"
readonly QTOX_PACKAGE_DIR
if [ -z "$ARCH" ]
then
echo "Error: No architecture was specified. Please specify either 'i686' or 'x86_64', case sensitive, as the first argument to the script."
exit 1
fi
if [[ "$ARCH" != "i686" ]] && [[ "$ARCH" != "x86_64" ]]
then
echo "Error: Incorrect architecture was specified. Please specify either 'i686' or 'x86_64', case sensitive, as the first argument to the script."
exit 1
fi
if [ -z "$QTOX_SRC_DIR" ]; then
echo "--src-dir must be specified"
fi
if [ -z "$BUILD_TYPE" ]
then
echo "Error: No build type was specified. Please specify either 'release' or 'debug', case sensitive, as the second argument to the script."
exit 1
fi
if [[ "$BUILD_TYPE" != "release" ]] && [[ "$BUILD_TYPE" != "debug" ]]
then
echo "Error: Incorrect build type was specified. Please specify either 'release' or 'debug', case sensitive, as the second argument to the script."
exit 1
fi
# Spell check on windows currently not supported, disable
if [[ "$BUILD_TYPE" == "release" ]]
then
cmake -DCMAKE_TOOLCHAIN_FILE=/build/windows-toolchain.cmake \
-DCMAKE_PREFIX_PATH=/windows \
-DCMAKE_BUILD_TYPE=Release \
-DSPELL_CHECK=OFF \
-DDESKTOP_NOTIFICATIONS=ON \
-DUPDATE_CHECK=ON \
-DSTRICT_OPTIONS=ON \
"$QTOX_SRC_DIR"
elif [[ "$BUILD_TYPE" == "debug" ]]
then
cmake -DCMAKE_TOOLCHAIN_FILE=/build/windows-toolchain.cmake \
-DCMAKE_PREFIX_PATH=/windows \
-DCMAKE_BUILD_TYPE=Debug \
-DSPELL_CHECK=OFF \
-DDESKTOP_NOTIFICATIONS=ON \
-DUPDATE_CHECK=ON \
-DSTRICT_OPTIONS=ON \
"$QTOX_SRC_DIR"
fi
make -j $(nproc)
mkdir -p "${QTOX_PREFIX_DIR}"
cp qtox.exe $QTOX_PREFIX_DIR
cp -r /export/* $QTOX_PREFIX_DIR
# Run tests (only on Travis)
set +u
if [[ -n "$TRAVIS_CI_STAGE" ]]
then
export WINEPATH='/export;/windows/bin'
export CTEST_OUTPUT_ON_FAILURE=1
make test
fi
set -u
# Strip
set +e
if [[ "$BUILD_TYPE" == "release" ]]
then
$ARCH-w64-mingw32-strip -s $QTOX_PREFIX_DIR/*.exe
fi
$ARCH-w64-mingw32-strip -s $QTOX_PREFIX_DIR/*.dll
$ARCH-w64-mingw32-strip -s $QTOX_PREFIX_DIR/*/*.dll
set -e
if [[ "$BUILD_TYPE" == "debug" ]]
then
cp -r /debug_export/* ${QTOX_PREFIX_DIR}
fi
# Create zip
pushd $QTOX_PREFIX_DIR
zip qtox-"$ARCH"-"$BUILD_TYPE".zip -r *
popd
# Create installer
if [[ "$BUILD_TYPE" == "release" ]]
then
mkdir -p $QTOX_PACKAGE_DIR
pushd $QTOX_PACKAGE_DIR
# The installer creation script expects all the files to be in qtox/*
mkdir -p qtox
cp -r $QTOX_PREFIX_DIR/* ./qtox
rm ./qtox/*.zip
cp -r $QTOX_SRC_DIR/windows/* .
# Select the installer script for the correct architecture
if [[ "$ARCH" == "i686" ]]
then
makensis qtox.nsi
elif [[ "$ARCH" == "x86_64" ]]
then
makensis qtox64.nsi
fi
popd
fi
# dll check
# Create lists of all .exe and .dll files
find "$QTOX_PREFIX_DIR" -iname '*.dll' > dlls
find "$QTOX_PREFIX_DIR" -iname '*.exe' > exes
# Create a list of dlls that are loaded during the runtime (not listed in the PE
# import table, thus ldd doesn't print those)
echo "$QTOX_PREFIX_DIR/libsnore-qt5/libsnore_backend_windowstoast.dll
$QTOX_PREFIX_DIR/iconengines/qsvgicon.dll
$QTOX_PREFIX_DIR/imageformats/qgif.dll
$QTOX_PREFIX_DIR/imageformats/qico.dll
$QTOX_PREFIX_DIR/imageformats/qjpeg.dll
$QTOX_PREFIX_DIR/imageformats/qsvg.dll
$QTOX_PREFIX_DIR/platforms/qdirect2d.dll
$QTOX_PREFIX_DIR/platforms/qminimal.dll
$QTOX_PREFIX_DIR/platforms/qoffscreen.dll
$QTOX_PREFIX_DIR/platforms/qwindows.dll" > runtime-dlls
if [[ "$ARCH" == "i686" ]]
then
echo "$QTOX_PREFIX_DIR/libssl-1_1.dll" >> runtime-dlls
elif [[ "$ARCH" == "x86_64" ]]
then
echo "$QTOX_PREFIX_DIR/libssl-1_1-x64.dll" >> runtime-dlls
fi
# Create a tree of all required dlls
# Assumes all .exe files are directly in $QTOX_PREFIX_DIR, not in subdirs
while IFS= read -r line
do
if [[ "$ARCH" == "i686" ]]
then
WINE_DLL_DIR="/root/.wine/drive_c/windows/system32"
elif [[ "$ARCH" == "x86_64" ]]
then
WINE_DLL_DIR="/root/.wine/drive_c/windows/system32 /root/.wine/drive_c/windows/syswow64"
fi
python3 /usr/local/bin/mingw-ldd.py $line --dll-lookup-dirs $QTOX_PREFIX_DIR $WINE_DLL_DIR --output-format tree >> dlls-required
done < <(cat exes runtime-dlls)
# Check that no extra dlls get bundled
while IFS= read -r line
do
if ! grep -q "$line" dlls-required
then
echo "Error: extra dll included: $line. If this is a mistake and the dll is actually needed (e.g. it's loaded at runtime), please add it to the runtime dll list."
exit 1
fi
done < dlls
# Check that no dll is missing
if grep -q 'not found' dlls-required
then
cat dlls-required
echo "Error: Missing some dlls."
exit 1
fi