StarMirror/qTox
1
0
Fork 0
mirror of https://github.com/qTox/qTox.git synced 2024-03-22 14:00:36 +08:00
Code Issues Projects Releases Wiki Activity
0c967725df
qTox/buildscripts/build_toxcore.sh

72 lines
1.7 KiB
Bash
Raw Normal View History

chore(CI): Use docker for CI scripts Motiviation: * Reproducing issues in CI is currently difficult * Predicting issues in CI is currently difficult if you are not on ubuntu 18.04 * Reproducing issues submitted from other distros is currently done by creating a VM of that distro and building qtox for it locally * Documentation for how to build on different distros is out of date * Issues on non-ubuntu distributions are not caught by CI * Cross compiling for windows locally is not trivial * Iterating when working with custom build scripts is slow, scripts don't necessarily support re-running without re-starting the docker container and re-building qtox again * Updating dependencies is a pain Changes: * docker-compose file has been added to the root of our repo. After `docker compose run --rm ubuntu` (or other supported distros), you are ready to compile and run qtox * Dependencies are owned by dependency install scripts in buildscripts/. This allows us to use the same exact dependencies in our OSX/windows/linux scripts * New docker images have been added for a variety of distributions. These are now run in CI in a variety of configurations * Docker images are cached in CI so rebuild time for the majority of jobs is quite quick * Build scripts have been trimmed to leverage state of docker containers. * Windows build script no longer installs anything, dependencies are now managed by the windows_builder docker images * Build scripts should now be easily re-runnable. Usage is now `docker compose run --rm <image>` and then run the scripts * All artifacts are now uploaded to github after build, this means we can take an appimage/flatpak/exe/dmg for any given PR and try it out without having to build it ourselves Notes: * Docker image size is quite important. We have a maximum of 5GB cache space on github actions. The majority of the linux distro docker images cache at ~300-400MB, which gives us room to test ~6 distros after accounting for the sizes of flatpak/windows docker images * Docker layer ordering is relatively intentional. Approximate order should be that large dependencies that change infrequently should be farther up. This lowers the amount of rebuilding we have to do when dependencies are updated * download_xxx.sh scripts are the cleanest way I could find to implement a shared dependency map between osx scripts and docker containers. Although it would be nice to have a single dependency mapping file, splitting it into individual scripts allows us to only rebuild some docker layers when dependencies are updated. * Github actions are split between docker image building and docker image use. This allows us to re-use the same docker images for multiple jobs, but only build it once * Unfortunately I could not find a way to de-duplicate the stitching between jobs, so we have a lot of copy pasta in that area
2021-11-08 13:27:45 +08:00
#!/bin/bash
fix(buildscripts): Fix license text for MIT derived works
2022-01-10 10:46:52 +08:00
# SPDX-License-Identifier: GPL-3.0-or-later AND MIT
# Copyright (c) 2017-2021 Maxim Biro <nurupo.contributions@gmail.com>
# Copyright (c) 2021 by The qTox Project Contributors
chore(CI): Use docker for CI scripts Motiviation: * Reproducing issues in CI is currently difficult * Predicting issues in CI is currently difficult if you are not on ubuntu 18.04 * Reproducing issues submitted from other distros is currently done by creating a VM of that distro and building qtox for it locally * Documentation for how to build on different distros is out of date * Issues on non-ubuntu distributions are not caught by CI * Cross compiling for windows locally is not trivial * Iterating when working with custom build scripts is slow, scripts don't necessarily support re-running without re-starting the docker container and re-building qtox again * Updating dependencies is a pain Changes: * docker-compose file has been added to the root of our repo. After `docker compose run --rm ubuntu` (or other supported distros), you are ready to compile and run qtox * Dependencies are owned by dependency install scripts in buildscripts/. This allows us to use the same exact dependencies in our OSX/windows/linux scripts * New docker images have been added for a variety of distributions. These are now run in CI in a variety of configurations * Docker images are cached in CI so rebuild time for the majority of jobs is quite quick * Build scripts have been trimmed to leverage state of docker containers. * Windows build script no longer installs anything, dependencies are now managed by the windows_builder docker images * Build scripts should now be easily re-runnable. Usage is now `docker compose run --rm <image>` and then run the scripts * All artifacts are now uploaded to github after build, this means we can take an appimage/flatpak/exe/dmg for any given PR and try it out without having to build it ourselves Notes: * Docker image size is quite important. We have a maximum of 5GB cache space on github actions. The majority of the linux distro docker images cache at ~300-400MB, which gives us room to test ~6 distros after accounting for the sizes of flatpak/windows docker images * Docker layer ordering is relatively intentional. Approximate order should be that large dependencies that change infrequently should be farther up. This lowers the amount of rebuilding we have to do when dependencies are updated * download_xxx.sh scripts are the cleanest way I could find to implement a shared dependency map between osx scripts and docker containers. Although it would be nice to have a single dependency mapping file, splitting it into individual scripts allows us to only rebuild some docker layers when dependencies are updated. * Github actions are split between docker image building and docker image use. This allows us to re-use the same docker images for multiple jobs, but only build it once * Unfortunately I could not find a way to de-duplicate the stitching between jobs, so we have a lot of copy pasta in that area
2021-11-08 13:27:45 +08:00
set -euo pipefail
chore(CI): Handle build scripts called with relative paths with spaces foo=dirname "$0" only works if the script is called with an absolute path and that path contains no spaces. Add quotes everywhere to contain spaces in directory names or file names, and add realpath to get the absolute path regardless of call path. Not necessarily needed, but like in e40586803743c9f08a9018caccf8038f754a31e4 it can avoid some annoying issues when e.g. that script was called with a relative path in CI.
2022-03-05 19:25:45 +08:00
readonly SCRIPT_DIR="$(dirname "$(realpath "$0")")"
chore(CI): Use docker for CI scripts Motiviation: * Reproducing issues in CI is currently difficult * Predicting issues in CI is currently difficult if you are not on ubuntu 18.04 * Reproducing issues submitted from other distros is currently done by creating a VM of that distro and building qtox for it locally * Documentation for how to build on different distros is out of date * Issues on non-ubuntu distributions are not caught by CI * Cross compiling for windows locally is not trivial * Iterating when working with custom build scripts is slow, scripts don't necessarily support re-running without re-starting the docker container and re-building qtox again * Updating dependencies is a pain Changes: * docker-compose file has been added to the root of our repo. After `docker compose run --rm ubuntu` (or other supported distros), you are ready to compile and run qtox * Dependencies are owned by dependency install scripts in buildscripts/. This allows us to use the same exact dependencies in our OSX/windows/linux scripts * New docker images have been added for a variety of distributions. These are now run in CI in a variety of configurations * Docker images are cached in CI so rebuild time for the majority of jobs is quite quick * Build scripts have been trimmed to leverage state of docker containers. * Windows build script no longer installs anything, dependencies are now managed by the windows_builder docker images * Build scripts should now be easily re-runnable. Usage is now `docker compose run --rm <image>` and then run the scripts * All artifacts are now uploaded to github after build, this means we can take an appimage/flatpak/exe/dmg for any given PR and try it out without having to build it ourselves Notes: * Docker image size is quite important. We have a maximum of 5GB cache space on github actions. The majority of the linux distro docker images cache at ~300-400MB, which gives us room to test ~6 distros after accounting for the sizes of flatpak/windows docker images * Docker layer ordering is relatively intentional. Approximate order should be that large dependencies that change infrequently should be farther up. This lowers the amount of rebuilding we have to do when dependencies are updated * download_xxx.sh scripts are the cleanest way I could find to implement a shared dependency map between osx scripts and docker containers. Although it would be nice to have a single dependency mapping file, splitting it into individual scripts allows us to only rebuild some docker layers when dependencies are updated. * Github actions are split between docker image building and docker image use. This allows us to re-use the same docker images for multiple jobs, but only build it once * Unfortunately I could not find a way to de-duplicate the stitching between jobs, so we have a lot of copy pasta in that area
2021-11-08 13:27:45 +08:00
build_toxcore() {
TOXCORE_SRC="$(realpath toxcore)"
mkdir -p "$TOXCORE_SRC"
pushd $TOXCORE_SRC >/dev/null || exit 1
chore(CI): Handle build scripts called with relative paths with spaces foo=dirname "$0" only works if the script is called with an absolute path and that path contains no spaces. Add quotes everywhere to contain spaces in directory names or file names, and add realpath to get the absolute path regardless of call path. Not necessarily needed, but like in e40586803743c9f08a9018caccf8038f754a31e4 it can avoid some annoying issues when e.g. that script was called with a relative path in CI.
2022-03-05 19:25:45 +08:00
"${SCRIPT_DIR}/download/download_toxcore.sh"
chore(CI): Use docker for CI scripts Motiviation: * Reproducing issues in CI is currently difficult * Predicting issues in CI is currently difficult if you are not on ubuntu 18.04 * Reproducing issues submitted from other distros is currently done by creating a VM of that distro and building qtox for it locally * Documentation for how to build on different distros is out of date * Issues on non-ubuntu distributions are not caught by CI * Cross compiling for windows locally is not trivial * Iterating when working with custom build scripts is slow, scripts don't necessarily support re-running without re-starting the docker container and re-building qtox again * Updating dependencies is a pain Changes: * docker-compose file has been added to the root of our repo. After `docker compose run --rm ubuntu` (or other supported distros), you are ready to compile and run qtox * Dependencies are owned by dependency install scripts in buildscripts/. This allows us to use the same exact dependencies in our OSX/windows/linux scripts * New docker images have been added for a variety of distributions. These are now run in CI in a variety of configurations * Docker images are cached in CI so rebuild time for the majority of jobs is quite quick * Build scripts have been trimmed to leverage state of docker containers. * Windows build script no longer installs anything, dependencies are now managed by the windows_builder docker images * Build scripts should now be easily re-runnable. Usage is now `docker compose run --rm <image>` and then run the scripts * All artifacts are now uploaded to github after build, this means we can take an appimage/flatpak/exe/dmg for any given PR and try it out without having to build it ourselves Notes: * Docker image size is quite important. We have a maximum of 5GB cache space on github actions. The majority of the linux distro docker images cache at ~300-400MB, which gives us room to test ~6 distros after accounting for the sizes of flatpak/windows docker images * Docker layer ordering is relatively intentional. Approximate order should be that large dependencies that change infrequently should be farther up. This lowers the amount of rebuilding we have to do when dependencies are updated * download_xxx.sh scripts are the cleanest way I could find to implement a shared dependency map between osx scripts and docker containers. Although it would be nice to have a single dependency mapping file, splitting it into individual scripts allows us to only rebuild some docker layers when dependencies are updated. * Github actions are split between docker image building and docker image use. This allows us to re-use the same docker images for multiple jobs, but only build it once * Unfortunately I could not find a way to de-duplicate the stitching between jobs, so we have a lot of copy pasta in that area
2021-11-08 13:27:45 +08:00
cmake -DCMAKE_INSTALL_PREFIX=/windows/ \
-DBOOTSTRAP_DAEMON=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_STATIC=OFF \
-DENABLE_SHARED=ON \
-DCMAKE_TOOLCHAIN_FILE=/build/windows-toolchain.cmake \
.
cmake --build . -- -j$(nproc)
cmake --build . --target install
popd >/dev/null
}
build_toxext() {
TOXEXT_SRC="$(realpath toxext)"
mkdir -p "$TOXEXT_SRC"
pushd $TOXEXT_SRC >/dev/null || exit 1
chore(CI): Handle build scripts called with relative paths with spaces foo=dirname "$0" only works if the script is called with an absolute path and that path contains no spaces. Add quotes everywhere to contain spaces in directory names or file names, and add realpath to get the absolute path regardless of call path. Not necessarily needed, but like in e40586803743c9f08a9018caccf8038f754a31e4 it can avoid some annoying issues when e.g. that script was called with a relative path in CI.
2022-03-05 19:25:45 +08:00
"${SCRIPT_DIR}/download/download_toxext.sh"
chore(CI): Use docker for CI scripts Motiviation: * Reproducing issues in CI is currently difficult * Predicting issues in CI is currently difficult if you are not on ubuntu 18.04 * Reproducing issues submitted from other distros is currently done by creating a VM of that distro and building qtox for it locally * Documentation for how to build on different distros is out of date * Issues on non-ubuntu distributions are not caught by CI * Cross compiling for windows locally is not trivial * Iterating when working with custom build scripts is slow, scripts don't necessarily support re-running without re-starting the docker container and re-building qtox again * Updating dependencies is a pain Changes: * docker-compose file has been added to the root of our repo. After `docker compose run --rm ubuntu` (or other supported distros), you are ready to compile and run qtox * Dependencies are owned by dependency install scripts in buildscripts/. This allows us to use the same exact dependencies in our OSX/windows/linux scripts * New docker images have been added for a variety of distributions. These are now run in CI in a variety of configurations * Docker images are cached in CI so rebuild time for the majority of jobs is quite quick * Build scripts have been trimmed to leverage state of docker containers. * Windows build script no longer installs anything, dependencies are now managed by the windows_builder docker images * Build scripts should now be easily re-runnable. Usage is now `docker compose run --rm <image>` and then run the scripts * All artifacts are now uploaded to github after build, this means we can take an appimage/flatpak/exe/dmg for any given PR and try it out without having to build it ourselves Notes: * Docker image size is quite important. We have a maximum of 5GB cache space on github actions. The majority of the linux distro docker images cache at ~300-400MB, which gives us room to test ~6 distros after accounting for the sizes of flatpak/windows docker images * Docker layer ordering is relatively intentional. Approximate order should be that large dependencies that change infrequently should be farther up. This lowers the amount of rebuilding we have to do when dependencies are updated * download_xxx.sh scripts are the cleanest way I could find to implement a shared dependency map between osx scripts and docker containers. Although it would be nice to have a single dependency mapping file, splitting it into individual scripts allows us to only rebuild some docker layers when dependencies are updated. * Github actions are split between docker image building and docker image use. This allows us to re-use the same docker images for multiple jobs, but only build it once * Unfortunately I could not find a way to de-duplicate the stitching between jobs, so we have a lot of copy pasta in that area
2021-11-08 13:27:45 +08:00
cmake -DCMAKE_INSTALL_PREFIX=/windows/ \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_TOOLCHAIN_FILE=/build/windows-toolchain.cmake \
.
cmake --build . -- -j$(nproc)
cmake --build . --target install
popd >/dev/null
}
build_toxext_messages() {
TOXEXT_MESSAGES_SRC="$(realpath toxext_messages)"
mkdir -p "$TOXEXT_MESSAGES_SRC"
pushd $TOXEXT_MESSAGES_SRC > /dev/null || exit 1
chore(CI): Handle build scripts called with relative paths with spaces foo=dirname "$0" only works if the script is called with an absolute path and that path contains no spaces. Add quotes everywhere to contain spaces in directory names or file names, and add realpath to get the absolute path regardless of call path. Not necessarily needed, but like in e40586803743c9f08a9018caccf8038f754a31e4 it can avoid some annoying issues when e.g. that script was called with a relative path in CI.
2022-03-05 19:25:45 +08:00
"${SCRIPT_DIR}/download/download_toxext_messages.sh"
chore(CI): Use docker for CI scripts Motiviation: * Reproducing issues in CI is currently difficult * Predicting issues in CI is currently difficult if you are not on ubuntu 18.04 * Reproducing issues submitted from other distros is currently done by creating a VM of that distro and building qtox for it locally * Documentation for how to build on different distros is out of date * Issues on non-ubuntu distributions are not caught by CI * Cross compiling for windows locally is not trivial * Iterating when working with custom build scripts is slow, scripts don't necessarily support re-running without re-starting the docker container and re-building qtox again * Updating dependencies is a pain Changes: * docker-compose file has been added to the root of our repo. After `docker compose run --rm ubuntu` (or other supported distros), you are ready to compile and run qtox * Dependencies are owned by dependency install scripts in buildscripts/. This allows us to use the same exact dependencies in our OSX/windows/linux scripts * New docker images have been added for a variety of distributions. These are now run in CI in a variety of configurations * Docker images are cached in CI so rebuild time for the majority of jobs is quite quick * Build scripts have been trimmed to leverage state of docker containers. * Windows build script no longer installs anything, dependencies are now managed by the windows_builder docker images * Build scripts should now be easily re-runnable. Usage is now `docker compose run --rm <image>` and then run the scripts * All artifacts are now uploaded to github after build, this means we can take an appimage/flatpak/exe/dmg for any given PR and try it out without having to build it ourselves Notes: * Docker image size is quite important. We have a maximum of 5GB cache space on github actions. The majority of the linux distro docker images cache at ~300-400MB, which gives us room to test ~6 distros after accounting for the sizes of flatpak/windows docker images * Docker layer ordering is relatively intentional. Approximate order should be that large dependencies that change infrequently should be farther up. This lowers the amount of rebuilding we have to do when dependencies are updated * download_xxx.sh scripts are the cleanest way I could find to implement a shared dependency map between osx scripts and docker containers. Although it would be nice to have a single dependency mapping file, splitting it into individual scripts allows us to only rebuild some docker layers when dependencies are updated. * Github actions are split between docker image building and docker image use. This allows us to re-use the same docker images for multiple jobs, but only build it once * Unfortunately I could not find a way to de-duplicate the stitching between jobs, so we have a lot of copy pasta in that area
2021-11-08 13:27:45 +08:00
cmake -DCMAKE_INSTALL_PREFIX=/windows/ \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_TOOLCHAIN_FILE=/build/windows-toolchain.cmake \
.
cmake --build . --target install
popd >/dev/null
}
build_toxcore
build_toxext
build_toxext_messages
Reference in New Issue Copy Permalink