Compare commits

...

15 Commits

Author SHA1 Message Date
Maxim Biro b3c3c49d26
fix: Disable IPv6 in Windows cross-compilation tests
Docker doesn't have an IPv6 network by default, so these tests were
failing when using IPv6.
2024-03-04 19:26:37 -05:00
Maxim Biro e742deddff
feat: Check hashes of Windows dependencies when cross-compiling
It's a disableable option since we allow the user to change versions of
dependencies and we obviously have hashes only for the default versions,
we are not able to verify hashes of any other version, so it might be
handy to be able to disable the check in that case.
2024-03-04 15:18:03 -05:00
Maxim Biro dfb9a0b02b
fix: Test the current Windows Dockerfile, not an old Dockerhub image
A bit infuriating that when you submit a PR modifying the Windows
cross-compilation script, the CI *builds* the current image in one job,
but then *runs* an older (!) image from Dockerhub in another. So it
tests that the modified image builds successfully but doesn't test if it
runs successfully, instead testing if an older image runs successfully /
testing if the current toxcore changes run successfully in an older
image.

This should build and run the same image with the current changes.
2024-03-04 15:18:01 -05:00
Maxim Biro 14de93ccec
chore: Use WineHQ's Wine as Debian Bookworm's crashes 2024-03-04 15:11:59 -05:00
Maxim Biro ed37616249
docs: Update the Windows cross-compilation section 2024-03-04 15:11:56 -05:00
Maxim Biro 9bb79c174f
cleanup: Remove a couple of unnecessary misc_tools dependencies
These programs link against libsodium, which already provides bin-to-hex
and hex-to-bin conversion functions. Removing the misc_tools dependency
shaves ~30KiB (in some cases 10%) off Windows static binaries using it.
2024-03-04 15:11:55 -05:00
Maxim Biro 19475adb70
chore: Statically link OpenMP into the cracker fun util on Windows
For some reason even with -DCMAKE_EXE_LINKER_FLAGS="-static" specified,
CMake still links OpenMP dynamically, even though it links other libs
statically, e.g. -pthread.
2024-03-04 15:11:54 -05:00
Maxim Biro 1be311e51f
feat: Build the fun utils when cross-compiling to Windows
While fun utils are of very low quality, both code-wise and usage-wise
-- not checking for failed mallocs, not offering usage instructions,
etc., there are a couple of them that Windows users might find useful,
like the vanity key generators or a savedata creator, for example.

The building of the fun utils was broken on Windows due to the utils
failing to find sodium.h, as no libsodium include dirs were set on the
fun utils.
2024-03-04 15:11:52 -05:00
Maxim Biro 88133f8446
chore: Strip Windows binaries 2024-03-04 15:11:51 -05:00
Maxim Biro 3cc0ae7535
refactor: Copy over all of the required static dependencies
Including libwinpthread and libssp.
2024-03-04 15:11:49 -05:00
Maxim Biro c4fa8f7fb1
feat: Generate .def, .exp and .lib files when building for Windows 2024-03-04 15:11:48 -05:00
Maxim Biro 74bbac5363
feat: Let CMake create the dll instead of doing so ourselves
We used to make the dll manually as we previously had 3 dlls:
libtoxcore.dll, libtoxav.dll and libtoxencryptsave.dll, but for Windows
we wanted them to be all combined into libtox.dll with all the
dependencies included: libsodium, libopus, libvpx, pthreads, etc, to
reduce the overall dll size and simplify linking. However, since CMake
now produces a single libtoxcore.dll with toxcore, toxav and
toxencryptsave included, we don't have to do this manually anymore.

This results in the dll being named libtoxcore.dll instead of the
libtox.dll that it previously was, matching the static libtoxcore.a's
name.
2024-03-04 15:11:46 -05:00
Maxim Biro 246642e9ae
feat: Harden Windows cross-compilation
The same hardening flags as the slated for the upcoming gcc's
`-fhardened` sets, sans `-Wl,-z,relro,-z,now` as MinGW-w64's gcc doesn't
support -z flags. This adds a dependency on libssp -- gcc's stack
protector library.
2024-03-04 15:11:45 -05:00
Maxim Biro 8d431c0d11
chore: Bump Windows build dependency versions 2024-03-04 15:11:43 -05:00
Maxim Biro e519f7998b
fix: Remove unnecessary wsock32 dependency on Windows
Looks like it was added by a mistake:

1. We didn't have this dependency before, in fact, autotools still
   doesn't have it, it was added only to CMake.

2. During my testing, when building the echo_bot example, there are no
   unresolved symbols left after linking against `libtox.a libsodium.a
   -pthread -lws2_32 -liphlpapi`, which indicates that `-lwsock32` is
   unnecessary.
2024-02-28 16:42:55 -05:00
17 changed files with 361 additions and 156 deletions

View File

@ -18,7 +18,7 @@ docker run \
-e ENABLE_ARCH_i686="$i686" \
-e ENABLE_ARCH_x86_64="$x86_64" \
-e ENABLE_TEST=true \
-e EXTRA_CMAKE_FLAGS="-DBOOTSTRAP_DAEMON=OFF -DMIN_LOGGER_LEVEL=DEBUG -DTEST_TIMEOUT_SECONDS=90 -DAUTOTEST=ON" \
-e EXTRA_CMAKE_FLAGS="-DBOOTSTRAP_DAEMON=OFF -DMIN_LOGGER_LEVEL=DEBUG -DTEST_TIMEOUT_SECONDS=90 -DAUTOTEST=ON -DUSE_IPV6=OFF" \
-e CMAKE_C_FLAGS="$C_FLAGS" \
-e CMAKE_CXX_FLAGS="$CXX_FLAGS" \
-e CMAKE_EXE_LINKER_FLAGS="$LD_FLAGS" \
@ -26,4 +26,6 @@ docker run \
-v "$PWD:/toxcore" \
-v "$PWD/result:/prefix" \
--rm \
-t \
--pull never \
"toxchat/windows:$WINDOWS_ARCH"

View File

@ -83,7 +83,7 @@ jobs:
- name: Build and test
run: .github/scripts/cmake-osx
build-msvc:
build-windows-msvc:
strategy:
matrix:
version: [2019, 2022]
@ -110,18 +110,6 @@ jobs:
cd _build
ctest -j50 --output-on-failure --rerun-failed --repeat until-pass:6 --build-config Debug
build-windows:
strategy:
matrix:
bits: [32, 64]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Cross compilation
run: .github/scripts/cmake-win${{ matrix.bits }} script
build-netbsd:
runs-on: ubuntu-latest
steps:

View File

@ -138,9 +138,15 @@ jobs:
cache-from: type=registry,ref=toxchat/c-toxcore:esp32
cache-to: type=inline
docker-win32:
docker-windows-mingw:
strategy:
matrix:
bits: [32, 64]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
@ -149,39 +155,27 @@ jobs:
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
- name: Build and store to local Docker daemon
uses: docker/build-push-action@v5
with:
context: "{{defaultContext}}:other/docker/windows"
push: ${{ github.event_name == 'push' }}
tags: toxchat/windows:win32
cache-from: type=registry,ref=toxchat/windows:win32
cache-to: type=inline
context: other/docker/windows
load: true
tags: toxchat/windows:win${{ matrix.bits }}
cache-from: type=registry,ref=toxchat/windows:win${{ matrix.bits }}
build-args: |
SUPPORT_ARCH_i686=true
SUPPORT_ARCH_x86_64=false
SUPPORT_ARCH_i686=${{ matrix.bits == '32' }}
SUPPORT_ARCH_x86_64=${{ matrix.bits == '64' }}
SUPPORT_TEST=true
docker-win64:
runs-on: ubuntu-latest
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
- name: Push the stored image to Dockerhub
if: ${{ github.event_name == 'push' }}
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: "{{defaultContext}}:other/docker/windows"
context: other/docker/windows
push: ${{ github.event_name == 'push' }}
tags: toxchat/windows:win64
cache-from: type=registry,ref=toxchat/windows:win64
cache-to: type=inline
tags: toxchat/windows:win${{ matrix.bits }}
build-args: |
SUPPORT_ARCH_i686=false
SUPPORT_ARCH_x86_64=true
SUPPORT_ARCH_i686=${{ matrix.bits == '32' }}
SUPPORT_ARCH_x86_64=${{ matrix.bits == '64' }}
SUPPORT_TEST=true
- name: Cross-compile
run: .github/scripts/cmake-win${{ matrix.bits }} script

View File

@ -454,7 +454,7 @@ elseif(TARGET Threads::Threads)
set(toxcore_LINK_LIBRARIES ${toxcore_LINK_LIBRARIES} Threads::Threads)
endif()
if(WIN32)
set(toxcore_LINK_LIBRARIES ${toxcore_LINK_LIBRARIES} iphlpapi wsock32 ws2_32)
set(toxcore_LINK_LIBRARIES ${toxcore_LINK_LIBRARIES} iphlpapi ws2_32)
endif()
################################################################################

View File

@ -299,15 +299,15 @@ Get the toxcore source code and navigate to `other/docker/windows`.
Build the container image based on the Dockerfile. The following options are
available to customize the building of the container image.
| Name | Description | Expected Value | Default Value |
| --------------------- | --------------------------------------------------- | ----------------------------------- | ------------- |
| `SUPPORT_ARCH_i686` | Support building 32-bit toxcore. | "true" or "false" (case sensitive). | true |
| `SUPPORT_ARCH_x86_64` | Support building 64-bit toxcore. | "true" or "false" (case sensitive). | true |
| `SUPPORT_TEST` | Support running toxcore automated tests. | "true" or "false" (case sensitive). | false |
| `CROSS_COMPILE` | Cross-compiling. True for Docker, false for Cygwin. | "true" or "false" (case sensitive). | true |
| `VERSION_OPUS` | Version of libopus to build toxcore with. | Numeric version number. | 1.3.1 |
| `VERSION_SODIUM` | Version of libsodium to build toxcore with. | Numeric version number. | 1.0.18 |
| `VERSION_VPX` | Version of libvpx to build toxcore with. | Numeric version number. | 1.11.0 |
| Name | Description | Expected Value | Default Value |
| -------------------------- | ----------------------------------------------------- | ----------------------------------- | ------------- |
| `SUPPORT_ARCH_i686` | Support building 32-bit toxcore. | "true" or "false" (case sensitive). | true |
| `SUPPORT_ARCH_x86_64` | Support building 64-bit toxcore. | "true" or "false" (case sensitive). | true |
| `SUPPORT_TEST` | Support running toxcore automated tests. | "true" or "false" (case sensitive). | false |
| `VERSION_OPUS` | Version of libopus to build toxcore with. | Numeric version number. | 1.4 |
| `VERSION_SODIUM` | Version of libsodium to build toxcore with. | Numeric version number. | 1.0.19 |
| `VERSION_VPX` | Version of libvpx to build toxcore with. | Numeric version number. | 1.14.0 |
| `ENABLE_HASH_VERIFICATION` | Verify the hashes of the default dependency versions. | "true" or "false" (case sensitive). | true |
Example of building a container image with options
@ -322,14 +322,13 @@ docker build \
Run the container to build toxcore. The following options are available to
customize the running of the container image.
| Name | Description | Expected Value | Default Value |
| -------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------- | --------------------------- |
| `ALLOW_TEST_FAILURE` | Don't stop if a test suite fails. | "true" or "false" (case sensitive). | `false` |
| `ENABLE_ARCH_i686` | Build 32-bit toxcore. The image should have been built with `SUPPORT_ARCH_i686` enabled. | "true" or "false" (case sensitive). | `true` |
| `ENABLE_ARCH_x86_64` | Build 64-bit toxcore. The image should have been built with `SUPPORT_ARCH_x86_64` enabled. | "true" or "false" (case sensitive). | `true` |
| `ENABLE_TEST` | Run the test suite. The image should have been built with `SUPPORT_TEST` enabled. | "true" or "false" (case sensitive). | `false` |
| `EXTRA_CMAKE_FLAGS` | Extra arguments to pass to the CMake command when building toxcore. | CMake options. | `-DTEST_TIMEOUT_SECONDS=90` |
| `CROSS_COMPILE` | Cross-compiling. True for Docker, false for Cygwin. | "true" or "false" (case sensitive). | `true` |
| Name | Description | Expected Value | Default Value |
| -------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------- | ------------------------------------------ |
| `ALLOW_TEST_FAILURE` | Don't stop if a test suite fails. | "true" or "false" (case sensitive). | `false` |
| `ENABLE_ARCH_i686` | Build 32-bit toxcore. The image should have been built with `SUPPORT_ARCH_i686` enabled. | "true" or "false" (case sensitive). | `true` |
| `ENABLE_ARCH_x86_64` | Build 64-bit toxcore. The image should have been built with `SUPPORT_ARCH_x86_64` enabled. | "true" or "false" (case sensitive). | `true` |
| `ENABLE_TEST` | Run the test suite. The image should have been built with `SUPPORT_TEST` enabled. | "true" or "false" (case sensitive). | `false` |
| `EXTRA_CMAKE_FLAGS` | Extra arguments to pass to the CMake command when building toxcore. | CMake options. | `-DTEST_TIMEOUT_SECONDS=90 -DUSE_IPV6=OFF` |
Example of running the container with options
@ -339,6 +338,7 @@ docker run \
-e ALLOW_TEST_FAILURE=true \
-v /path/to/toxcore/sourcecode:/toxcore \
-v /path/to/where/output/build/result:/prefix \
-t \
--rm \
toxcore
```
@ -346,6 +346,101 @@ docker run \
After the build succeeds, you should see the built toxcore libraries in
`/path/to/where/output/build/result`.
The file structure should look similar to the following
```
result
├── [4.0K] i686
│   ├── [ 36K] bin
│   │   ├── [636K] DHT_bootstrap.exe
│   │   ├── [572K] cracker.exe
│   │   ├── [359K] cracker_simple.exe
│   │   ├── [378K] create_bootstrap_keys.exe
│   │   ├── [378K] create_minimal_savedata.exe
│   │   ├── [958K] create_savedata.exe
│   │   ├── [ 18K] libtoxcore.def
│   │   ├── [2.6M] libtoxcore.dll
│   │   ├── [ 65K] libtoxcore.exp
│   │   ├── [428K] libtoxcore.lib
│   │   ├── [989K] save-generator.exe
│   │   ├── [381K] sign.exe
│   │   └── [408K] strkey.exe
│   ├── [4.0K] include
│   │   └── [4.0K] tox
│   │   ├── [177K] tox.h
│   │   ├── [ 10K] tox_dispatch.h
│   │   ├── [ 26K] tox_events.h
│   │   ├── [6.4K] tox_private.h
│   │   ├── [ 26K] toxav.h
│   │   └── [ 12K] toxencryptsave.h
│   └── [4.0K] lib
│   ├── [577K] libopus.a
│   ├── [660K] libsodium.a
│   ├── [ 10K] libssp.a
│   ├── [1016K] libtoxcore.a
│   ├── [456K] libtoxcore.dll.a
│   ├── [2.7M] libvpx.a
│   ├── [ 72K] libwinpthread.a
│   └── [4.0K] pkgconfig
│   ├── [ 250] libsodium.pc
│   ├── [ 357] opus.pc
│   ├── [ 247] toxcore.pc
│   └── [ 309] vpx.pc
└── [4.0K] x86_64
├── [ 36K] bin
│   ├── [504K] DHT_bootstrap.exe
│   ├── [474K] cracker.exe
│   ├── [277K] cracker_simple.exe
│   ├── [287K] create_bootstrap_keys.exe
│   ├── [288K] create_minimal_savedata.exe
│   ├── [769K] create_savedata.exe
│   ├── [ 18K] libtoxcore.def
│   ├── [2.4M] libtoxcore.dll
│   ├── [ 64K] libtoxcore.exp
│   ├── [420K] libtoxcore.lib
│   ├── [800K] save-generator.exe
│   ├── [289K] sign.exe
│   └── [317K] strkey.exe
├── [4.0K] include
│   └── [4.0K] tox
│   ├── [177K] tox.h
│   ├── [ 10K] tox_dispatch.h
│   ├── [ 26K] tox_events.h
│   ├── [6.4K] tox_private.h
│   ├── [ 26K] toxav.h
│   └── [ 12K] toxencryptsave.h
└── [4.0K] lib
├── [697K] libopus.a
├── [575K] libsodium.a
├── [ 11K] libssp.a
├── [905K] libtoxcore.a
├── [449K] libtoxcore.dll.a
├── [2.9M] libvpx.a
├── [ 68K] libwinpthread.a
└── [4.0K] pkgconfig
├── [ 252] libsodium.pc
├── [ 359] opus.pc
├── [ 249] toxcore.pc
└── [ 311] vpx.pc
12 directories, 60 files
```
- `libtoxcore.dll` is the shared library. It is fully self-contained, with no
additional dependencies aside from the Windows OS dlls, and can be used in
MSVC, MinGW, Clang, etc. projects. Despite its name, it provides toxcore,
toxav, toxencryptsave -- all of Tox.
- `libtoxcore.a` is the static library. In order to use it, it needs to be
linked against the other provided .a libraries (but not the .dll.a!) and
additionally -liphlpapi and -lws2_32 Windows dlls. It similarly provides all
of Tox APIs.
- `libtoxcore.dll.a` is the MinGW import library for `libtoxcore.dll`.
- `libtoxcore.lib` is the MSVC import library for `libtoxcore.dll`.
- `libtoxcore.exp` and `libtoxcore.def` are the exported by `libtoxcore`
symbols.
- `*.exe` are statically compiled executables -- `DHT_bootstrap` and
[the fun utils](#secondary).
## Pre-built binaries
### Linux

View File

@ -48,8 +48,8 @@ function(_make_version_script target)
endfunction()
option(STRICT_ABI "Enforce strict ABI export in dynamic libraries" OFF)
if(WIN32 OR APPLE)
# Windows and OSX don't have this linker functionality.
if((WIN32 AND NOT MINGW) OR APPLE)
# Windows and macOS don't have this linker functionality.
set(STRICT_ABI OFF)
endif()

View File

@ -1,10 +1,13 @@
FROM debian:bullseye-slim
FROM debian:bookworm-slim
# When editing, make sure to update /other/windows_build_script_toxcore.sh and
# INSTALL.md to match.
# Build-time environment variables
ARG VERSION_MSGPACK=4.0.0 \
ARG VERSION_OPUS=1.4 \
VERSION_SODIUM=1.0.19 \
VERSION_OPUS=1.3.1 \
VERSION_VPX=1.11.0 \
VERSION_VPX=1.14.0 \
ENABLE_HASH_VERIFICATION=true \
\
SUPPORT_TEST=false \
SUPPORT_ARCH_i686=true \
@ -18,6 +21,7 @@ ENV SUPPORT_TEST=${SUPPORT_TEST} \
CROSS_COMPILE=${CROSS_COMPILE}
WORKDIR /work
COPY check_sha256.sh .
COPY get_packages.sh .
RUN ./get_packages.sh
@ -30,6 +34,6 @@ ENV ENABLE_TEST=false \
ALLOW_TEST_FAILURE=false \
ENABLE_ARCH_i686=true \
ENABLE_ARCH_x86_64=true \
EXTRA_CMAKE_FLAGS="-DTEST_TIMEOUT_SECONDS=90"
EXTRA_CMAKE_FLAGS="-DTEST_TIMEOUT_SECONDS=90 -DUSE_IPV6=OFF"
ENTRYPOINT ["bash", "./build_toxcore.sh"]

View File

@ -7,6 +7,8 @@ fi
#=== Cross-Compile Dependencies ===
. ./check_sha256.sh
build() {
ARCH=${1}
@ -21,7 +23,7 @@ build() {
mkdir -p "$PREFIX_DIR"
export MAKEFLAGS=j"$(nproc)"
export CFLAGS=-O3
export CFLAGS="-O3 -D_FORTIFY_SOURCE=3 -D_GLIBCXX_ASSERTIONS -ftrivial-auto-var-init=zero -fPIE -pie -fstack-protector-strong -fstack-clash-protection -fcf-protection=full"
CURL_OPTIONS=(-L --connect-timeout 10)
@ -41,19 +43,42 @@ build() {
echo
echo "=== Building Sodium $VERSION_SODIUM $ARCH ==="
curl "${CURL_OPTIONS[@]}" -O "https://github.com/jedisct1/libsodium/releases/download/$VERSION_SODIUM-RELEASE/libsodium-$VERSION_SODIUM.tar.gz"
check_sha256 "018d79fe0a045cca07331d37bd0cb57b2e838c51bc48fd837a1472e50068bbea" "libsodium-$VERSION_SODIUM.tar.gz"
tar -xf "libsodium-$VERSION_SODIUM.tar.gz"
cd "libsodium-stable"
./configure --host="$WINDOWS_TOOLCHAIN" --prefix="$PREFIX_DIR" --disable-shared --enable-static
./configure \
--host="$WINDOWS_TOOLCHAIN" \
--prefix="$PREFIX_DIR" \
--disable-shared \
--enable-static
make
make install
cd ..
echo
echo "=== Building Opus $VERSION_OPUS $ARCH ==="
curl "${CURL_OPTIONS[@]}" -O "https://archive.mozilla.org/pub/opus/opus-$VERSION_OPUS.tar.gz"
if [ "$ARCH" = "i686" ]; then
LIB_OPUS_CFLAGS=""
else
# This makes the build work with -fstack-clash-protection, as otherwise it crashes with:
# silk/float/encode_frame_FLP.c: In function 'silk_encode_frame_FLP':
# silk/float/encode_frame_FLP.c:379:1: internal compiler error: in i386_pe_seh_unwind_emit, at config/i386/winnt.cc:1274
# Should get patched in a future gcc version: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458
LIB_OPUS_CFLAGS="-fno-asynchronous-unwind-tables"
fi
curl "${CURL_OPTIONS[@]}" -O "https://ftp.osuosl.org/pub/xiph/releases/opus/opus-$VERSION_OPUS.tar.gz"
check_sha256 "c9b32b4253be5ae63d1ff16eea06b94b5f0f2951b7a02aceef58e3a3ce49c51f" "opus-$VERSION_OPUS.tar.gz"
tar -xf "opus-$VERSION_OPUS.tar.gz"
cd "opus-$VERSION_OPUS"
./configure --host="$WINDOWS_TOOLCHAIN" --prefix="$PREFIX_DIR" --disable-extra-programs --disable-doc --disable-shared --enable-static
CFLAGS="$CFLAGS $LIB_OPUS_CFLAGS" \
./configure \
--host="$WINDOWS_TOOLCHAIN" \
--prefix="$PREFIX_DIR" \
--disable-extra-programs \
--disable-doc \
--disable-shared \
--enable-static
make
make install
cd ..
@ -72,9 +97,19 @@ build() {
LIB_VPX_CFLAGS="-fno-asynchronous-unwind-tables"
fi
curl "${CURL_OPTIONS[@]}" "https://github.com/webmproject/libvpx/archive/v$VERSION_VPX.tar.gz" -o "libvpx-$VERSION_VPX.tar.gz"
check_sha256 "5f21d2db27071c8a46f1725928a10227ae45c5cd1cad3727e4aafbe476e321fa" "libvpx-$VERSION_VPX.tar.gz"
tar -xf "libvpx-$VERSION_VPX.tar.gz"
cd "libvpx-$VERSION_VPX"
CFLAGS="$LIB_VPX_CFLAGS" CROSS="$WINDOWS_TOOLCHAIN"- ./configure --target="$LIB_VPX_TARGET" --prefix="$PREFIX_DIR" --disable-examples --disable-unit-tests --disable-shared --enable-static
CFLAGS="$CFLAGS $LIB_VPX_CFLAGS" \
CROSS="$WINDOWS_TOOLCHAIN"- \
./configure \
--target="$LIB_VPX_TARGET" \
--prefix="$PREFIX_DIR" \
--disable-examples \
--disable-unit-tests \
--disable-tools \
--disable-shared \
--enable-static
make
make install
cd ..

111
other/docker/windows/build_toxcore.sh Normal file → Executable file
View File

@ -2,6 +2,9 @@
set -e -x
# Note: when modifying this script, don't forget to update the appropriate
# parts of the cross-compilation section of the INSTALL.md.
#=== Cross-Compile Toxcore ===
build() {
@ -23,13 +26,8 @@ build() {
rm -rf /tmp/*
# where to install static/shared toxcores before deciding whether they should be copied over to the user
STATIC_TOXCORE_PREFIX_DIR="/tmp/static_prefix"
SHARED_TOXCORE_PREFIX_DIR="/tmp/shared_prefix"
mkdir -p "$STATIC_TOXCORE_PREFIX_DIR" "$SHARED_TOXCORE_PREFIX_DIR"
export MAKEFLAGS=j"$(nproc)"
export CFLAGS=-O3
export CFLAGS="-D_FORTIFY_SOURCE=3 -D_GLIBCXX_ASSERTIONS -ftrivial-auto-var-init=zero -fPIE -pie -fstack-protector-strong -fstack-clash-protection -fcf-protection=full"
echo
echo "=== Building toxcore $ARCH ==="
@ -61,19 +59,39 @@ build() {
echo "SET(CROSSCOMPILING_EMULATOR /usr/bin/wine)" >>windows_toolchain.cmake
fi
if [ "$ARCH" = "i686" ]; then
TOXCORE_CFLAGS=""
else
# This makes the build work with -fstack-clash-protection, as otherwise it crashes with:
#/tmp/toxcore/toxcore/logger.c: In function 'logger_abort':
#/tmp/toxcore/toxcore/logger.c:124:1: internal compiler error: in seh_emit_stackalloc, at config/i386/winnt.cc:1055
# Should get patched in a future gcc version: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458
TOXCORE_CFLAGS="-fno-asynchronous-unwind-tables"
fi
# Patch CMakeLists.txt to make cracker.exe statically link against OpenMP. For some reason
# -DCMAKE_EXE_LINKER_FLAGS="-static" doesn't do it.
sed -i "s|OpenMP::OpenMP_C)|$(realpath -- /usr/lib/gcc/"$WINDOWS_TOOLCHAIN"/*-win32/libgomp.a) \${CMAKE_THREAD_LIBS_INIT})\ntarget_compile_options(cracker PRIVATE -fopenmp)|g" ../other/fun/CMakeLists.txt
# Silly way to bypass a shellharden check
read -ra EXTRA_CMAKE_FLAGS_ARRAY <<<"$EXTRA_CMAKE_FLAGS"
cmake -DCMAKE_TOOLCHAIN_FILE=windows_toolchain.cmake \
-DCMAKE_INSTALL_PREFIX="$STATIC_TOXCORE_PREFIX_DIR" \
-DENABLE_SHARED=OFF \
CFLAGS="$CFLAGS $TOXCORE_CFLAGS" \
cmake \
-DCMAKE_TOOLCHAIN_FILE=windows_toolchain.cmake \
-DCMAKE_INSTALL_PREFIX="$RESULT_PREFIX_DIR" \
-DCMAKE_BUILD_TYPE="Release" \
-DENABLE_SHARED=ON \
-DENABLE_STATIC=ON \
-DCMAKE_C_FLAGS="$CMAKE_C_FLAGS" \
-DCMAKE_CXX_FLAGS="$CMAKE_CXX_FLAGS" \
-DCMAKE_EXE_LINKER_FLAGS="$CMAKE_EXE_LINKER_FLAGS -fstack-protector" \
-DCMAKE_SHARED_LINKER_FLAGS="$CMAKE_SHARED_LINKER_FLAGS" \
-DSTRICT_ABI=ON \
-DEXPERIMENTAL_API=ON \
-DBUILD_FUN_UTILS=ON \
-DCMAKE_EXE_LINKER_FLAGS="-static" \
-DCMAKE_SHARED_LINKER_FLAGS="-static" \
"${EXTRA_CMAKE_FLAGS_ARRAY[@]}" \
-S ..
cmake --build . --target install -- -j"$(nproc)"
cmake --build . --target install --parallel "$(nproc)"
# CMake doesn't install fun utils, so do it manually
cp -a other/fun/*.exe "$RESULT_PREFIX_DIR/bin/"
if [ "$ENABLE_TEST" = "true" ]; then
rm -rf /root/.wine
@ -87,10 +105,12 @@ build() {
winecfg
export CTEST_OUTPUT_ON_FAILURE=1
# add libgcc_s_sjlj-1.dll libwinpthread-1.dll into PATH env var of wine
# we don't have to do this since autotests are statically compiled now,
# but just in case add MinGW-w64 dll locations to the PATH anyway
export WINEPATH="$(
cd /usr/lib/gcc/"$WINDOWS_TOOLCHAIN"/*posix/
cd /usr/lib/gcc/"$WINDOWS_TOOLCHAIN"/*win32/
winepath -w "$PWD"
cd -
)"\;"$(winepath -w /usr/"$WINDOWS_TOOLCHAIN"/lib/)"
if [ "$ALLOW_TEST_FAILURE" = "true" ]; then
@ -102,47 +122,48 @@ build() {
fi
fi
# move static dependencies
cp -a "$STATIC_TOXCORE_PREFIX_DIR"/* "$RESULT_PREFIX_DIR"
cp -a "$DEP_PREFIX_DIR"/* "$RESULT_PREFIX_DIR"
# make libtox.dll
cd "$SHARED_TOXCORE_PREFIX_DIR"
for archive in "$STATIC_TOXCORE_PREFIX_DIR"/lib/libtox*.a; do
"$WINDOWS_TOOLCHAIN"-ar xv "$archive"
# generate def, lib and exp as they supposedly help with linking against the dlls,
# especially the lib is supposed to be of great help when linking on msvc.
# cd in order to keep the object names inside .lib and .dll.a short
cd "$RESULT_PREFIX_DIR"/bin/
for TOX_DLL in *.dll; do
gendef - "$TOX_DLL" >"${TOX_DLL%.*}.def"
# we overwrite the CMake-generated .dll.a for the better
# compatibility with the .lib being generated here
"$WINDOWS_TOOLCHAIN"-dlltool \
--input-def "${TOX_DLL%.*}.def" \
--output-lib "${TOX_DLL%.*}.lib" \
--output-exp "${TOX_DLL%.*}.exp" \
--output-delaylib "../lib/${TOX_DLL%.*}.dll.a" \
--dllname "$TOX_DLL"
done
cd -
# copy over the deps
if [ "$CROSS_COMPILE" = "true" ]; then
LIBWINPTHREAD="/usr/$WINDOWS_TOOLCHAIN/lib/libwinpthread.a"
cd /usr/lib/gcc/"$WINDOWS_TOOLCHAIN"/*win32/
LIBSSP="$PWD/libssp.a"
cd -
else
LIBWINPTHREAD="/usr/$WINDOWS_TOOLCHAIN/sys-root/mingw/lib/libwinpthread.a"
LIBSSP="/usr/$WINDOWS_TOOLCHAIN/sys-root/mingw/lib/libssp.a"
fi
cp -a "$LIBWINPTHREAD" "$LIBSSP" "$RESULT_PREFIX_DIR/lib/"
for STATIC_LIB in "$DEP_PREFIX_DIR"/lib/*.a; do
[[ "$STATIC_LIB" == *.dll.a ]] && continue
cp -a "$STATIC_LIB" "$RESULT_PREFIX_DIR/lib/"
done
cp "$DEP_PREFIX_DIR"/lib/pkgconfig/* "$RESULT_PREFIX_DIR/lib/pkgconfig/"
"$WINDOWS_TOOLCHAIN"-gcc -Wl,--export-all-symbols \
-Wl,--out-implib=libtox.dll.a \
-shared \
-o libtox.dll \
*.obj \
"$STATIC_TOXCORE_PREFIX_DIR"/lib/*.a \
"$DEP_PREFIX_DIR"/lib/*.a \
"$LIBWINPTHREAD" \
-liphlpapi \
-lws2_32 \
-static-libgcc \
-lssp
cp libtox.dll.a "$RESULT_PREFIX_DIR"/lib
mkdir -p "$RESULT_PREFIX_DIR"/bin
cp libtox.dll "$RESULT_PREFIX_DIR"/bin
# strip everything
set +e
"$WINDOWS_TOOLCHAIN"-strip --strip-unneeded "$RESULT_PREFIX_DIR"/bin/*.* "$RESULT_PREFIX_DIR"/lib/*.*
set -e
rm -rf /tmp/*
# remove everything from include directory except tox headers
mv "$RESULT_PREFIX_DIR"/include/tox "$RESULT_PREFIX_DIR"/tox
rm -rf "$RESULT_PREFIX_DIR"/include/*
mv "$RESULT_PREFIX_DIR"/tox "$RESULT_PREFIX_DIR"/include/tox
sed -i "s|^prefix=.*|prefix=$RESULT_PREFIX_DIR|g" "$RESULT_PREFIX_DIR"/lib/pkgconfig/*.pc
sed -i "s|^libdir=.*|libdir=$RESULT_PREFIX_DIR/lib|g" "$RESULT_PREFIX_DIR"/lib/*.la
}
#=== Test Supported vs. Enabled ===

View File

@ -0,0 +1,16 @@
#!/usr/bin/env bash
check_sha256() {
[ "$ENABLE_HASH_VERIFICATION" = "true" ] && _check_sha256 "$@"
}
_check_sha256() {
if ! (echo "$1 $2" | sha256sum -c --status -); then
echo "Error: sha256 of $2 doesn't match the known one."
echo "Expected: $1 $2"
echo "Got: $(sha256sum "$2")"
return 1
fi
echo "sha256 matches the expected one: $1"
return 0
}

View File

@ -1,10 +1,11 @@
#!/usr/bin/env sh
#!/usr/bin/env bash
set -e -x
#=== Install Packages ===
apt-get update
apt-get upgrade -y
# Arch-independent packages required for building toxcore's dependencies and toxcore itself
apt-get install -y \
@ -13,9 +14,10 @@ apt-get install -y \
ca-certificates \
cmake \
curl \
libtool \
libc-dev \
libtool \
make \
mingw-w64-tools \
pkg-config \
tree \
yasm
@ -38,12 +40,27 @@ if [ "$SUPPORT_TEST" = "true" ]; then
apt-get install -y \
texinfo
CURL_OPTIONS=(-L --connect-timeout 10)
# While we would prefer to use Debian's Wine packages, use WineHQ's packages
# instead as Debian Bookworm's Wine crashes when creating a 64-bit prefix.
# see https://github.com/TokTok/c-toxcore/pull/2713#issuecomment-1967319113
# for the crash details
curl "${CURL_OPTIONS[@]}" -o /etc/apt/keyrings/winehq-archive.key \
https://dl.winehq.org/wine-builds/winehq.key
curl "${CURL_OPTIONS[@]}" -O --output-dir /etc/apt/sources.list.d/ \
https://dl.winehq.org/wine-builds/debian/dists/bookworm/winehq-bookworm.sources
. ./check_sha256.sh
check_sha256 "78b185fabdb323971d13bd329fefc8038e08559aa51c4996de18db0639a51df6" \
"/etc/apt/keyrings/winehq-archive.key"
check_sha256 "8dd8ef66c749d56e798646674c1c185a99b3ed6727ca0fbb5e493951e66c0f9e" \
"/etc/apt/sources.list.d/winehq-bookworm.sources"
dpkg --add-architecture i386
apt-get update
apt-get install -y \
wine \
wine32 \
wine64
winehq-stable
fi
# Clean up to reduce image size

View File

@ -6,36 +6,46 @@ function(target_link_toxcore target)
endif()
endfunction()
function(target_link_sodium target)
if(TARGET unofficial-sodium::sodium)
target_link_libraries(${target} PRIVATE unofficial-sodium::sodium)
else()
target_link_libraries(${target} PRIVATE ${LIBSODIUM_LIBRARIES})
target_link_directories(${target} PRIVATE ${LIBSODIUM_LIBRARY_DIRS})
target_include_directories(${target} SYSTEM PRIVATE ${LIBSODIUM_INCLUDE_DIRS})
target_compile_options(${target} PRIVATE ${LIBSODIUM_CFLAGS_OTHER})
endif()
endfunction()
add_executable(save-generator save-generator.c)
target_link_libraries(save-generator PRIVATE misc_tools)
target_link_toxcore(save-generator)
add_executable(strkey strkey.c)
target_link_libraries(strkey PRIVATE ${LIBSODIUM_LIBRARIES})
target_link_toxcore(strkey)
target_link_sodium(strkey)
add_executable(create_bootstrap_keys create_bootstrap_keys.c)
target_link_libraries(create_bootstrap_keys PRIVATE ${LIBSODIUM_LIBRARIES})
target_link_toxcore(create_bootstrap_keys)
target_link_sodium(create_bootstrap_keys)
add_executable(create_minimal_savedata create_minimal_savedata.c)
target_link_libraries(create_minimal_savedata PRIVATE ${LIBSODIUM_LIBRARIES})
target_link_sodium(create_minimal_savedata)
add_executable(create_savedata create_savedata.c)
target_link_libraries(create_savedata PRIVATE ${LIBSODIUM_LIBRARIES})
target_link_sodium(create_savedata)
target_link_toxcore(create_savedata)
add_executable(sign sign.c)
target_link_libraries(sign PRIVATE ${LIBSODIUM_LIBRARIES} misc_tools)
target_link_sodium(sign)
add_executable(cracker_simple cracker_simple.c)
target_link_libraries(cracker_simple ${LIBSODIUM_LIBRARIES} misc_tools)
target_link_sodium(cracker_simple)
# MSVC doesn't support OpenMP
if(NOT MSVC)
find_package(OpenMP)
if(OpenMP_C_FOUND)
add_executable(cracker cracker.c)
target_link_libraries(cracker PRIVATE OpenMP::OpenMP_C ${LIBSODIUM_LIBRARIES})
endif()
find_package(OpenMP)
if(OpenMP_C_FOUND)
add_executable(cracker cracker.c)
target_link_libraries(cracker PRIVATE OpenMP::OpenMP_C)
target_link_sodium(cracker)
else()
add_executable(cracker cracker.c)
target_link_sodium(cracker)
endif()

View File

@ -33,6 +33,7 @@
#include <omp.h>
#define NUM_THREADS() ((unsigned) omp_get_max_threads())
#else
#pragma message("Being built without OpenMP support -- the program will utilize a single thread only.")
#define NUM_THREADS() (1U)
#endif
@ -112,8 +113,9 @@ static size_t match_hex_prefix(const uint8_t *key, const uint8_t *prefix, size_t
static void cracker_core(uint64_t range_start, uint64_t range_end, uint64_t range_offs, uint64_t priv_key_shadow[4],
uint32_t *longest_match, uint8_t hex_prefix[MAX_CRACK_BYTES], size_t prefix_chars_len)
{
#if defined(_OPENMP)
#pragma omp parallel for firstprivate(priv_key_shadow) shared(longest_match, range_start, range_end, range_offs, hex_prefix, prefix_chars_len) schedule(static) default(none)
#endif
for (uint64_t batch = range_start; batch < range_end; ++batch) {
uint8_t *priv_key = (uint8_t *) priv_key_shadow;
/*
@ -134,14 +136,19 @@ static void cracker_core(uint64_t range_start, uint64_t range_end, uint64_t rang
// Global compare and update
uint32_t l_longest_match;
#if defined(_OPENMP)
#pragma omp atomic read
#endif
l_longest_match = *longest_match;
if (matching > l_longest_match) {
#if defined(_OPENMP)
#pragma omp atomic write
#endif
*longest_match = matching;
#if defined(_OPENMP)
#pragma omp critical
#endif
{
printf("%u chars matching: \n", matching);
printf("Public key: ");

View File

@ -13,11 +13,8 @@
#include <stdio.h>
#include <string.h>
/* Sodium includes*/
#include <sodium/crypto_scalarmult_curve25519.h>
#include <sodium/randombytes.h>
#include <sodium.h>
#include "../../testing/misc_tools.h"
#include "../../toxcore/ccompat.h"
// Secret key and public key length
@ -40,7 +37,13 @@ int main(int argc, char *argv[])
long long unsigned int num_tries = 0;
size_t len = strlen(argv[1]) / 2;
unsigned char *key = hex_string_to_bin(argv[1]);
unsigned char *key = (unsigned char *)malloc(len);
const char *hex_end = nullptr;
if (sodium_hex2bin(key, len, argv[1], strlen(argv[1]), nullptr, nullptr, &hex_end) != 0
|| hex_end != argv[1] + strlen(argv[1])) {
printf("Invalid key provided\n");
return 1;
}
uint8_t pub_key[KEY_LEN], priv_key[KEY_LEN], c_key[KEY_LEN];
if (len > KEY_LEN) {

View File

@ -18,7 +18,6 @@
#include <sodium.h>
#include <string.h>
#include "../../testing/misc_tools.h" // hex_string_to_bin
#include "../../toxcore/ccompat.h"
static int load_file(const char *filename, unsigned char **result)
@ -70,7 +69,12 @@ int main(int argc, char *argv[])
}
if (argc == 5 && argv[1][0] == 's') {
unsigned char *secret_key = hex_string_to_bin(argv[2]);
const char *hex_end = nullptr;
if (sodium_hex2bin(sk, sizeof(sk), argv[2], strlen(argv[2]), nullptr, nullptr, &hex_end) != 0
|| hex_end != argv[2] + strlen(argv[2])) {
printf("Invalid secret key provided.\n");
goto fail;
}
unsigned char *data = nullptr;
int size = load_file(argv[3], &data);
@ -80,9 +84,8 @@ int main(int argc, char *argv[])
unsigned long long smlen;
unsigned char *sm = (unsigned char *)malloc(size + crypto_sign_ed25519_BYTES * 2);
crypto_sign_ed25519(sm, &smlen, data, size, secret_key);
crypto_sign_ed25519(sm, &smlen, data, size, sk);
free(data);
free(secret_key);
if (smlen - size != crypto_sign_ed25519_BYTES) {
free(sm);
@ -110,8 +113,13 @@ int main(int argc, char *argv[])
}
if (argc == 4 && argv[1][0] == 'c') {
unsigned char *public_key = hex_string_to_bin(argv[2]);
unsigned char *data;
const char *hex_end = nullptr;
if (sodium_hex2bin(pk, sizeof(pk), argv[2], strlen(argv[2]), nullptr, nullptr, &hex_end) != 0
|| hex_end != argv[2] + strlen(argv[2])) {
printf("Invalid public key provided.\n");
goto fail;
}
unsigned char *data = nullptr;
int size = load_file(argv[3], &data);
if (size < 0) {
@ -127,7 +135,7 @@ int main(int argc, char *argv[])
unsigned char *m = (unsigned char *)malloc(size);
unsigned long long mlen;
if (crypto_sign_ed25519_open(m, &mlen, signe, size, public_key) == -1) {
if (crypto_sign_ed25519_open(m, &mlen, signe, size, pk) == -1) {
printf("Failed checking sig.\n");
free(m);
free(signe);

View File

@ -1,8 +1,12 @@
#!/bin/sh
export VERSION_SODIUM="1.0.18"
export VERSION_OPUS="1.3.1"
export VERSION_VPX="1.9.0"
# When editing, make sure to update /other/docker/windows/Dockerfile and
# INSTALL.md to match.
export VERSION_OPUS="1.4"
export VERSION_SODIUM="1.0.19"
export VERSION_VPX="1.14.0"
export ENABLE_HASH_VERIFICATION=true
export SUPPORT_TEST=false
export SUPPORT_ARCH_i686=true
@ -15,6 +19,6 @@ export ENABLE_TEST=false
export ALLOW_TEST_FAILURE=false
export ENABLE_ARCH_i686=true
export ENABLE_ARCH_x86_64=true
export EXTRA_CMAKE_FLAGS="-DTEST_TIMEOUT_SECONDS=90"
export EXTRA_CMAKE_FLAGS="-DTEST_TIMEOUT_SECONDS=90 -DUSE_IPV6=OFF"
sh ./other/docker/windows/build_toxcore.sh

View File

@ -1,7 +1,9 @@
#ifndef C_TOXCORE_TESTING_MISC_TOOLS_H
#define C_TOXCORE_TESTING_MISC_TOOLS_H
#include "../toxcore/tox.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
@ -13,7 +15,6 @@ extern "C" {
void c_sleep(uint32_t x);
uint8_t *hex_string_to_bin(const char *hex_string);
char *id_toa(const uint8_t *id);
void to_hex(char *out, uint8_t *in, int size);
int tox_strncasecmp(const char *s1, const char *s2, size_t n);
int cmdline_parsefor_ipv46(int argc, char **argv, bool *ipv6enabled);