Commit Graph

1145 Commits

Author SHA1 Message Date
Christian Blichmann
6de30ea27f CI: Cache dependencies
This will speed up our builds a bit and prevent unnecessary network traffic.

Setup according to the documentation for the `actions/cache@v2` action:
https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows

Currently caching the `${{github.workspace}}/_deps` directory, as used by CMake.

Cache keys look like this: `ubuntu-20.04-clang11` (`${{matrix.os}-${{matrix.compiler}}${matrix.compiler-version}}`)

PiperOrigin-RevId: 431895214
Change-Id: I4ecac7c00eec8516f85f45aa2220303b811b2389
2022-03-02 03:07:21 -08:00
Copybara-Service
2d324bd50d Merge pull request #121 from oshogbo:doc2
PiperOrigin-RevId: 431618531
Change-Id: Ieabcfa982770831acd565cd17fcf121570850ab9
2022-03-01 00:15:58 -08:00
Sandboxed API Team
9a7ba28ea7 Allow sanitizer to print reports
PiperOrigin-RevId: 430271415
Change-Id: Ieb23663aa6ff5997ce0a6b1e81dcb2385ac4b509
2022-02-22 12:33:55 -08:00
Mariusz Zaborski
e49318571b Sandbox zip 2022-02-22 10:51:53 -05:00
Copybara-Service
4024694eb6 Merge pull request #112 from oshogbo:zstd_opt
PiperOrigin-RevId: 430179725
Change-Id: Ic3c93a51a199eaf087cea2e58c819eb07bf52a1a
2022-02-22 04:21:24 -08:00
Copybara-Service
a805034070 Merge pull request #125 from oshogbo:cmake_c_blosc
PiperOrigin-RevId: 430174230
Change-Id: I6e4f53f3cae4e2e3487419903f7f8c6f98ac828e
2022-02-22 03:44:16 -08:00
Copybara-Service
176a19989b Merge pull request #114 from oshogbo:cmake_quote
PiperOrigin-RevId: 430174200
Change-Id: I95831e97b75c0f3df552e13ae00665b5c9c91333
2022-02-22 03:43:27 -08:00
Christian Blichmann
99f1ce93ba
Merge branch 'main' into cmake_quote 2022-02-22 11:57:33 +01:00
Wiktor Garbacz
a2daa0a275 Fix BlockSyscallsWithErrno
PiperOrigin-RevId: 429982218
Change-Id: I42b187e678542b295542ca44882945c7695178e1
2022-02-21 00:46:50 -08:00
Sandboxed API Team
e9c041f0c2 [Cleanup] Fix apache license url
PiperOrigin-RevId: 429974822
Change-Id: Id07aa9baf374458b9ff789fc93eff2b51d77917c
2022-02-20 23:50:29 -08:00
Mariusz Zaborski
3680d50565 contrib: Sort the order of projects 2022-02-18 08:00:21 -05:00
Mariusz Zaborski
b9ec42d220 c-blosc: to default build 2022-02-18 07:57:03 -05:00
Copybara-Service
2fb08b99da Merge pull request #113 from oshogbo:c-blosc
PiperOrigin-RevId: 429535319
Change-Id: Ide9e81a76d28e1f2e4eefbd499ef8bcd22e1a1b0
2022-02-18 04:47:09 -08:00
Christian Blichmann
10c04ed42f CMake: Reorder PIE checks, fix bracket limit for Clang
The default limit for recent versions of Clang is 256 which is less than the
number of syscalls in our syscall tables (around 340). This change increases
this limit to an arbitrary 768.

PiperOrigin-RevId: 429258387
Change-Id: I4927eee78edc8aaa2a758b29811d02326e5aa953
2022-02-17 02:31:24 -08:00
Christian Blichmann
befdb09597 Link more complex test cases dynamically
Linking glibc in fully static mode is mostly unsupported. While such binaries
can easily be produced, conflicting symbols will often make them crash at
runtime. This happens because glibc will always (try to) load some dynamically
linked libraries, even when statically linked. This includes things like the
resolver, unicode/locale handling and others.

Internally at Google, this is not a concern due to the way glibc is being built
there. But in order to make all of our tests run in the open-source version of
this code, we need to change strategy a bit.

As a rule of thumb, glibc can safely be linked statically if a program is
resonably simple and does not use any networking of locale dependent
facilities. Calling syscalls directly instead of the corresponding libc
wrappers works as well, of course.

This change adjusts linker flags and sandbox policies to be more compatible
with regular Linux distributions.

Tested:
- `ctest -R '[A-Z].*'` (all SAPI/Sandbox2 tests)
PiperOrigin-RevId: 429025901
Change-Id: I46b677d9eb61080a8fe868002a34a77de287bf2d
2022-02-16 05:59:13 -08:00
Wiktor Garbacz
d2dfcf0800 Per-C++ specs main shouldn't be declared with C language linkage
PiperOrigin-RevId: 429025497
Change-Id: I7f732f4e42b64463847e192c6ca5cff820ab19ba
2022-02-16 05:56:25 -08:00
Copybara-Service
1dedbb9650 Merge pull request #110 from oshogbo:zopfli_fd
PiperOrigin-RevId: 429016804
Change-Id: Ib1d9b616325c2b6443149bed25859247f2fb68e7
2022-02-16 04:56:18 -08:00
Mariusz Zaborski
dc43384516 ZStandard: Optimize passing the memory
In the case of decompression, we are using the same chunk of memory
twice. The first is to obtain the size of decompressed data, and
the second is to decompress it. In the current code, we are copying
the chunk of memory twice. The memory doesn't change between
the calls, so there is no point in copying it over and over.

Let's synchronize memory only during first call.
2022-02-16 07:29:37 -05:00
Christian Blichmann
cef861a0f2 CMake: Properly inherit sapi_base propterties
- `SKIP_BUILD_RPATH` needs to be set per directory (not allow-listed for
  `INTERFACE` properties)
- Use correct name for position independent code
- `-pie`/`-fPIE` will only propagate fully on 3.14+

PiperOrigin-RevId: 428986266
Change-Id: Idf9d7fc184fbeec8ec1b77505246e262d9b8d880
2022-02-16 01:48:13 -08:00
Christian Blichmann
c93dae9519
Merge branch 'main' into c-blosc 2022-02-16 10:02:08 +01:00
Christian Blichmann
aefdb94575 Update zlib examples
- Link `zipe.c` statically (safe)
- Update policy to allow any use of `stat()`

PiperOrigin-RevId: 428971638
Change-Id: Ib0f5f496ea2389582986b41a8830592e6c1d4390
2022-02-16 00:08:28 -08:00
Copybara-Service
ddef30148c Merge pull request #117 from DemiMarie:add-idn2-and-turbojpeg
PiperOrigin-RevId: 428763952
Change-Id: I0b958540213052d87a59d5605014fb82c8ac137b
2022-02-15 06:22:29 -08:00
Christian Blichmann
e8cadf8f7d Allow mprotect(_, _, PROT_READ) for all static binaries
Newer toolchains/libcs will use this syscall on x86-64 as well.

PiperOrigin-RevId: 428705078
Change-Id: I705efe37db9ebdd922036b39e4fb3c22dc749a1a
2022-02-15 00:14:25 -08:00
Demi Marie Obenour
b2f4e0068e Add libidn2 and TurboJPEG sandboxes to the build
This exposed a compilation error and an incorrect TEST_FILES_DIR.  Fix
both.
2022-02-14 12:51:22 -05:00
Mariusz Zaborski
d39d63d6ec Sandbox c-blosc 2022-02-14 10:43:06 -05:00
Mariusz Zaborski
ad486b9e86 CMake: quote sources 2022-02-14 10:41:06 -05:00
Copybara-Service
38eea151a6 Merge pull request #116 from DemiMarie:fix-syntax-error
PiperOrigin-RevId: 428501046
Change-Id: I5114b7f1ab79cc90b0a3ee9a06495a2ee9752955
2022-02-14 07:20:40 -08:00
Christian Blichmann
789c436a3e CI: Run tests in VM based builders
This adds a first basic test to be run using GitHub Actions on push and pull
request for the CMake build (internally we run everything on Bazel/Blaze).

The Ubuntu runners are implemented as full VMs, so we can run tests directly.

In order to run Sandboxed API/Sandbox2 tests inside a container, it must be
started as privileged, unconfined and retain its capabilities.
Since GitHub does not support modifying the Docker invocation for container
based workflows, we need to manually run the `docker` command.

Until #118 is fixed, this change makes GitHub ignore the test failure on
Fedora.

PiperOrigin-RevId: 428485354
Change-Id: I6b55c5441c4c27b018d19498d2296c7d3da65846
2022-02-14 05:57:19 -08:00
Demi Marie Obenour
544d438e71 Fix a syntax error in the zstd example
It breaks the SAPI_ENABLE_CONTRIB_TESTS build.
2022-02-12 21:43:49 -05:00
Christian Blichmann
d1ed8ac66e Avoid compiler crash with Clang 6.0
Instead of C++17 structured bindings, use a plain `const auto&` and annotate
arguments with comments instead.

We still support Clang 6.0, as that is the compiler that ships with Ubuntu
18.04 LTS by default.

PiperOrigin-RevId: 428016214
Change-Id: I3a43b2d47c6825ac4425d22018750282cfe23c1b
2022-02-11 09:09:01 -08:00
Christian Blichmann
36d0f928c6 Apply page offset during stack unwinding/symbolization
This fixes a couple of tests in the open source version of the code.
Internally, since we are using a different ELF loader, the page offset
will always be zero. Hence we never notices this was broken.

PiperOrigin-RevId: 427996428
Change-Id: I44c5b5610b074cf69b9f0c5eeb051be50923e351
2022-02-11 07:19:34 -08:00
Copybara-Service
1b816e9dce Merge pull request #105 from DemiMarie:turbojpeg
PiperOrigin-RevId: 427991276
Change-Id: I66fa4fcb16b18f38a3ec0dfce46863269606d5e8
2022-02-11 06:44:49 -08:00
Christian Blichmann
7004d59150 Remove pffft submodule entry
PiperOrigin-RevId: 427686350
Change-Id: If75bb7e094f8a1c14b3e0c8f26d0f21b1e45a6dc
2022-02-10 01:41:12 -08:00
Demi Marie Obenour
e613bdfaeb Sandbox TurboJPEG
Took a few tries to get everything implemented.
2022-02-09 11:37:43 -05:00
Copybara-Service
585e55a1e0 Merge pull request #108 from oshogbo:zstd_fd_pr
PiperOrigin-RevId: 427469394
Change-Id: I789b45066800af5a29cbf33313cd7a4fabd56be3
2022-02-09 07:47:35 -08:00
Sandboxed API Team
59b942b256 Add a little more logging to failure cases.
PiperOrigin-RevId: 427459159
Change-Id: I34b6027cccfc4b3903ef4deeb9c133598b6667d4
2022-02-09 06:54:07 -08:00
Christian Blichmann
7e5a398164 Migrate the pffft sandbox to contrib/
PiperOrigin-RevId: 427443359
Change-Id: I852a818ae302a86abe32a2820f349f67861e342e
2022-02-09 05:20:29 -08:00
Christian Blichmann
fdc38d58c6 CMake: Quiet config of libhunspell, quote sources
PiperOrigin-RevId: 427418932
Change-Id: Ie8992176ed29437a7eb7550e7ec6628103a447ff
2022-02-09 02:32:20 -08:00
Christian Blichmann
0576efe994 Update jsonnet README
PiperOrigin-RevId: 427415134
Change-Id: Ib6a729331bde4e29d89a24298dc9d7ce47c356f3
2022-02-09 02:10:49 -08:00
Christian Blichmann
4ad8484e63 Tag additional test as not compatible with QEMU user-mode emulation
PiperOrigin-RevId: 427409251
Change-Id: I5d853908353923b5b31c8bbb6152bc4f94219b45
2022-02-09 01:41:38 -08:00
Mariusz Zaborski
1beba0b3ad ZStandard: use long includes 2022-02-08 13:11:16 -05:00
Mariusz Zaborski
e3da4da69b ZStandard: Simplify creation of paths 2022-02-08 13:11:16 -05:00
Mariusz Zaborski
44088bb7d5 ZStandard: Fix the order of headers 2022-02-08 13:11:16 -05:00
Mariusz Zaborski
5c154af744 ZStandard: introduce a wrapper
The goal is to use a file descriptor as an input for ZStandard
library. Thanks to that we shouldn't send a chunk of memory over
expensive protocol.
2022-02-08 13:11:13 -05:00
Mariusz Zaborski
b3aca9ef48 zopfli: introduce a wrapper
The goal is to use a file descriptor as an input.
Thanks to that we shouldn't send a chunk of memory over
expensive protocol.
2022-02-08 13:07:51 -05:00
Christian Blichmann
dc03c38df1 Enable stack traces on AArch64
Note that `//sandboxed_api/sandbox2:stack_trace_test` may still fail for
unrelated reasons, as we are linking libc statically, which is brittle. A
follow-up change will fix this.

PiperOrigin-RevId: 427175045
Change-Id: Ifb5ec2ac3d60f4bcc9708f26c834c83b75e769d7
2022-02-08 06:23:52 -08:00
Copybara-Service
d76a0e959e Merge pull request #100 from oshogbo:zopfli
PiperOrigin-RevId: 426911996
Change-Id: Ib257a94b6f5cbe932766d2c25c21ad082e0369a4
2022-02-07 07:30:51 -08:00
Christian Blichmann
5c98e30ef2 CMake: Force inclusion of exported functions in add_sapi_library()
Newer versions of the GNU gold linkers sometimes omit the symbols for
sandboxed functions, leading to errors like `function not found` when
Sandboxed API tries to call them at runtime.

The fix is simple: Do what Bazel does and wrap the static library in
a `-Wl,--whole-archive`/`-Wl,--no-whole-archive` pair.

PiperOrigin-RevId: 426910732
Change-Id: Id4183f2964a2dea4fc1dfd8c409bc0b1af9db170
2022-02-07 07:23:34 -08:00
Mariusz Zaborski
0b877243be Sandbox zopfli 2022-02-06 16:33:07 -05:00
Copybara-Service
0f78eb4c59 Merge pull request #103 from oshogbo:hunspell
PiperOrigin-RevId: 426372638
Change-Id: Ia15b7bb6772cdae0f3f44097d3ff66ef2527f927
2022-02-04 04:45:04 -08:00