Commit Graph

950 Commits

Author SHA1 Message Date
Oliver Kunz
077203fcf2 Change to proto2::MessageLite and resolve reflextion for mobile builds
PiperOrigin-RevId: 432164927
Change-Id: I0821cf443393b0bb16a68fc5750a9633a3f27725
2022-03-03 04:48:30 -08:00
Sandboxed API Team
e1a9513783 Move few policies from tsan to All section.
munmap is widely used by sanitizer, but it
probably works for Asan/Msan because it's enabled
by unrelated Allow* call.

Move mprotect to shared part as well. It will be
needed for compress_stack_depot.

PiperOrigin-RevId: 431989551
Change-Id: I7695a2de81d8d0b2112d3308778b2e9a9c7cb596
2022-03-02 11:38:35 -08:00
Sandboxed API Team
546365655d Introduce commandline flag to pass forkserver_bin path for Android builds.
PiperOrigin-RevId: 431942480
Change-Id: I5382b4fc8e8a66bb823dda597e1b812421364212
2022-03-02 08:12:21 -08:00
Sandboxed API Team
3f042fa54f Fix monitor for Android-ARM64
PiperOrigin-RevId: 431926820
Change-Id: Ie5adc1ec6accc7e68782c26b65fac0c32cded498
2022-03-02 06:42:42 -08:00
Christian Blichmann
692f0260b3 clang_generator: Emit types outside of namespace, skip Abseil enums
PiperOrigin-RevId: 431913470
Change-Id: Ia44f6642a37501ba1630321ba1430d1bf10cf377
2022-03-02 05:17:32 -08:00
Christian Blichmann
60fcc5b63e Limit the number of includes fed into the header generator
Use [`direct_headers`](https://bazel.build/rules/lib/CompilationContext#direct_headers)
from the Bazel/Blaze compilation context instead of _all_ transitive headers.

For the clang based generator, this means we don't try to parse
`textual_headers`, which will fail (they are by definition not
stand-alone, after all).

PiperOrigin-RevId: 431899423
Change-Id: I7a9dfa0dd93eba14b506b0e7ca6db3ed59b55dd6
2022-03-02 03:41:41 -08:00
Christian Blichmann
6de30ea27f CI: Cache dependencies
This will speed up our builds a bit and prevent unnecessary network traffic.

Setup according to the documentation for the `actions/cache@v2` action:
https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows

Currently caching the `${{github.workspace}}/_deps` directory, as used by CMake.

Cache keys look like this: `ubuntu-20.04-clang11` (`${{matrix.os}-${{matrix.compiler}}${matrix.compiler-version}}`)

PiperOrigin-RevId: 431895214
Change-Id: I4ecac7c00eec8516f85f45aa2220303b811b2389
2022-03-02 03:07:21 -08:00
Copybara-Service
2d324bd50d Merge pull request #121 from oshogbo:doc2
PiperOrigin-RevId: 431618531
Change-Id: Ieabcfa982770831acd565cd17fcf121570850ab9
2022-03-01 00:15:58 -08:00
Sandboxed API Team
9a7ba28ea7 Allow sanitizer to print reports
PiperOrigin-RevId: 430271415
Change-Id: Ieb23663aa6ff5997ce0a6b1e81dcb2385ac4b509
2022-02-22 12:33:55 -08:00
Copybara-Service
4024694eb6 Merge pull request #112 from oshogbo:zstd_opt
PiperOrigin-RevId: 430179725
Change-Id: Ic3c93a51a199eaf087cea2e58c819eb07bf52a1a
2022-02-22 04:21:24 -08:00
Copybara-Service
a805034070 Merge pull request #125 from oshogbo:cmake_c_blosc
PiperOrigin-RevId: 430174230
Change-Id: I6e4f53f3cae4e2e3487419903f7f8c6f98ac828e
2022-02-22 03:44:16 -08:00
Copybara-Service
176a19989b Merge pull request #114 from oshogbo:cmake_quote
PiperOrigin-RevId: 430174200
Change-Id: I95831e97b75c0f3df552e13ae00665b5c9c91333
2022-02-22 03:43:27 -08:00
Christian Blichmann
99f1ce93ba
Merge branch 'main' into cmake_quote 2022-02-22 11:57:33 +01:00
Wiktor Garbacz
a2daa0a275 Fix BlockSyscallsWithErrno
PiperOrigin-RevId: 429982218
Change-Id: I42b187e678542b295542ca44882945c7695178e1
2022-02-21 00:46:50 -08:00
Sandboxed API Team
e9c041f0c2 [Cleanup] Fix apache license url
PiperOrigin-RevId: 429974822
Change-Id: Id07aa9baf374458b9ff789fc93eff2b51d77917c
2022-02-20 23:50:29 -08:00
Mariusz Zaborski
3680d50565 contrib: Sort the order of projects 2022-02-18 08:00:21 -05:00
Mariusz Zaborski
b9ec42d220 c-blosc: to default build 2022-02-18 07:57:03 -05:00
Copybara-Service
2fb08b99da Merge pull request #113 from oshogbo:c-blosc
PiperOrigin-RevId: 429535319
Change-Id: Ide9e81a76d28e1f2e4eefbd499ef8bcd22e1a1b0
2022-02-18 04:47:09 -08:00
Christian Blichmann
10c04ed42f CMake: Reorder PIE checks, fix bracket limit for Clang
The default limit for recent versions of Clang is 256 which is less than the
number of syscalls in our syscall tables (around 340). This change increases
this limit to an arbitrary 768.

PiperOrigin-RevId: 429258387
Change-Id: I4927eee78edc8aaa2a758b29811d02326e5aa953
2022-02-17 02:31:24 -08:00
Christian Blichmann
befdb09597 Link more complex test cases dynamically
Linking glibc in fully static mode is mostly unsupported. While such binaries
can easily be produced, conflicting symbols will often make them crash at
runtime. This happens because glibc will always (try to) load some dynamically
linked libraries, even when statically linked. This includes things like the
resolver, unicode/locale handling and others.

Internally at Google, this is not a concern due to the way glibc is being built
there. But in order to make all of our tests run in the open-source version of
this code, we need to change strategy a bit.

As a rule of thumb, glibc can safely be linked statically if a program is
resonably simple and does not use any networking of locale dependent
facilities. Calling syscalls directly instead of the corresponding libc
wrappers works as well, of course.

This change adjusts linker flags and sandbox policies to be more compatible
with regular Linux distributions.

Tested:
- `ctest -R '[A-Z].*'` (all SAPI/Sandbox2 tests)
PiperOrigin-RevId: 429025901
Change-Id: I46b677d9eb61080a8fe868002a34a77de287bf2d
2022-02-16 05:59:13 -08:00
Wiktor Garbacz
d2dfcf0800 Per-C++ specs main shouldn't be declared with C language linkage
PiperOrigin-RevId: 429025497
Change-Id: I7f732f4e42b64463847e192c6ca5cff820ab19ba
2022-02-16 05:56:25 -08:00
Copybara-Service
1dedbb9650 Merge pull request #110 from oshogbo:zopfli_fd
PiperOrigin-RevId: 429016804
Change-Id: Ib1d9b616325c2b6443149bed25859247f2fb68e7
2022-02-16 04:56:18 -08:00
Mariusz Zaborski
dc43384516 ZStandard: Optimize passing the memory
In the case of decompression, we are using the same chunk of memory
twice. The first is to obtain the size of decompressed data, and
the second is to decompress it. In the current code, we are copying
the chunk of memory twice. The memory doesn't change between
the calls, so there is no point in copying it over and over.

Let's synchronize memory only during first call.
2022-02-16 07:29:37 -05:00
Christian Blichmann
cef861a0f2 CMake: Properly inherit sapi_base propterties
- `SKIP_BUILD_RPATH` needs to be set per directory (not allow-listed for
  `INTERFACE` properties)
- Use correct name for position independent code
- `-pie`/`-fPIE` will only propagate fully on 3.14+

PiperOrigin-RevId: 428986266
Change-Id: Idf9d7fc184fbeec8ec1b77505246e262d9b8d880
2022-02-16 01:48:13 -08:00
Christian Blichmann
c93dae9519
Merge branch 'main' into c-blosc 2022-02-16 10:02:08 +01:00
Christian Blichmann
aefdb94575 Update zlib examples
- Link `zipe.c` statically (safe)
- Update policy to allow any use of `stat()`

PiperOrigin-RevId: 428971638
Change-Id: Ib0f5f496ea2389582986b41a8830592e6c1d4390
2022-02-16 00:08:28 -08:00
Copybara-Service
ddef30148c Merge pull request #117 from DemiMarie:add-idn2-and-turbojpeg
PiperOrigin-RevId: 428763952
Change-Id: I0b958540213052d87a59d5605014fb82c8ac137b
2022-02-15 06:22:29 -08:00
Christian Blichmann
e8cadf8f7d Allow mprotect(_, _, PROT_READ) for all static binaries
Newer toolchains/libcs will use this syscall on x86-64 as well.

PiperOrigin-RevId: 428705078
Change-Id: I705efe37db9ebdd922036b39e4fb3c22dc749a1a
2022-02-15 00:14:25 -08:00
Demi Marie Obenour
b2f4e0068e Add libidn2 and TurboJPEG sandboxes to the build
This exposed a compilation error and an incorrect TEST_FILES_DIR.  Fix
both.
2022-02-14 12:51:22 -05:00
Mariusz Zaborski
d39d63d6ec Sandbox c-blosc 2022-02-14 10:43:06 -05:00
Mariusz Zaborski
ad486b9e86 CMake: quote sources 2022-02-14 10:41:06 -05:00
Copybara-Service
38eea151a6 Merge pull request #116 from DemiMarie:fix-syntax-error
PiperOrigin-RevId: 428501046
Change-Id: I5114b7f1ab79cc90b0a3ee9a06495a2ee9752955
2022-02-14 07:20:40 -08:00
Christian Blichmann
789c436a3e CI: Run tests in VM based builders
This adds a first basic test to be run using GitHub Actions on push and pull
request for the CMake build (internally we run everything on Bazel/Blaze).

The Ubuntu runners are implemented as full VMs, so we can run tests directly.

In order to run Sandboxed API/Sandbox2 tests inside a container, it must be
started as privileged, unconfined and retain its capabilities.
Since GitHub does not support modifying the Docker invocation for container
based workflows, we need to manually run the `docker` command.

Until #118 is fixed, this change makes GitHub ignore the test failure on
Fedora.

PiperOrigin-RevId: 428485354
Change-Id: I6b55c5441c4c27b018d19498d2296c7d3da65846
2022-02-14 05:57:19 -08:00
Demi Marie Obenour
544d438e71 Fix a syntax error in the zstd example
It breaks the SAPI_ENABLE_CONTRIB_TESTS build.
2022-02-12 21:43:49 -05:00
Christian Blichmann
d1ed8ac66e Avoid compiler crash with Clang 6.0
Instead of C++17 structured bindings, use a plain `const auto&` and annotate
arguments with comments instead.

We still support Clang 6.0, as that is the compiler that ships with Ubuntu
18.04 LTS by default.

PiperOrigin-RevId: 428016214
Change-Id: I3a43b2d47c6825ac4425d22018750282cfe23c1b
2022-02-11 09:09:01 -08:00
Christian Blichmann
36d0f928c6 Apply page offset during stack unwinding/symbolization
This fixes a couple of tests in the open source version of the code.
Internally, since we are using a different ELF loader, the page offset
will always be zero. Hence we never notices this was broken.

PiperOrigin-RevId: 427996428
Change-Id: I44c5b5610b074cf69b9f0c5eeb051be50923e351
2022-02-11 07:19:34 -08:00
Copybara-Service
1b816e9dce Merge pull request #105 from DemiMarie:turbojpeg
PiperOrigin-RevId: 427991276
Change-Id: I66fa4fcb16b18f38a3ec0dfce46863269606d5e8
2022-02-11 06:44:49 -08:00
Christian Blichmann
7004d59150 Remove pffft submodule entry
PiperOrigin-RevId: 427686350
Change-Id: If75bb7e094f8a1c14b3e0c8f26d0f21b1e45a6dc
2022-02-10 01:41:12 -08:00
Demi Marie Obenour
e613bdfaeb Sandbox TurboJPEG
Took a few tries to get everything implemented.
2022-02-09 11:37:43 -05:00
Copybara-Service
585e55a1e0 Merge pull request #108 from oshogbo:zstd_fd_pr
PiperOrigin-RevId: 427469394
Change-Id: I789b45066800af5a29cbf33313cd7a4fabd56be3
2022-02-09 07:47:35 -08:00
Sandboxed API Team
59b942b256 Add a little more logging to failure cases.
PiperOrigin-RevId: 427459159
Change-Id: I34b6027cccfc4b3903ef4deeb9c133598b6667d4
2022-02-09 06:54:07 -08:00
Christian Blichmann
7e5a398164 Migrate the pffft sandbox to contrib/
PiperOrigin-RevId: 427443359
Change-Id: I852a818ae302a86abe32a2820f349f67861e342e
2022-02-09 05:20:29 -08:00
Christian Blichmann
fdc38d58c6 CMake: Quiet config of libhunspell, quote sources
PiperOrigin-RevId: 427418932
Change-Id: Ie8992176ed29437a7eb7550e7ec6628103a447ff
2022-02-09 02:32:20 -08:00
Christian Blichmann
0576efe994 Update jsonnet README
PiperOrigin-RevId: 427415134
Change-Id: Ib6a729331bde4e29d89a24298dc9d7ce47c356f3
2022-02-09 02:10:49 -08:00
Christian Blichmann
4ad8484e63 Tag additional test as not compatible with QEMU user-mode emulation
PiperOrigin-RevId: 427409251
Change-Id: I5d853908353923b5b31c8bbb6152bc4f94219b45
2022-02-09 01:41:38 -08:00
Mariusz Zaborski
1beba0b3ad ZStandard: use long includes 2022-02-08 13:11:16 -05:00
Mariusz Zaborski
e3da4da69b ZStandard: Simplify creation of paths 2022-02-08 13:11:16 -05:00
Mariusz Zaborski
44088bb7d5 ZStandard: Fix the order of headers 2022-02-08 13:11:16 -05:00
Mariusz Zaborski
5c154af744 ZStandard: introduce a wrapper
The goal is to use a file descriptor as an input for ZStandard
library. Thanks to that we shouldn't send a chunk of memory over
expensive protocol.
2022-02-08 13:11:13 -05:00
Mariusz Zaborski
b3aca9ef48 zopfli: introduce a wrapper
The goal is to use a file descriptor as an input.
Thanks to that we shouldn't send a chunk of memory over
expensive protocol.
2022-02-08 13:07:51 -05:00