Commit Graph

1219 Commits

Author SHA1 Message Date
Christian Blichmann
67bc67bbef clang_generator: Handle inter-type dependencies better
This change changes the emitter to work on `clang::TypeDecl`s instead of
`clang::QualType`s, as the latter only describe complete definitions. This
allows us to better deal with situations where we would otherwise have a kind
of circular type dependencies: For example, a `typedef` that depends on a
`struct` which in turn has a member that references the typedef.
For this to work, we now also record the actual source order of declarations,
similar to what the libclang based header generator does.

Also add some more tests for the newly covered cases.

PiperOrigin-RevId: 488370914
Change-Id: I2d1baa863fb12d1801bf11a20f3f88df7fde3129
2022-11-14 08:19:13 -08:00
Liam Miller-Cushon
b626bf40da Migrate from cfg = "host" to cfg = "exec"
PiperOrigin-RevId: 488355973
Change-Id: I965bbef502e26b0f5908f92ebc614aef31c42659
2022-11-14 07:08:08 -08:00
Christian Blichmann
1ae04ac332 clang_generator: Use fully-qualified names, emit in source order
When generating headers from libraries that export functions as `extern "C"`
but still use namespaces (C-compatible C++ libraries), we want to generate
a Sandboxed API that includes fully-qualified namespace names as well.

In addition, we want the generated API to have the same source order as the
original library. Not only is this less surprising when reading the generated
code, it's also more accurate. Previously, we'd bundle all definitions in a
namespace and sort those alphabetically, but for code that relies on symbols
from another namespace to be available, generation will fail:

```c++
namespace zzz {
using entity_count_t = uint64_t;
}  // namespace zzz
namespace sheep_counter {
using sheep_count_t = :💤:entity_count_t;
extern "C" void IncreaseSheepCounter(sheep_count_t increment);
}  // namespace sheep_counter
```

PiperOrigin-RevId: 486586024
Change-Id: I419c9db8e9cb5b904364b353e2dc3d7f1030fab3
2022-11-07 00:37:53 -08:00
Christian Blichmann
ce26b55e26 clang_generator: Turn fatal error on struc-by-value into warning
An `absl::StatusCode` of `kCancelled` now indicates warnings inside the emitter.

PiperOrigin-RevId: 485851898
Change-Id: I10a57cbc1b6c2d4b708c3c19aa0fa71451845a22
2022-11-03 06:02:32 -07:00
Rebecca Chen
3abfefaf3b Silence some pytype errors.
PiperOrigin-RevId: 485526322
Change-Id: I55541b3f7c2f8e89efef0d37bbd2ef4e330bd3e7
2022-11-02 01:08:49 -07:00
Christian Blichmann
4b56d5606d clang_generator: Fail header generation with struct-by-value
The libclang based header generator disallows functions that pass structs (or
more generally "record types") by value. While this can be implemented, the
such functions as emitted by the clang_generator never worked.

We should revisit this when we implement support for passing 128-bit integer
types directly, as those will work the same as small structs.

PiperOrigin-RevId: 485522603
Change-Id: Iae8284720da52496d7a48fe3ca3c3c8605e6d19d
2022-11-02 00:43:26 -07:00
Christian Blichmann
7e0f72e445 clang_generator: Refactor frontend test utilities
This makes it easier to write tests with expected compilation/parsing failures.

PiperOrigin-RevId: 485331205
Change-Id: Ia545934849d38bded9320537c71e970fb4730cb6
2022-11-01 09:00:34 -07:00
Christian Blichmann
c3889ce379 Fix command-line handling in sandbox2tool
This addresses #164.

PiperOrigin-RevId: 483675926
Change-Id: I1461c9bb2c3865d86cd99f9285e51ce20ac460b8
2022-10-25 08:05:23 -07:00
Christian Blichmann
6222ffe04f contrib: More fixes for flags and logging migration
PiperOrigin-RevId: 483656661
Change-Id: I5b8783de3f9f48056779fcce832a5b23df74cb72
2022-10-25 06:33:42 -07:00
Christian Blichmann
6fbfb8f9bd Remove Tag constructor, add standard comment for absl::WrapUnique(new T)
PiperOrigin-RevId: 483654433
Change-Id: I16b058a6b186f764f45bc5540f3f49d5a294ddeb
2022-10-25 06:20:51 -07:00
Christian Blichmann
8d04efa62d contrib: Replace uses of CHECK_NOTNULL
Abseil's standard name for this is `ABSL_DIE_IF_NULL`.

PiperOrigin-RevId: 483648443
Change-Id: I9d6826443be72b30f71c18972436fa5f9c05048a
2022-10-25 05:50:59 -07:00
Wiktor Garbacz
3b95bec700 Remove unused SAPI_INTERNAL_UNREACHABLE
PiperOrigin-RevId: 483393540
Change-Id: I315da44529fb2c0d8b6b6347275fdf66c5aa4712
2022-10-24 08:48:35 -07:00
Christian Blichmann
4c87556901 Use Abseil's log/flags instead of glog/gflags
Follow-up changes might be required to fully fix up the contrib sandboxes.

PiperOrigin-RevId: 482475998
Change-Id: Iff631eb838a024b2f047a1be61bb27e35a8ff2f4
2022-10-20 06:48:51 -07:00
Benjamin Kramer
7dca070083 Integrate LLVM at llvm/llvm-project@06da9b94ae
Updates LLVM usage to match
[06da9b94ae37](https://github.com/llvm/llvm-project/commit/06da9b94ae37)

PiperOrigin-RevId: 481181270
Change-Id: I8c015fbd95d7f5f543d4eed12ed5d4efc11ef9c3
2022-10-14 10:42:23 -07:00
Sandboxed API Team
882e474032 Bazel: Add copts attribute to macros which use _default_copts where necessary
Propagate `_default_copts` to rules which use them.

PiperOrigin-RevId: 480871797
Change-Id: I8be371d2db5db103e6ab6ef31a181f2f818a1fc2
2022-10-13 06:01:26 -07:00
Christian Blichmann
79b6784b82 #Cleanup: Consistently use std::make_unique
PiperOrigin-RevId: 480597371
Change-Id: I145586382ad7a7694384cc672986132376a47465
2022-10-12 05:23:42 -07:00
Wiktor Garbacz
cb8efdc270 Sandbox2: Graciously handle mapping over Comms/Exec fds
Try to move the affected FDs transparently to avoid conflict.

PiperOrigin-RevId: 480105375
Change-Id: I0cd093fce120505d1cd4a1d081b3c0e63bf0210a
2022-10-10 09:39:01 -07:00
Christian Blichmann
b9c2830ebc Use new sandbox2::Comms ctor for default connection params
This change allows Sandbox2 to change how the default FD for comms is chosen.

PiperOrigin-RevId: 479526309
Change-Id: I69add85a244bc0385eaa164ab0ea3b036503c6d3
2022-10-07 02:08:20 -07:00
Sandboxed API Team
859e282d3b Use new sandbox2::Comms ctor for default connection params
This change allows Sandbox2 to change how the default FD for comms is chosen.

PiperOrigin-RevId: 479521810
Change-Id: Ia2ca1df95eb21783207ffb625c924790de20480d
2022-10-07 01:44:46 -07:00
Wiktor Garbacz
3198ff06d3 Explicit Comms constructor with default params
This is to abstract the FD number away, so that we can change the way the FD number is chosen/communicated.

PiperOrigin-RevId: 479282707
Change-Id: Ic6726bcd0a17e97bde60804476ecbca2ffbf6525
2022-10-06 04:56:18 -07:00
Christian Blichmann
5b61445de9 Internal change
We have removed an internal-only sandbox mechanism that has been deprecated
for years. Some formatting/include changes may leak into the OSS version.

PiperOrigin-RevId: 475230500
Change-Id: Ib4efdf3282529ea50e8302e5ef7acfdd7d4c68e5
2022-09-19 01:58:32 -07:00
Wiktor Garbacz
d2c8c70d8e Internal change
PiperOrigin-RevId: 475224729
Change-Id: Id7c05c7542c44f58e7f4027c6932acd42f3a7857
2022-09-19 01:17:22 -07:00
Christian Blichmann
4d24c4c01b Update to latest Abseil and Protobuf
This is the first in a series of changes that will remove our custom logging
implementation in favor of the newly released Abseil log library.

PiperOrigin-RevId: 475221012
Change-Id: I5d21ad104049dc70abe2a8d49659128e9cf3e9c0
2022-09-19 00:52:35 -07:00
Christian Blichmann
8de530036f Internal change.
Some includes may leak to OSS.

PiperOrigin-RevId: 474748898
Change-Id: Iff9dc4f91af211572ff4bbcf57330b36d7a957ab
2022-09-16 00:37:02 -07:00
Christian Blichmann
39a1bc9d7a Skip system headers in Clang generator
When not requesting any particular function, `sapi_library()` will try and
make available _all_ functions it finds. In this case, system headers should
be skipped to avoid inflating the API surface. Standard library functions
can still be manually requested by adding them to the `functions` (Bazel)/
`FUNCTIONS` (CMake) argument.

PiperOrigin-RevId: 472272506
Change-Id: I8f8d79796d3044e598eebb7f87ce4cf464b47ed7
2022-09-05 07:15:46 -07:00
Sandboxed API Team
75c7081622 For the SECCOMP event, check if the event msg is in the range of one of the known architectures.
If it isn't, assume that the process has exited and the event msg contains an exit code.

PiperOrigin-RevId: 471258449
Change-Id: I44408c30fe7fb39e20b55cea871f3efb68fcde67
2022-08-31 08:09:37 -07:00
Sandboxed API Team
e541f79abd forkserver_bin is usually embedded via cc_embed_data. So there is no real reason why it should be stamped.
PiperOrigin-RevId: 470013947
Change-Id: I7ff11fafdebb49e14c2b5dcae48c31fda6da2833
2022-08-25 09:54:24 -07:00
Christian Blichmann
7008aa21b6 Remove leftover definition from move to SyscallTable
PiperOrigin-RevId: 467930784
Change-Id: Id149fe9ef85718f28fcb396b03b574c32dc846d8
2022-08-16 08:24:56 -07:00
Sandboxed API Team
28504f1817 Make code not have a -Warray-parameter warning.
PiperOrigin-RevId: 467842322
Change-Id: Ic262a3f98fa823ef524ac02d08b2f5b8f4adf71d
2022-08-15 22:55:51 -07:00
Sandboxed API Team
deb3c8e77b Batch threads waiting for the monitor's attention.
Instead of doing waitpid() and processing one thread at a time, gather all waiting threads and then process them.

This avoids starving older threads when newer threads raise a lot of events.

PiperOrigin-RevId: 466366533
Change-Id: I81a878f038feac86407a8e961ecba181004f0f8a
2022-08-09 08:28:03 -07:00
Sandboxed API Team
26b2519aed Integrate LLVM at llvm/llvm-project@ea460b7ddb
Updates LLVM usage to match
[ea460b7ddb8a](https://github.com/llvm/llvm-project/commit/ea460b7ddb8a)

PiperOrigin-RevId: 465026637
Change-Id: Ie32d8e89e2824b6e7d4b3f8f1588e55b6c72bb1e
2022-08-03 04:55:55 -07:00
Sandboxed API Team
78ee270388 Remove information about in-progress syscalls on process exit.
PiperOrigin-RevId: 463091104
Change-Id: I402cb61e9e816a20a87274ea874cddf91c101e14
2022-07-25 08:28:25 -07:00
Sandboxed API Team
4d906e7143 Fix visibility
PiperOrigin-RevId: 461617454
Change-Id: Id77bfbec2cc095005a434251c056b19c3c6a64c4
2022-07-18 07:44:38 -07:00
Wiktor Garbacz
1e4cf06f69 Block installing user notify inside Sandbox2
PiperOrigin-RevId: 458781163
Change-Id: Ifcaf940d8a70a9a4ab5b24aefdaaae622cfce4f3
2022-07-03 11:20:31 -07:00
Sandboxed API Team
055839ab11 Update rules that require a cpp toolchain to use a helper function that defines the toolchain type.
PiperOrigin-RevId: 457645244
Change-Id: Ia99251f24e4baec9deb0ff57b86cb388c9cdfb68
2022-06-27 22:24:54 -07:00
Sandboxed API Team
e5bc3e69cd "Stack traces have been disabled" message goes to VLOG instead of INFO.
PiperOrigin-RevId: 456755121
Change-Id: I7eb7badcd5901a33dd2b2afc0833f00eeedacada
2022-06-23 06:42:35 -07:00
Christian Blichmann
82e164949b CMake: Move proto import path replacement to build time
With this change, the generated build system can now decide when the SAPI
protos are out of date and need to be rebuilt. Previously, the protos had
to always be regenerated which meant compiling a big part of the codebase
after each CMake configure run, even if nothing else changed.

PiperOrigin-RevId: 455599389
Change-Id: If75b6e870cd37f3136681fc73a404c5678623e52
2022-06-17 05:56:31 -07:00
Sandboxed API Team
81871a98f7 Internal-only change.
PiperOrigin-RevId: 455553721
Change-Id: I923ab39b9bcd92a6a8e0dd8f95b01cc135ace919
2022-06-17 00:37:39 -07:00
Christian Blichmann
57a4e7e7bc GitHub Actions: Rename workflows, fix libxls
- Shorter names, as build matrix already contains OS/compiler or
  OS/contrib-package combinations
- libxls was missing `gettext` and `autoconf-archive` packages

PiperOrigin-RevId: 455079247
Change-Id: Iae55644a818f3e1840cc18344caa9cc4277d012b
2022-06-15 02:59:23 -07:00
Christian Blichmann
1ef1fd90ec libidn2: Use pkg-config for include dir
PiperOrigin-RevId: 455050938
Change-Id: I73db6d3036ade8fc4638d7a3a5cfd659dd83555c
2022-06-15 00:07:28 -07:00
Christian Blichmann
e29e5cb1a2 GitHub Actions: Add libzip and tests
- Update Sandbox policy
- Compile libzip wrapper functions as part of libzip in CMake

PiperOrigin-RevId: 454837665
Change-Id: Ife6cc99296873e030b9613959eff88d4b0746a5e
2022-06-14 05:42:12 -07:00
Christian Blichmann
3cb19e7378 GitHub Actions: Add libxls and tests
PiperOrigin-RevId: 454835693
Change-Id: I575ef9519ca829a2d4815e2dde4348cb9a42f36c
2022-06-14 05:27:46 -07:00
Christian Blichmann
45d4b1ba5b GitHub Actions: Add libidn2 and tests
PiperOrigin-RevId: 454784764
Change-Id: Id8de7f34a0a5e0236f2fb92841e117cdcf386d2c
2022-06-14 00:05:25 -07:00
Christian Blichmann
2a65b72ea6 Add c-blosc and hunspell and their tests to CI
- Minor reformatting in `CMakeLists.txt`
- Update c-blosc to latest revision

PiperOrigin-RevId: 454148849
Change-Id: I7a659c0786b1dc35d94059a518a0ec2859055432
2022-06-10 06:15:34 -07:00
Christian Blichmann
4ec09d0061 Update and move lodepng to contrib/
- Drop patch from `CMakeLists.txt` in favor forcing C compilation
- Use `sapi` namespace and new logging integration
- Update sandbox policy to allow to retrieve thread ids
- Add tests to GitHub Workflow

PiperOrigin-RevId: 454133584
Change-Id: I50946245c723eb1e496ed1403b70ba08d977494e
2022-06-10 04:24:12 -07:00
Christian Blichmann
b11ce4b24a Add brotli and tests to CI
Drive-by:
- Update sandbox policy
- Formatting fixes
- Updated comments
PiperOrigin-RevId: 453901669
Change-Id: I40e0fbd26525ba564d4e062c79752a0102c48b15
2022-06-09 05:28:31 -07:00
Christian Blichmann
4872ba6569 libraw: Fix left-over from cleanup
Our internal builds do not build the `contrib/` sandboxes, so I didn't notice
the failing libraw build :-/

PiperOrigin-RevId: 453868469
Change-Id: Ic084b066197ace6f52c3e7ed541a811c501d20b1
2022-06-09 02:04:39 -07:00
Christian Blichmann
95afede8a0 GitHub Actions: Add Ubuntu 22.04 builds
Ubuntu 22.04 is an LTS version, so we should add it. This change set
`ignore-errors = true`, as the corresponding build environment is still in
beta on GitHub.

PiperOrigin-RevId: 453868434
Change-Id: I988e38cda30deedd0704314f21a1f4c33c1456a0
2022-06-09 02:03:48 -07:00
Piotr Bartman
0e7abb70fe Copybara import of the project:
--
656cd15cb6 by Piotr Bartman <prbartman@gmail.com>:

LibRaw

--
a505222184 by Piotr Bartman <prbartman@gmail.com>:

CMake cleanup

--
7fc66b2026 by Piotr Bartman <prbartman@gmail.com>:

cleanup + img.raw

--
1b1c085cfe by Piotr Bartman <prbartman@gmail.com>:

libraw: data checking

--
7e76425c37 by Piotr Bartman <prbartman@gmail.com>:

libraw: CR @cblichmann
PiperOrigin-RevId: 453859071
Change-Id: Ib9e1887f97d48ecbebda05c5c6df01e3642bbfba
2022-06-09 00:55:24 -07:00
Christian Blichmann
de836031d4 cmake: Use configure_file() to implement forced C++ linkage
This finally prevents Ninja from rebuilding everything needlessly each time a
CMake reconfiguration was triggered. The root cause is that we used
`file(WRITE ...)`, which always unconditionally overwrites, so Ninja sees
those files as "dirty".

PiperOrigin-RevId: 453849514
Change-Id: Ib878df21371387baa7bf791a0a054e1ea5d6b6ae
2022-06-08 23:38:50 -07:00