This change includes a small refactoring to remember which types the generator
has already seen during header generations. Otherwise we may loop indefinitely
on certain complex types. One such type is `std::FILE` in Clang's libc++.
PiperOrigin-RevId: 335589238
Change-Id: I5bbe03b6c7fc89c743163f5534075d7912ed4e58
- Use a `constexpr inline` string constant for the forkserver env var
- Add annotation for the comms channel mutex
PiperOrigin-RevId: 335395005
Change-Id: Ic058c19c3704f182aa7ed7b8e8964b2fc5082800
We need to add the `oss-internship-2020` and `examples/hello_sapi`
directories to `.bazelignore`, so that `bazel build ...` works on a clean
working copy. This is because the Bazel builds in these directories use their
own `WORKSPACE.bazel` and this does not nest well, leading to all kinds of
hard to debug errors.
PiperOrigin-RevId: 333728800
Change-Id: Ie2e68dd39bf6f8eb21af29d8ae3ae12971b408db
Ignore cursors with types that are not implemented in python bindings
PiperOrigin-RevId: 333708345
Change-Id: I618a61c960247a9bdf89bc56dcac92e2d37b3220
- Support multiple input files
- Better testability
- Support for the `--sapi_isystem` argument, same as the Python generator
PiperOrigin-RevId: 333686891
Change-Id: I3e618165c1bd58bb755e1193617fb0737c29ee77
The Bazel build already queries the current toolchain for its system include
directories. This change brings feature parity and is necessary for systems
with unusual include locations.
PiperOrigin-RevId: 332195812
Change-Id: Ie81d614d21e90b4bd9edf2084ef80bf0d85dd750
This is a work in progress:
- Syscall tables need work
- Only tested on real hardware using one of our test hosts
As a drive-by, this change also enables the open source version to function on
POWER.
Another side-effect of this change is that the default policies no longer
check for different host architectures at runtime. On x86_64, we do not need
to check for PPC or AArch64 specifice and vice versa.
PiperOrigin-RevId: 331137472
Change-Id: Ic6d6be5cbe61d83dbe13d5a0be036871754b2eb8
This allows us to remove some uses of macros.
Related changes:
- Make it clear that we support hosting sandboxed binaries from 64-bit
processes only. CPU architectures are x86-64 and POWER64 (little endian).
- Introduced CPU architecture macros, abstracting away compiler specifics
PiperOrigin-RevId: 330918134
Change-Id: Ife7ad5f14723eec9f68055127b0583b8aecd38dd
This removes our own fork of `absl::StatusOr<>`. Sandboxed API still includes
a custom matcher for Googletest, as that is not open source yet. For
compatibility, the `statusor.h` header is still retained and now aliases
`sapi::StatusOr<>` to `absl::StatusOr<>`.
PiperOrigin-RevId: 329916309
Change-Id: I0544b73a9e312dce499bc4128c28457e04ab9929
- Use default initialization
- Rely on `static_assert()` and use `if constexpr` when checking SAPI
variable type
- Small style fixes
PiperOrigin-RevId: 322107281
Change-Id: I48cf43f354b60e31e6207552dbbfa16e3acd5615
- Drop `delim` argument from the `GetStackTrace()` family of functions.
We only ever used plain spaces.
- Use an `std::vector<std::string>` for the symbolized stack frames and
adjust the unwind proto accordingly.
This change now prints each stack frame on its own line while skipping
duplicate ones:
```
I20200717 11:47:16.811381 3636246 monitor.cc:326] Stack trace: [
I20200717 11:47:16.811415 3636246 monitor.cc:337] map:/lib/x86_64-linux-gnu/libc-2.30.so+0xceee7(0x7fb871602ee7)
I20200717 11:47:16.811420 3636246 monitor.cc:337] Rot13File+0x130(0x55ed24615995)
I20200717 11:47:16.811424 3636246 monitor.cc:337] ffi_call_unix64+0x55(0x55ed2461f2dd)
I20200717 11:47:16.811429 3636246 monitor.cc:337] map:[stack]+0x1ec80(0x7ffee4257c80)
I20200717 11:47:16.811455 3636246 monitor.cc:339] (last frame repeated 196 times)
I20200717 11:47:16.811460 3636246 monitor.cc:347] ]
```
PiperOrigin-RevId: 322089140
Change-Id: I05b0de2f4118fed90fe920c06bbd70ea0d1119e2
Initializing `absl::Span`s like by assigning them from a temporary
array leaves them pointing to invalid data. Due to the way the linker
initializes these constant tables, _most_ of them will still be valid
_most_ of the time, leading to crashes when running sandboxees with the
`--sandbox2_danger_danger_permit_all_and_log` option.
PiperOrigin-RevId: 321112099
Change-Id: I891118da08cbb6000b3e2e275618bc4edaa1d020
Bazel 3.x now requires specifying `commit`, `tag` or `branch` in its
`git_repository` rule.
PiperOrigin-RevId: 320572176
Change-Id: I81048d997f595202f4dfbd3c1e9c8321240a28a3
Otherwise ExecuteProcess is implicitly `[[noreturn]]` and this
might cause policy violations in `__asan_handle_no_return`
for ASAN builds.
PiperOrigin-RevId: 319203128
Change-Id: I5c8ba71ce88261f803aa3f16730eccea0d803dd1
--
fd2e99fa87c34f2fb1a20052c030ad7a4139b4e1 by Christian Blichmann <mail@blichmann.eu>:
Support LLVM >= 7.0.1 in Clang based header generator
This change glosses over small API changes introduced since LLVM/Clang
7.0.1, which ships with Debian Stable "Buster". Ubuntu 18.04 LTS "Bionic"
also shipped this (and subsequently updated to version 9).
Hence, compiling the generator should now work on all reasonable Debian
based distributions.
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/sandboxed-api/pull/44 from cblichmann:20200609-llvm-version-compat fd2e99fa87c34f2fb1a20052c030ad7a4139b4e1
PiperOrigin-RevId: 315637014
Change-Id: I6585041d8bebade15e44c057b1a69287bbc0e733
The "hello_sapi" example lives in a different WORKSPACE as it is intended to
show how to embed SAPI in your own projects. However, this is not compatible
with simply running `bazel build //sandboxed_api/...` after checkout.
This change simply replace `copts = ["-I."]` with `includes = ["."]`, so that
generated headers can be found reliably, regardless of how the example is
compiled.
PiperOrigin-RevId: 313782756
Change-Id: Iac26e828146b01545c81d9500f5f68fa0f2d4ddf
- Extract dependent types directly from the Clang AST and re-serialize
back into compilable code
- Collect types and emit diagnostics
- Format generated code
Signed-off-by: Christian Blichmann <mail@blichmann.eu>
This change contains a "hello world"-style example library to be
sandboxed. It consists of a stand-alone CMake and Bazel project that
uses Sandboxed API as its dependency.
To use Sandboxed API in an external project, it should be enough to
copy the files in the `sandboxed_api/examples/hello_sapi` directory
as a starting point.
When using SAPI from an external project via CMake, the small zlib patch
cannot be found during the config phase.
This change also fixes an oversight in that the tests depend on some of
the example code.
Regardless embedding code should set `SAPI_ENABLE_EXAMPLES` to `OFF`.
Disabling tests by setting `SAPI_ENABLE_TESTS` to `OFF` stays optional.
PiperOrigin-RevId: 311290454
Change-Id: Ibe105895859b793b20f47c89b1a9e11cbfef2e2f
Build macros in earlier versions of Bazel (pre-1.0) needed to specify
targets in a different format, depending from where they were
included/expanded at build time.
For example, a `sapi_library()` macro invocation in one of the directories
under `examples`, always needed to depend on the SAPI main library as
`//sandboxed_api:sapi`. When using SAPI from another Bazel project, the
same macro would have needed to depend on
`@com_google_sandboxed_api//sandboxed_api:sapi`. The `sapi_library()`
macro was thus checking the repository name and conditionally changed
the dependencies. This approach is brittle and as of Bazel 3.1.0 no
longer works.
This CL simple removes the conditional prefix and unconditionally uses
`@com_google_sandboxed_api`.
Tested on Bazel 1.2.1, 2.2.0 and 3.1.0
Similar to what the Bazel build does, this change adds zlib as an additional
dependency when `SAPI_ENABLE_EXAMPLES` is set to `ON`.
PiperOrigin-RevId: 309203959
Change-Id: I201a9e6415789afb1e058bc48cebbc0fc0004fe9
BoringSSL (which is the crypto library used by most Google products) is starting to use madvise(_, _, MADV_WIPEONFORK) to protect random-number state from being duplicated by fork(). This causes extra madvise calls that sandboxes need to permit in order to continue functioning.
PiperOrigin-RevId: 309173849
Change-Id: I007dacc1ff1fd0ccc138caaa08735cfe5bc78234
Fixes some template issues that could lead to code not compiling when it
otherwise should.
PiperOrigin-RevId: 308809964
Change-Id: I9f2f9d4aff5f1a9cb967fb705a86fd7f49114f7a
The underlying issue is with a Copybara transform and the internal Blaze BUILD file.
PiperOrigin-RevId: 308787839
Change-Id: I32af664b3eac4c925d39f50b967756198eff23f3
Note: This intentionally omits perfect-forwarding value assignments. This
avoids overly complex template expressions. The regular assignments are still
efficient.
PiperOrigin-RevId: 304159053
Change-Id: I3460f46ca5779a0619cf90ae22625de8fad7669c
Drive-by:
- Make Bazel config action more robust
- Disable dependency tracking in libunwind configure
Signed-off-by: Christian Blichmann <mail@blichmann.eu>
When embedding SAPI in an external CMake project, the version of
`protobuf_generate_cpp` that we lifted from upstream protobuf produces
the wrong generated file paths.
For example, given this project structure:
```
/parent/
+-- myproject/
+-- myproject_build/ <- CMake build directory
+-- sandboxed-api/ <- Checkout from GitHub
```
And a CMake file in `myproject/CMakeLists.txt` that embeds SAPI like
this:
```
cmake_minimum_required(VERSION 3.12)
project(SandboxedTest LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
add_subdirectory(
${PROJECT_SOURCE_DIR}/../sandboxed-api
${PROJECT_BINARY_DIR}/sandboxed-api
)
```
Then `protobuf_generate_cpp` correctly invokes the protoc compiler to
generate
`/parent/myproject_build/sandboxed-api/sandboxed_api/proto_arg.proto.pb.cc'.
However, the path of the generated source file that is passed to the C++
compiler will be
`/parent/myproject_build/sandboxed-api/sandboxed_api/../../myproject_build/sandboxed-api/sandboxed_api/proto_arg.pb.cc`.
Note the duplicated project build directory component in the
canonicalized version:
`/parent/myproject_build/myproject_build/sandboxed-api/sandboxed_api/proto_arg.pb.cc`.
This change simple omits the computation of any relative file paths and
simply uses `_pb_PROTOC_OUT_DIR` which defauls to
`CMAKE_CURRENT_BINARY_DIR`, which should always contain the correct
path.
Signed-off-by: Christian Blichmann <mail@blichmann.eu>
Created documentation for network proxy. fixed 2 things in documentation (namespaces are enabled by default for a while).
PiperOrigin-RevId: 300321016
Change-Id: Id9c54b29551e8d3b70e814e2fdbfee594126aa90
Currently we extract all functions from the compilation unit - it doesn't really make sense as it will try to process functions in included files too.
As we require sapi_in flag, we have information which files are of interest.
This change stores the top path in _TranslationUnit and uses it when looking for function definitions to filter only functions from the path we provided.
PiperOrigin-RevId: 297342507
Change-Id: Ie411321d375168f413f9f153a606c1113f55e79a
The next release contains one major change: for function parameter annotations,
pytype will no longer treat (x: X = None) as equivalent to
(x: Optional[X] = None). This pytype behavior was based on an outdated version
of the type-checking spec. Annotations that were relying on the behavior now
need to explicitly declare themselves as Optional.
PiperOrigin-RevId: 297065077
Change-Id: Iade679e5928bb3839485e8b8571945456ba6e982
Sending -1 as fd will fail and take forkserver down.
This should not happen normally so turned it into a check.
PiperOrigin-RevId: 285391908
Change-Id: Idbb05004c36cb0be57be1bd26df1c57cecfb0019
- Same upstream dependency versions for CMake and Bazel
- Use maybe() from build_defs to conditionally define Bazel repositories
PiperOrigin-RevId: 282331418
Change-Id: I8a7db84a1d3115f3c1f31cc1dfcc8c29fe391a62
Practically all current compilers fully support C++17 by now, with the exception of `filesystem` (which we won't use).
PiperOrigin-RevId: 281494879
Change-Id: Ia0ef650ede4e5907e4a6421b8ad11743c8bae88f
This addresses a latency issue - chroot_fs_refs called inside pivot_root
in the kernel can take several milliseconds on machines with many threads
running.
This might not always reduce latency for custom forkservers, as additional
fork can be more costly than pivot_root.
PiperOrigin-RevId: 281306284
Change-Id: If503ac76a70e5438e94caf708d79cb0219c66def
The previous one was not quite correct in cases where the outside binary path
did not match the inside path. This should be fixed with this.
PiperOrigin-RevId: 276075886
Change-Id: I1c7c4fa0191960437a2d2360b805c7098b1407c9
Caused failures if root is mounted as nodev, nosuid etc.
Root is pivoted and unmounted right after this anyhow.
PiperOrigin-RevId: 273707731
Change-Id: I75f1edaf2877c096e4f5bb7dc1b2bb8eb5c437a3
This makes the class more ergonomic because
* You don't have to heap allocate the builder.
* You can create a policy builder "template" and re-use it across sandboxes to avoid repetitive work.
PiperOrigin-RevId: 273555679
Change-Id: I4084ee9c74f95ebfde873eb0dc021b3b3cdc5ea2
These where inadvertently introduced in an internal cleanup change.
This change also removes a C++17-ism in var_proto.h. To make things easier for
downstream projects, we should stick to C++11 for the time being.
PiperOrigin-RevId: 271117700
Change-Id: I4eaacec88be16e1a561d3f77a61acce0a1af0b9d
This change removes the in-tree documentation in favor of the one hosted on
the Google's Developer site. This makes it easier to maintain for the
sandbox team, as there is now only a single source of truth for both internal
and external documentation.
If you find errors/bugs in the documentation, please file regular GitHub
issues.
PiperOrigin-RevId: 267132623
Change-Id: I4c950fcef77da0b361cb35b99aa2f187efe6f320
This fixes some CMake-3.12+-isms, like `list(JOIN ...)` and setting link
properties on OBJECT libraries.
PiperOrigin-RevId: 265033725
Change-Id: I0ee1ec0e1b1097ea8226ad6fdeff794a97c2881b
The temporary solution for binaries/tests that fully statically linked is to
link against `gflags` using `-Wl,--whole-archive`. This will no longer be
necessary, once Abseil ships with logging. Then we can (finally) use Abseil
flags and use a logging library that does not depend on a different flags
library.
PiperOrigin-RevId: 260705702
Change-Id: I8562faaff59f9c3e0e1d331186d2806d387438fb
This change moves away from a classical superbuild which downloads and builds
at build time. Instead, we now follow a "Fetch Content" workflow (available as
FetchContent in CMake 3.11+) and download dependencies at config time.
Rationale: Superbuild projects have the disadvantage that projects cannot
directly access their individual declared targets. This is not a problem with
regular libraries, as those are usually/supposed to be installed. With
Sandboxed API, this is not desirable, as it has dependencies like Abseil and
glog, which are almost always consumed by including their source tree using
add_subdirectory().
Fixes#10 and makes external embedding easier.
PiperOrigin-RevId: 260129870
Change-Id: I70f295f29a6e4fc8c330512c94b01ef10c017166
Also properly check status of send and use one-byte messages
to avoid issues with partial send, receive.
PiperOrigin-RevId: 258362495
Change-Id: I889b4699c100c80d15b129bf3a254f5442405bc2
PR_SET_PDEATHSIG is already issued for init process in SanitizeCurrentProcess
Same for setsid
PiperOrigin-RevId: 258142844
Change-Id: Iad9e94bd402d576c1b24caab0b03efc50e2df07e
* Move add_sapi_library() into a build defs file
* Override protobuf_generate_cpp() instead of naming it sapi_protobuf_generate_cpp()
* Factor out inclusion/find_package() calls of external dependencies
PiperOrigin-RevId: 258133422
Change-Id: Ibdbab0c735157eac0ed6122ab78f9d583c6905cc
* Implement add_sapi_library() similar to the Bazel rule
* Fix SuperBuild so that glog properly depends on gflags (needed for --logtostderr)
* Raise minimum required version of CMake to 3.10 (Ubuntu 18.04 LTS)
* Update sum sandbox policy to allow to get IDs and PIDs
PiperOrigin-RevId: 258124160
Change-Id: I64c2f8b27a3e842874adca9100bfce20a2b74f17
This change adds CMake targets for the actual Sandboxed API library. On its
own, this is not very useful, as there is no CMake equivalent to the
sapi_library() rule, yet. This will be worked on next.
PiperOrigin-RevId: 257177362
Change-Id: Ic06c945ab534b27306021d970a83691aae2e7e60
When fetching exit status only lower 8-bits will be read.
Thus if getpid()&0xff == 0 the test can fail.
PiperOrigin-RevId: 257163766
Change-Id: I690c645fde33d1205578fd8873c5fc2974352ada
Since we use glog, we need to still rely on gFlags for as long as Abseil
logging is not yet released. To do this, we have the util/flag.h header in
SAPI, faking the newer style API of Abseil's flags library.
The rationale behind this is that glog includes/builds with gflags for
command line flag parsing and having more than one flag parsing library
active at the same time is asking for trouble.
PiperOrigin-RevId: 256982667
Change-Id: I9c5406cb9ef4458daebfec0b0713f9bddbf5c3ad
This makes it easier to work with the generated classes from client code.
PiperOrigin-RevId: 256132914
Change-Id: If7fb24a2cac998d2cdb04e8b627272064bf9793d
- made sure that define order is correct
- made sure to emit all defines related to target define
- fixed a bug where '(' was separated with macro name with space, this breaking the macro
PiperOrigin-RevId: 256129616
Change-Id: I636b13a72c6198fb59e8e387f42567c442b24352
Since the interface generator is invoked via a Bazel macro, it will be
expanded in the embedding context of the project using SAPI, so package access
needs to go through the full workspace root @com_google_sandboxed_api.
This change also modifies the CMakeLists.txt accordingly, as the "external"
subdirectory is no longer needed/wanted.
PiperOrigin-RevId: 255918784
Change-Id: I052c687509f65fef7f011a9d1a074a171595330f
If using Sandboxed API from an external Bazel WORKSPACE, dependencies can be
imported by declaring the repository and calling `sapi_deps()`:
```python
git_repository(
name = "com_google_sandboxed_api",
commit = "master",
remote = "https://github.com/google/sandboxed-api.git",
)
load(
"@com_google_sandboxed_api//sandboxed_api/bazel:sapi_deps.bzl",
"sapi_deps",
)
sapi_deps()
```
PiperOrigin-RevId: 255373664
Change-Id: I4c41ffc75e1ca95eb126702cb2d5cedb255191d5
This is tracked at
https://github.com/bazelbuild/bazel/issues/8672
and should only affect the tests for now.
PiperOrigin-RevId: 254943708
Change-Id: Iee18a3b8c1b570c7d04d09d533ecff33c3de7d57
We now require that Debian users install the `python3` and `python3-pip`
packages. This change lets the Python code search for `libclang.so`, which can
be located in different directories, depending on version, and is not found by
default otherwise.
Fixes#28
PiperOrigin-RevId: 254745872
Change-Id: Ia77680da2a3235c0a9518125676aa8a460e38e76
Starting with Bazel 0.27.0 --incompatible_depset_is_not_iterable became the
default.
Fixes#27
PiperOrigin-RevId: 254703981
Change-Id: I727e4b1c6a907c8794a00f61646c13ad8d4437b7
This code assumes, like Bazel's runfiles that the data dependency to access
exists in the same sub-tree as the binary:
WORKSPACE
+- sandboxed_api/sandbox2/examples/crc4
+- crc4bin
+- crc4sandbox
The code requires the directory structure to exist, so that in the example
above, crc4sandbox can use
GetDataDependencyFilepath("sandboxed_api/sandbox2/examples/crc4/crc4bin")
regardless of how it was called.
PiperOrigin-RevId: 251834480
Change-Id: I6470b62ce9b403297116481a0c17c070992f2e81
This test runs, but fails due to a different problem:
StackTraceTest.ForkEnterNsLibunwindDoesNotLeakFDs
PiperOrigin-RevId: 251218516
Change-Id: If06cdbcb71fad84ebd9d934ff173d7ef1a1eebc0
Currently mostly no-op, but this is the first step to turn namespaces on
by default.
PiperOrigin-RevId: 249439158
Change-Id: I5eeb1216dc868c660f62ad50c34f626afbf7db61
Process might be killed between waitpid and PTRACE_CONT,
even though a PTRACE_EVENT_EXIT will be gererated, continuing
will fail with ESRCH in that case.
PiperOrigin-RevId: 249245726
Change-Id: Ib673529229a306d2266fa60caa3039b6bcd80a65
After all requested filesystem mounts are fully mounted under a sandboxee's virtual chroot, print a list of the outside paths and a list of the inside chroot paths that the outside paths are mapped to. This provides a valuable insight while debugging sandboxed binaries.
PiperOrigin-RevId: 247130923
Change-Id: I42b4b3db68d826587c0fe8127aabbead38bc6f20
- Add a superbuild in cmake/SuperBuild.cmake that downloads and builds
dependencies
- Builds for sandbox2/ and a its tests
- Helper CMake function to strip proto paths
- Module to find libcap
- Custom build for libunwind that wraps its symbols
- Fix environment so that CTest executes tests similar to Bazel
- Filewrapper functionality, like Bazel's cc_embed_data()
- Build forkserver with embedded binary
- Enable ASM language so that libunwind builds correctly
- Allow glog target to propagate transitively (to propagate its include dirs)
Signed-off-by: Christian Blichmann <cblichmann@google.com>
On some newer compiler versions, compiler optimizations and loop unrolling
change the memory layout so that 64 bytes are not enough to overwrite the
return address reliably.
PiperOrigin-RevId: 240343358
Change-Id: Ifb1a1dc1cb482793b7387887f0fd68a237879227
Note: These commits only change very minor formatting issues in
the GitHub version. There is more than meets the eye, though. These
changes help to be able to accept pull requests.
PiperOrigin-RevId: 239225828
Change-Id: Ib31bf114e7cc4ccda49f7dcc4e9e24eebc735065
