Commit Graph

93 Commits

Author SHA1 Message Date
Christian Blichmann
67bc67bbef clang_generator: Handle inter-type dependencies better
This change changes the emitter to work on `clang::TypeDecl`s instead of
`clang::QualType`s, as the latter only describe complete definitions. This
allows us to better deal with situations where we would otherwise have a kind
of circular type dependencies: For example, a `typedef` that depends on a
`struct` which in turn has a member that references the typedef.
For this to work, we now also record the actual source order of declarations,
similar to what the libclang based header generator does.

Also add some more tests for the newly covered cases.

PiperOrigin-RevId: 488370914
Change-Id: I2d1baa863fb12d1801bf11a20f3f88df7fde3129
2022-11-14 08:19:13 -08:00
Christian Blichmann
1ae04ac332 clang_generator: Use fully-qualified names, emit in source order
When generating headers from libraries that export functions as `extern "C"`
but still use namespaces (C-compatible C++ libraries), we want to generate
a Sandboxed API that includes fully-qualified namespace names as well.

In addition, we want the generated API to have the same source order as the
original library. Not only is this less surprising when reading the generated
code, it's also more accurate. Previously, we'd bundle all definitions in a
namespace and sort those alphabetically, but for code that relies on symbols
from another namespace to be available, generation will fail:

```c++
namespace zzz {
using entity_count_t = uint64_t;
}  // namespace zzz
namespace sheep_counter {
using sheep_count_t = :💤:entity_count_t;
extern "C" void IncreaseSheepCounter(sheep_count_t increment);
}  // namespace sheep_counter
```

PiperOrigin-RevId: 486586024
Change-Id: I419c9db8e9cb5b904364b353e2dc3d7f1030fab3
2022-11-07 00:37:53 -08:00
Christian Blichmann
ce26b55e26 clang_generator: Turn fatal error on struc-by-value into warning
An `absl::StatusCode` of `kCancelled` now indicates warnings inside the emitter.

PiperOrigin-RevId: 485851898
Change-Id: I10a57cbc1b6c2d4b708c3c19aa0fa71451845a22
2022-11-03 06:02:32 -07:00
Rebecca Chen
3abfefaf3b Silence some pytype errors.
PiperOrigin-RevId: 485526322
Change-Id: I55541b3f7c2f8e89efef0d37bbd2ef4e330bd3e7
2022-11-02 01:08:49 -07:00
Christian Blichmann
4b56d5606d clang_generator: Fail header generation with struct-by-value
The libclang based header generator disallows functions that pass structs (or
more generally "record types") by value. While this can be implemented, the
such functions as emitted by the clang_generator never worked.

We should revisit this when we implement support for passing 128-bit integer
types directly, as those will work the same as small structs.

PiperOrigin-RevId: 485522603
Change-Id: Iae8284720da52496d7a48fe3ca3c3c8605e6d19d
2022-11-02 00:43:26 -07:00
Christian Blichmann
7e0f72e445 clang_generator: Refactor frontend test utilities
This makes it easier to write tests with expected compilation/parsing failures.

PiperOrigin-RevId: 485331205
Change-Id: Ia545934849d38bded9320537c71e970fb4730cb6
2022-11-01 09:00:34 -07:00
Christian Blichmann
4c87556901 Use Abseil's log/flags instead of glog/gflags
Follow-up changes might be required to fully fix up the contrib sandboxes.

PiperOrigin-RevId: 482475998
Change-Id: Iff631eb838a024b2f047a1be61bb27e35a8ff2f4
2022-10-20 06:48:51 -07:00
Benjamin Kramer
7dca070083 Integrate LLVM at llvm/llvm-project@06da9b94ae
Updates LLVM usage to match
[06da9b94ae37](https://github.com/llvm/llvm-project/commit/06da9b94ae37)

PiperOrigin-RevId: 481181270
Change-Id: I8c015fbd95d7f5f543d4eed12ed5d4efc11ef9c3
2022-10-14 10:42:23 -07:00
Christian Blichmann
79b6784b82 #Cleanup: Consistently use std::make_unique
PiperOrigin-RevId: 480597371
Change-Id: I145586382ad7a7694384cc672986132376a47465
2022-10-12 05:23:42 -07:00
Christian Blichmann
39a1bc9d7a Skip system headers in Clang generator
When not requesting any particular function, `sapi_library()` will try and
make available _all_ functions it finds. In this case, system headers should
be skipped to avoid inflating the API surface. Standard library functions
can still be manually requested by adding them to the `functions` (Bazel)/
`FUNCTIONS` (CMake) argument.

PiperOrigin-RevId: 472272506
Change-Id: I8f8d79796d3044e598eebb7f87ce4cf464b47ed7
2022-09-05 07:15:46 -07:00
Sandboxed API Team
26b2519aed Integrate LLVM at llvm/llvm-project@ea460b7ddb
Updates LLVM usage to match
[ea460b7ddb8a](https://github.com/llvm/llvm-project/commit/ea460b7ddb8a)

PiperOrigin-RevId: 465026637
Change-Id: Ie32d8e89e2824b6e7d4b3f8f1588e55b6c72bb1e
2022-08-03 04:55:55 -07:00
Christian Blichmann
69ed3d6946 clang_generator: Fix build breakage with most recent LLVM
PiperOrigin-RevId: 447443100
Change-Id: I7b7ca475be159dc5e2c2f4e6f1596ff7bb0438e2
2022-05-09 05:24:25 -07:00
Christian Blichmann
6cbde854d6 #Cleanup: Consistently use char* argv[] instead of char**
PiperOrigin-RevId: 444782296
Change-Id: If8e7647be28f794392675ae001abbe9b809da0ac
2022-04-27 00:43:51 -07:00
Christian Blichmann
839914d6dd cmake: Rename build options to follow Abseil naming
`BUILD_TESTING` is a CMake provided option and we should use similar naming,
just like how Abseil does it.

- `SAPI_ENABLE_TESTS` -> `SAPI_BUILD_TESTING`
- `SAPI_ENABLE_CONTRIB_TESTS` -> `SAPI_CONTRIB_BUILD_TESTING`
- `SAPI_ENABLE_EXAMPLES` -> `SAPI_BUILD_EXAMPLES`

Drive-by:
- Fix option name in GitHub action
PiperOrigin-RevId: 443305932
Change-Id: Ice2b42be1229a0f9ae7c2ceda9ce87187baf22c4
2022-04-21 01:17:39 -07:00
Christian Blichmann
c0cfeed925 cmake: Include CTest in all projects, honor BUILD_TESTING setting
Including the `CTest` modules ensures that the `BUILD_TESTING` option is
defined and automatically calls `enable_testing()` if needed. It does not
change the default or introduce any dependencies on its own.

This follows what Abseil already does in their top-level `CMakeLists.txt`.

PiperOrigin-RevId: 443305646
Change-Id: If067c17470f497437c7748aab4aab5227c26e84f
2022-04-21 01:15:34 -07:00
Christian Blichmann
0d833be608 clang_generator: Remove .isystem parameter file support
The corresponding command-line option `sapi_isystem` will be ignored for
compatibility.

PiperOrigin-RevId: 439806387
Change-Id: I8ad6d7feed2fba5fca9940281f03cfc757ada5be
2022-04-06 05:17:44 -07:00
Christian Blichmann
3f72b3cab7 bazel: Set generator version to default v1
No change in behavior.

PiperOrigin-RevId: 438766481
Change-Id: Iaf52d32856007fb83ee0032d4d5097482f45f47d
2022-04-01 01:45:50 -07:00
Christian Blichmann
4b2c730c5e Bazel: Make generator configurable
This change introduces an experimental `generator_version` attribute to the
`sapi_library()` rule. Version `1` will select the current interface
generator, which is based on libclang and Python. Setting the attribute to
version `2`, will select the newer interface generator written in C++ that
uses a full clang compiler frontend for parsing. Both emit equivalent header
output, differences in parsing and/or edge cases notwithstanding.

The default, as of now, is still the old version `1` generator.

Note: CMake allows to select the new interface generator globally by setting
`SAPI_ENABLE_GENERATOR`.
PiperOrigin-RevId: 438765013
Change-Id: I69c49a6bcf1751724edb0bce5c3b2beea2097138
2022-04-01 01:34:48 -07:00
Christian Blichmann
74bb2c35ca bazel: Add build rules for the new interface generator
This adds a workspace rule that inspects the current system first and
downloads a suitable version of LLVM/Clang from GitHub if it can't
find one. In the latter case, the necessary parts are build from source,
which can take a while (~10-15m, depending on the build machine).

In order to be found, LLVM/Clang system libraries must be version 11
or higher. On Debian/Ubuntu, install `llvm-13-dev` and `libclang-13-dev`.

The new `llvm_config.bzl` implements this logic. It is loosely based on
upstream's https://github.com/llvm/llvm-project/blob/main/utils/bazel/configure.bzl.
Note that due to the way Bazel separates local repositories, we have to
duplictate some of this code.

PiperOrigin-RevId: 438759950
Change-Id: Ia65f473b4cdef6507e3816bf09794ea10963d87a
2022-04-01 00:55:30 -07:00
Christian Blichmann
dd512a1253 clang_generator: Map C's _Bool to bool
Fixes #154.

PiperOrigin-RevId: 436167183
Change-Id: I594e5b59c5b3aba640a98c399dcb994000dbffb0
2022-03-21 04:11:21 -07:00
Christian Blichmann
6d3adc45f7 clang_generator: Improve formatting of function prototype comments
This uses the Google formatting style to format the prototype comments, with an
internal line length of 75, which accomodates the indentation in the generated
API class.

PiperOrigin-RevId: 435303665
Change-Id: I4dcdf0ed773a79ebc55ead3843f07ca8556fd985
2022-03-17 03:53:13 -07:00
Christian Blichmann
e8fe398340 clang_generator: Enable mixed header processing
This implements a custom compilation database to conditionally add the correct
language flags to the compiler frontend. Otherwise, a C header might receive
`--std=c++17` and fail.

Note: All headers are always processed in C++ mode. We expect that headers of
well-behaved C libraries contain `#ifdef __cplusplus`/`extern "C" {}` guards.
PiperOrigin-RevId: 435302048
Change-Id: Ib84e6e1f301ba434999846a012b3f8c16884648e
2022-03-17 03:41:53 -07:00
Christian Blichmann
4e71f1d0a3 clang_generator: Do not collect structs/unions if declared in another record
The enclosing type is enough to reconstruct the AST when writing the header and this
change avoids emitting the same struct twice.

PiperOrigin-RevId: 435300029
Change-Id: I34bd660db5ba5c68b64cce73ecf2f026727ac57b
2022-03-17 03:30:12 -07:00
Christian Blichmann
92ccfeae67 clang_generator: Correctly emit nested C-like structs
- Add more tests for this

PiperOrigin-RevId: 435296715
Change-Id: I7b42dbc58dc054d2565af9ad22498d98416b7af7
2022-03-17 03:13:25 -07:00
Christian Blichmann
b8579e4746 clang_generator: Use a btree_map for fixed iteration order
PiperOrigin-RevId: 435291910
Change-Id: I198247409d095183849a221af9c3be21b5bb859b
2022-03-17 02:43:08 -07:00
Christian Blichmann
d5ebb81598 clang_generator: Handle intrinsics directly
PiperOrigin-RevId: 435289903
Change-Id: I68a37bb7b25a7b77c046d00a2740aa9de2fcaa89
2022-03-17 02:31:57 -07:00
Christian Blichmann
80b325aa40 clang_generator: Emit the correct enum names in SAPI variables
This is a follow up to fa9e6e8a5c.

Drive-by:
- Replace deprecated calls to `getNameAsString()`
PiperOrigin-RevId: 435287759
Change-Id: I81d8c2f93b1ab23c781421b114779b7a241e4a7e
2022-03-17 02:19:58 -07:00
Christian Blichmann
fa9e6e8a5c clang_generator: Correctly emit typedefs with anonymous enums/structs
This change also adds some more basic testing and test utils.

PiperOrigin-RevId: 433203779
Change-Id: I57616af3719ccbc41201dc6d4b0b60ddaf70ebab
2022-03-08 07:16:54 -08:00
Copybara-Service
26a077bb3d Merge pull request #131 from DemiMarie:fix-fedora-cmake-build
PiperOrigin-RevId: 433174006
Change-Id: Icca1816a2513f4e4553ef1e671ca16bafb4fa40c
2022-03-08 04:10:29 -08:00
Christian Blichmann
eec22e8aaf Partially revert 692f026: Emit related types within wrapper namespace
There are a lot of internal users depending on the old behavior of the
libclang-based generator.

PiperOrigin-RevId: 432281224
Change-Id: If82333fc3001f52de59e57a874f28bf8815d0877
2022-03-03 14:14:52 -08:00
Demi Marie Obenour
a132d309a5 Fix the Fedora build using CMake
The build previously failed with confusing CMake errors.
2022-03-02 16:22:29 -05:00
Christian Blichmann
692f0260b3 clang_generator: Emit types outside of namespace, skip Abseil enums
PiperOrigin-RevId: 431913470
Change-Id: Ia44f6642a37501ba1630321ba1430d1bf10cf377
2022-03-02 05:17:32 -08:00
Christian Blichmann
d451478e26 Change license link to HTTPS URL
PiperOrigin-RevId: 424811734
Change-Id: If5ea692edc56ddc9c99fd478673df41c0246e9cc
2022-01-28 01:39:09 -08:00
Rebecca Chen
34551d2cec Add whitespace around keywords to satisfy the LibCST parser.
PiperOrigin-RevId: 424307703
Change-Id: I12949320efe57667624126d64cf99ec0d50edfa5
2022-01-26 03:47:51 -08:00
Christian Blichmann
3745d58587
filewrapper: _Exit instead of CHECK failing
Raw `SAPI_RAW_PCHECK` may dump core, depending on environment settings
(issue #89).
This is undesirable in the face of invalid command-line arguments.

Signed-off-by: Christian Blichmann <cblichmann@google.com>
2022-01-03 15:00:35 +01:00
Sandboxed API Team
a44e57e243 Update references to the new documentation
PiperOrigin-RevId: 416317448
Change-Id: Ic148364e012405cc34840c12428cbd912ed377ae
2021-12-14 09:03:29 -08:00
Wiktor Garbacz
bc9d7a8db6 Properly handle unsigned-by-default char types
PiperOrigin-RevId: 364774936
Change-Id: I2e411555d98cad128945949ea3eedb045af0421d
2021-03-24 04:48:16 -07:00
Sandboxed API Team
6dcef3d5c9 Integrate LLVM at llvm/llvm-project@1c762a81d2
Updates LLVM usage to match
[1c762a81d20f](https://github.com/llvm/llvm-project/commit/1c762a81d20f)

PiperOrigin-RevId: 354567452
Change-Id: I29758805e7e2030d014bbc0007f5c548f119246f
2021-01-29 11:02:56 -08:00
Christian Blichmann
75bbd0e1c1 Internal change
Only externally visible changes should be a few changed includes as well as
some formatting changes.

PiperOrigin-RevId: 353226662
Change-Id: Iebf5be13774efcbd94c5d5a17b9b27e47275b229
2021-01-22 06:01:34 -08:00
Christian Blichmann
dbaf95c724 Move utility code into sandboxed_api/util
This change should make it less confusing where utility code comes from.
Having it in two places made sense when we were debating whether to publish
Sandbox2 separately, but not any longer.

Follow-up changes will move `sandbox2/util.h` and rename the remaining
`sandbox2/util` folder.

PiperOrigin-RevId: 351601640
Change-Id: I6256845261f610e590c25e2c59851cc51da2d778
2021-01-13 09:25:52 -08:00
Sandboxed API Team
3d737e3830 Automated rollback of commit 7440916b80.
PiperOrigin-RevId: 351144593
Change-Id: Ic1401e16dcf2b6b009a9f53395929e31a68cfa77
2021-01-11 06:58:30 -08:00
Maciej Szawłowski
7440916b80 Internal tests tweak.
PiperOrigin-RevId: 351138044
Change-Id: I98e27ca5f0ea456a906c5382aa22135f21c8a1d9
2021-01-11 06:07:45 -08:00
Copybara-Service
918a409126 Merge pull request #78 from cblichmann:gen2-clang-path
PiperOrigin-RevId: 350524271
Change-Id: I46080c86c8e03f13a7468a73d752e968d5e9b56d
2021-01-07 03:19:55 -08:00
Christian Blichmann
f9b4083dee
Header generator: Enable to find latest Clang
This also allows to install `libclang1` instead of `libclang1-dev` as
one of the build dependencies on Ubuntu/Debian.

Signed-off-by: Christian Blichmann <cblichmann@google.com>
2021-01-07 11:15:13 +01:00
Wiktor Garbacz
fe79c95a50 IWYU in sapi code generator
PiperOrigin-RevId: 350134692
Change-Id: I5389df8a02f6679a3881b20b599ff5d9fc81650e
2021-01-05 07:18:41 -08:00
Christian Blichmann
07d4d02628 Build fixes for older GCC and Ubuntu
- Ubuntu 18.04 ships with GCC 7, which needs `std::move()` when returning an `absl::StatusOr<>`
- Ignore C++ AST nodes of type `cindex.TypeKind.UNEXPOSED` in Python generator
- Remove default values in `ubuntu-cmake.yml`

PiperOrigin-RevId: 347605109
Change-Id: Ibe167249ecf4ef1af1654d63c2e067fc02e5782d
2020-12-15 07:09:22 -08:00
Anton D. Kachalov
0e8d16e011 Enable shared libraries build and cross-compilation
This allows resource-constrained environments to benefit from the
space savings of dynamic linking. This is not meant to be used in
the general case.

PiperOrigin-RevId: 347398828
Change-Id: Ia634959148a31159878f48c44255dd733424a2b8
2020-12-14 09:16:14 -08:00
Wiktor Garbacz
742fafa433 Prefixed (unique) names for executables in cmake
PiperOrigin-RevId: 347335966
Change-Id: Ic8cc22b882fa489d37b636406a1a5fe51745d808
2020-12-14 01:13:31 -08:00
Christian Blichmann
728355da87 Emit non-type template args as part of forward decls
This change allows us to emit forward declarations to classes that are
templated. For headers generated by the proto compiler this is sometimes
necessary.

Note:
- This will only emit types for a single level of template instantiations.
  That is, template template arguments are not supported.
- Typedefs only occurring in template arguments will be fully desugared
  and thus will not be available under their aliased name in the generated
  API code. This is consistent with the Python based generator (which
  does not emit these at all and relies on text extraction).

Signed-off-by: Christian Blichmann <cblichmann@google.com>
2020-10-28 16:48:04 +01:00
Sandboxed API Team
ea379ef4d6 Cleans up statusor.h includes.
PiperOrigin-RevId: 339050213
Change-Id: Iea5747f907b294503cdb37e1c25cf787c7e83dcf
2020-10-26 09:08:41 -07:00