Summary:
Pull Request resolved: https://github.com/facebookexperimental/object-introspection/pull/507
OI is currently completely incompatible with anything except `x86_64`. Changing that generally is a big effort, but `oilgen` should work on other architectures. Begin adding some support for `aarch64` architecture.
This change sets up a file structure for architecture support. It pulls the 2 functions needed to make `Descs.{h,cpp}` architecture agnostic into architecture specific files for `x86_64` and `aarch64`. This enables `oilgen` (the binary) to build. At this stage `oilgen` is unable to generate working code for `aarch64`, but at least this is a step in the right direction.
Differential Revision: D61661524
OI's build is challenging and has often been a problem for the Open Source
community. It requires an extremely specific set of dependencies that are very
hard to achieve on most systems. There are frequent breakages, like when
updating to CentOS Stream 9, or when trying to update the CI's clang from
clang-12 to clang-15 - OI requires the clang libraries to be version 15 but
can't be compiled with it on the CI!
This changes provides a mostly working build environment with `nix`. This
environment is pinned to a specific nixpkgs revision using `flake.lock`, and
only updates when we explicitly tell it to.
Summary of changes:
- Update CMakeLists.txt required version to 3.24. This allows specifying
`FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages.
This is available on most up to date distros (3.30.2 is current).
- Extends `flake.nix` to be able to build OI. Adds instructions for building
and developing OI using `nix`.
- Partially runs the tests in GitHub Actions. A huge amount must be excluded
because of incompatibilites between the clangStdenv compiler and drgn. We
have similar, though fewer, issues when building with the clang-12/libstdcxx
mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible.
- Updates CircleCI to build CMake from source as we don't have a newer image
available. Also add some newly found dependencies (not sure how it was
working without them before).
Test plan:
This change requires less testing than previous build related changes because
it deprecates most of the build types.
- The internal BUCK build is unaffected. No special testing.
- The semi-internal CMake build is gone. Use Nix.
- The Nix build for clang-15 and some tests are continuously tested in GitHub
actions.
- Tested the set of Nix commands in the README. All work except the one that
points to GitHub as this must be merged first.
- The existing CircleCI runs on Ubuntu 20.04 are maintained.
- Unable to test the new `test-report.yml` as it must be merged due to the
permissions it needs. Will follow up with testing after this is merged. See:
https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories
The list of exclusions for GitHub Actions/nix testing is currently very long, I
think 29% of the tests. This should be stable and reproducible though, and
likely needs deep changes to OI to fix. That's why fixes are excluded from this
PR. It's all to do with the forked drgn not being able to parse clang's newer
DWARF output, and can't be fixed by rolling back as we required a relatively
new libcxx.
Summary:
OI was previously using `std::regex_match` to match container names. This was bad because `libstdc++`'s implementation of regex is awful. In the case of limited inlining it was causing a stack overflow when running CodeGen for large types (I think types with large names but I never got to the bottom of it).
Replace this with the competent `boost::regex_match` that we already have a dependency on.
Reviewed By: ajor
Differential Revision: D53002752
Add the option to calculate total size (inclusive size) by wrapping the
existing iterator. This change provides a new iterator, `SizedIterator`, which
wraps an existing iterator and adds a new field `size` to the output element.
This is achieved with a two pass algorithm on the existing iterator:
1. Gather metadata for each element. This includes the total size up until that
element and the range of elements that should be included in the size.
2. Return the result from the underlying iterator with the additional
field.
This algorithm is `O(N)` time on the number of elements in the iterator and
`O(N)` time, storing 16 bytes per element. This isn't super expensive but is a
lot more than the current algorithm which requires close to constant space.
Because of this I've implemented it as a wrapper on the iterator rather than on
by default, though it is now on in every one of our integration test cases.
Test plan:
- Added to the integration tests for full coverage.
Summary:
Adds the option for required features to container definitions. These cause the container not to be passed to `DrgnParser` if that feature is not enabled during code generation. The thrift isset type does not currently work with `tree-builder-v2` and only provides benefit with `capture-thrift-isset`. This change makes sure the container is ignored if it won't be useful, allowing code generation under `tree-builder-v2`.
Test Plan: - CI
Differential Revision: D49960512
Pulled By: JakeHillion
