TODO: Replace the references to local paths.
oilgen (the basis of Ahead Of Time compilation for OIL) has never been passed
through our large test suite and has instead had more focused testing and large
examples. When consuming DWARF information in a similar fashion to JIT OIL this
was okay, but with the new Clang AST based mechanism it means we have very
little coverage.
This change adds an oilgen test for every test case that has an oil test.
Relying on the build system to create the test target as before would make it
difficult to have failing tests, so we move the build into the integration test
runner. This involves:
1. Writing the input source to a file.
2. Consuming it with oilgen to get the implementation object file.
3. Compiling the input source and linking it with this file.
4. Running the newly created target.
This approach can give the full error message at any stage that fails and will
fail the test appropriately. The downside is the build system integration is
more difficult, as we need the correct compiler flags for the target and to use
the correct compiler. It would be very tricky to replicate this in a build
system that's not CMake, so we will likely only run these tests in open source.
Test plan:
- CI
Add the option to calculate total size (inclusive size) by wrapping the
existing iterator. This change provides a new iterator, `SizedIterator`, which
wraps an existing iterator and adds a new field `size` to the output element.
This is achieved with a two pass algorithm on the existing iterator:
1. Gather metadata for each element. This includes the total size up until that
element and the range of elements that should be included in the size.
2. Return the result from the underlying iterator with the additional
field.
This algorithm is `O(N)` time on the number of elements in the iterator and
`O(N)` time, storing 16 bytes per element. This isn't super expensive but is a
lot more than the current algorithm which requires close to constant space.
Because of this I've implemented it as a wrapper on the iterator rather than on
by default, though it is now on in every one of our integration test cases.
Test plan:
- Added to the integration tests for full coverage.
Remove the CodeGen v1 sections of the CI config because both OID and OIL use
CodeGen v2.
We were missing running any test that wasn't `Oi{d,l}Integration.*` before.
This now runs the unit tests again and requires a minor fix to one unit test.
Test plan:
- CI
We previously moved container identification later in CodeGen in order
to preserve information for AlignmentCalc.
However, Flattener needs to know if a class is a container in order to
apply its special handling for this case.
This new approach moves container identification in front of Flattener,
but has Container own a type node, representing its layout. This
underlying type node can be used for calculating a container's
alignment in a later pass.
They must not appear in the final generated code as we'd end up with
invalid types with void members, e.g.:
struct Foo {
int a;
void myIncompleteMember;
int c;
};
Removing them from the type graph early also ensures that padding is
calculated correctly.
Summary:
Remove the now useless `handler` and adds the `traversal_func` and `processor`
entries for `std::list`. This type is a bit weird as most of our sequential
containers don't have any overhead on storing the element. I went for the same
approach we take for maps where we have a shared `[]` element covering the map
overhead and below that a `key` & `value`. As we only have a single element
under it which doesn't have a logical name I went for `*`.
Closes#315.
Test Plan:
- CI
- Copied the relevant `std::vector` tests and updated the existing one.
Summary:
Currently there are two features between CodeGen v2 (TypeGraph) and TreeBuilder
v2. These are TypedDataSegment and TreeBuilderTypeChecking. Each of these
features currently has a full set of tests run in the CI and each have specific
exclusions.
Collapse these features into TreeBuilder v2. This allows for significantly
simplified testing as any OIL tests run under TreeBuilder v2 and any OID tests
run under TreeBuilder v1.
The reasoning behind this is I no longer intend to partially roll out this
feature. Full TreeBuilder v2 applies different conditions to containers than
the intermediate states, and writing these only to have them never deployed is
a waste of time.
Test Plan:
- it builds
- CI
Not all containers have 8-byte alignment, so if we want to avoid lots of
manual logic for calculating container alignment on a case-by-case
basis, we must calculate alignment from the member variables before the
Class nodes have been replaced by Container nodes.
Leave it to the new mutator pass IdentifyContainers to replace Class
nodes with Container nodes where appropriate.
This will allow us to run passes over the type graph before identifying
containers, and therefore before we have lost information about the
internal details of the container (e.g. alignment of member variables).
For the containers which are allowed to be declared with incomplete
types, it is only the contained types which are allowed to be
incomplete. Other template parameters (e.g. allocators) must always be
defined before use.
Summary:
Adds the option for required features to container definitions. These cause the container not to be passed to `DrgnParser` if that feature is not enabled during code generation. The thrift isset type does not currently work with `tree-builder-v2` and only provides benefit with `capture-thrift-isset`. This change makes sure the container is ignored if it won't be useful, allowing code generation under `tree-builder-v2`.
Test Plan: - CI
Differential Revision: D49960512
Pulled By: JakeHillion
Summary:
Previously OID/OIL required exactly one configuration file. This change makes it so you can supply 0 or more configuration files. 0 is useful if you have pre-generated the cache or use some sort of remote generation system. 1 is useful for the common case, where you have a configuration file that describes your entire source and use just that. More are useful if you have supplemental bits of config you wish to apply/override - see the changes to the integration test framework where we do exactly this.
Test Plan:
This isn't super well tested. It works for the test cases which add features via the config or enable `codegen.ignore`.
- CI
Reviewed By: ajor
Differential Revision: D49758032
Pulled By: JakeHillion
Lots of places rely on reference stability of ContainerInfo objects
(CodeGen's deduplication, Container nodes' containerInfo_ member).
In the key capture work, we need to be able to append to this list,
which would invalidate references before this change.
We only want to do the extra work if it's explicitly requested.
chaseRawPointers is already explicitly requested whenever it's needed
and readEnumValues currently isn't needed at all.
Summary:
Update to clang-15 compiler and libraries as clang-12 is ancient.
The changes to oilgen are necessary because the new internal toolchain is being more picky about linking PIC to PIC. In certain modes we build with PIC, but try to link a non-PIC oilgen artifact. Add the ability to build the oilgen artifacts with PIC which sorts this.
Reviewed By: ttreyer
Differential Revision: D46220858
