ClangTypeParser has emitted a duplicate type for `std::allocatr<int8_t>`.
Rather than fixing this, add the same check the compiler will do for the
duplicate templates that `addNames` emits. That is, `template<>
NameProvider<Foo>` will collide if `Foo` is used twice. We can do this by
adding a set of these strings for now. If this shows up regularly it will
likely make sense to deduplicate the type graph with a deduplication pass.
Test plan:
- Fixes the issue in prod. This change is quite logical.
Summary:
oilgen: migrate to source parsing
Using debug information generated from partial source (that is, not the final
binary) has been insufficient to generally generate OIL code.
A particular example is pointers to templates:
```cpp
#include <oi/oi.h>
template <typename T>
struct Foo {
T t;
};
template <typename T>
struct Bar {
Foo<T>& f;
};
void foo(const Bar<int>& b) {
oi::introspect(b);
}
```
The pointer/reference to `Foo<int>` appears in DWARF with
`DW_AT_declaration(true)` because it could be specialised before its usage.
However, with OIL, we are creating an implicit usage site in the
`oi::introspect` call that the compiler is unable to see.
This change reworks OILGen to work from a Clang command line rather than debug
information. We setup and run a compiler on the source, giving us access to an
AST and Semantic Analyser. We then:
- Find the `oi::introspect` template.
- Iterate through each of its callsites for their type.
- Run `ClangTypeParser::parse` on each type.
- Run codegen.
- Compile into an object file.
Having access to the semantic analyser allows us to forcefully complete a type,
as it would be if it was used in the initial code.
Test Plan:
hope
`buck2 run fbcode//mode/opt fbcode//object-introspection/oil/examples/compile-time:compile-time`
Reviewed By: tyroguru
Differential Revision: D51854477
Pulled By: JakeHillion
Currently we rely on `SymbolService::getTypeName` for getting the hash that's
included in the generated function's name. The value of this must stay the same
to match with the value expected by OIDebugger - changing it causes failure to
relocate when attaching with OID and JIT OIL.
Calculate this name in the `codegenFromDrgn` method and pass it through where
appropriate rather than passing the `drgn_type` itself through.
We don't need to name the type like that when using AoT OIL. Let's
hash the linkage name instead as that is more unique.
Test Plan:
- CI
We previously moved container identification later in CodeGen in order
to preserve information for AlignmentCalc.
However, Flattener needs to know if a class is a container in order to
apply its special handling for this case.
This new approach moves container identification in front of Flattener,
but has Container own a type node, representing its layout. This
underlying type node can be used for calculating a container's
alignment in a later pass.
Summary:
tbv2: add dynamic context passed through all functions
Previously for we had some shared state between all requests, noticeably the
pointers set. This change adds a by reference value to all requests which can
hold additional mutable state. The pointers set is moved into this mutable
state for OIL, which means each concurrent request will have its own pointer
set. Doing things this way allows more features to be added in the future
without such a big code modification.
Closes https://github.com/facebookexperimental/object-introspection/issues/404
Pull Request resolved: https://github.com/facebookexperimental/object-introspection/pull/410
Test Plan: - CI
Differential Revision: D51394035
Pulled By: JakeHillion
fbshipit-source-id: 55d2ba9b5e056148a29dc821020cfc3d94e5175a
Summary:
tbv2: replace DB template param with Ctx
TreeBuilder v2 adds a DB template parameter to every function. This is used as
part of the static type to decide what type of DataBuffer is being used:
currently `BackInserterDataBuffer<std::vector<uint8_t>>` for OIL and it would
be `DataSegmentDataBuffer` for OID.
This change replaces the `DB` template parameter with a more general `Ctx`. Due
to issues with dependent naming it also adds a `using DB` to each `TypeHandler`
which has the same function as before. This allows us to add more "static
context" (typedefs and constants) to functions without changing this signature
again, because changing the signature of everything is a massive pain.
Currently this change achieves nothing because Ctx contains only DB in a static
wrapper. In the next change I'm going to pass a reference of type Ctx around to
add a "dynamic context" to invocations which will contain the pointer array. In
future we'll then be able to add either static or dynamic context without any
signature adjustments.
Test Plan:
- CI
---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebookexperimental/object-introspection/pull/409).
* https://github.com/facebookexperimental/object-introspection/issues/410
* __->__ https://github.com/facebookexperimental/object-introspection/issues/409
Reviewed By: ajor
Differential Revision: D51352092
Pulled By: JakeHillion
Summary:
Currently there are two features between CodeGen v2 (TypeGraph) and TreeBuilder
v2. These are TypedDataSegment and TreeBuilderTypeChecking. Each of these
features currently has a full set of tests run in the CI and each have specific
exclusions.
Collapse these features into TreeBuilder v2. This allows for significantly
simplified testing as any OIL tests run under TreeBuilder v2 and any OID tests
run under TreeBuilder v1.
The reasoning behind this is I no longer intend to partially roll out this
feature. Full TreeBuilder v2 applies different conditions to containers than
the intermediate states, and writing these only to have them never deployed is
a waste of time.
Test Plan:
- it builds
- CI
Not all containers have 8-byte alignment, so if we want to avoid lots of
manual logic for calculating container alignment on a case-by-case
basis, we must calculate alignment from the member variables before the
Class nodes have been replaced by Container nodes.
Leave it to the new mutator pass IdentifyContainers to replace Class
nodes with Container nodes where appropriate.
This will allow us to run passes over the type graph before identifying
containers, and therefore before we have lost information about the
internal details of the container (e.g. alignment of member variables).
Summary:
Adds the option for required features to container definitions. These cause the container not to be passed to `DrgnParser` if that feature is not enabled during code generation. The thrift isset type does not currently work with `tree-builder-v2` and only provides benefit with `capture-thrift-isset`. This change makes sure the container is ignored if it won't be useful, allowing code generation under `tree-builder-v2`.
Test Plan: - CI
Differential Revision: D49960512
Pulled By: JakeHillion
Lots of places rely on reference stability of ContainerInfo objects
(CodeGen's deduplication, Container nodes' containerInfo_ member).
In the key capture work, we need to be able to append to this list,
which would invalidate references before this change.
Summary:
Previously on large types OIL would have problems with corrupting the `std::stack<exporter::inst::Inst>` that is passed to the processors. This change hides the implementation of the stack from the processors by wrapping the call to emplace in a `std::function` written by the non-generated code, which solves the test case I've seen for this crashing. It also allows us to easily change the stack implementation in future - I plan to change it to a `std::stack<T, std::vector<T>>` in a follow up.
Reviewed By: tyroguru
Differential Revision: D49273116
Summary:
Update `OIGenerator` and out BUCK stuff for compile time OIL with OIL v2. Main things:
- Switch `OIGenerator` from the `getObjectSize` call to the new `introspect` call.
- Switch from looking at template parameters to looking at function parameters, as this was exposing a bug in our elfutils/drgn and this way it's the same as OID.
- Migrate `OIGenerator` to CodeGen v2 and update CodeGen v2 to accept a linkage name.
- Update the compile time example to be the same as the JIT example, using the new interface and the JSON exporter.
- Clean up the `ObjectIntrospection.h` header.
Differential Revision: D48687728
fbshipit-source-id: 2c3c041fd1b6499c5e02eb5e2082a977bfa529d7
Types within containers were previously named TODO. This sorts it out so
they're named as their most resolved type. The current implementation
skips Typedef names.
This extracts the compatibility logic from AddPadding, which allows for it to be
simplified and will make it easier to extend and eventually remove in the
future. No functional changes.
This lets us remove fields from types when they are no longer needed,
speeding up later passes.
A secondary benefit of pruning unused types means that we sometimes
remove types for which we can't generate correct C++ code. This can
allow us to CodeGen for complex types which reference these broken types
without actually requiring them (e.g. as template parameters).
Add a new feature flag "prune-type-graph" to control this pass. It makes
sense to prune most of the time, but for testing CodeGen functionality
on a wider range of types, it will be useful to have the option to not
prune.
In general, we can't tell which member is active in a union so it is not
safe to try and measure any of them.
Explicitly set the alignment of unions (and structs/classes) in CodeGen
if it is available, as the C++ compiler can no longer infer it from the
members.
Also reshuffle CodeGen's passes to fix an alignment bug with removed
members.
Change RemoveMembers to actually remove members instead of replacing
them with padding. AddPadding must be run afterwards to fill in the
gaps.
This will eventually be used to enable running with Tree Builder v2.
For now, when it is disabled it puts CodeGen v2 into compatibility mode,
disabling features which weren't present in CodeGen v1 so that its
output can be understood by Tree Builder v1.
As we now store ContainerInfo objects in OICodeGen::Config, we can not
copy it any more. Change all places that took copies to take const
references instead.
The copy in OICodeGen modified membersToStub, the contents of which form
part of OICache's hash. However, as OICache also previously had its own
copy, it would not have been OICodeGen's modifications.
- Change member and parent offsets to work in bits, not bytes
- Printer still displays offsets in bytes, with decimals when using
bitfields
- AddPadding: Don't pad bitfields
- CodeGen: Emit code for bitfields
For std::vector and std::list, template parameters are not required to
be defined before they can be used. Delay sorting them until the end.
Also fix a CodeGen bug where we were defining typedefs in the middle of
the forward declarations. They only need to be defined when other types
are defined.
