Commit Graph

83 Commits

Author SHA1 Message Date
Jake Hillion
db289c1a1a tbv2: use std::decay_t with smart pointers
CodeGen v2 permits template parameters to be qualified. This means that if we
call `make_field` with a template parameter it will be qualified. However, we
don't qualify the types when generating meta functions such as `NameProvider`
and `TypeHandler`. This means these qualified types don't match up with the
expected type.

Use `std::decay_t` when forwarding the type to `NameProvider` and `TypeHandler`
so they're always the base type that they were generated with. Most of this is
covered by `make_field`, but there are direct references to `TypeHandler<Ctx,
T>` in a lot of `TypeHandler::type` fields. Fix the problematic types manually
for now, there may need to be a better solution with meta functions for this in
the future.

Test Plan:
- CI
- Added a test for `std::unique_ptr<const uint64_t>` to exercise this. Failed
  before, passes after.
- Added a test for `std::unique_ptr<const std::vector<uint64_t>>` to test a
  non-primitive type. Failed before, passes after.
2024-01-31 17:27:04 +00:00
Janeczko Jakub
cc401c1e5b
fix string type sso computation (#469)
* fix string type sso computation

* fix rest of sbo/sso calculation

* make placement of uintptr_t cast consistent

* separate check-inline into function
2024-01-31 11:19:24 +00:00
Thierry Treyer
1c65ed8ec7 Implement Container V2 for std::deque 2024-01-23 19:43:56 +01:00
Jake Hillion
4c047b5f91 tbv2: add is_primitive to output
C++ has a concept of Primitive which holds in the type graph. However we don't
currently expose this information to the end user. Expose this from the OIL
iterator to allow future features like primitive rollups.

This affects containers like maps which have a fake `[]` element with no type.
They use this to group together the key/value in a map and to account for any
per element storage overhead. Currently the decision is to make the fake `[]`
element a primitive if all of its children are primitives. This allows for more
effective primitive rollups if that is implemented. This implementation detail
may be changed in future.

Test Plan:
- CI
- Updated simple tests.
2024-01-16 11:14:13 +00:00
Jake Hillion
0e72947786 tbv2: add support for std::reference_wrapper
Closes #307

Test plan:
- CI
- Updated and enabled tests.
2024-01-15 16:20:26 +00:00
Jake Hillion
08e2faa90e tbv2: correctly account for list overhead
`std::list` has per element overhead for the individual heap allocations. This
was already calculated in the container implementation but not used. Allocate
the overhead of each element in the `std::list` to the `std::list` itself as
with other sequential containers.

Test Plan:
- CI
- Updated test cases
2024-01-11 15:41:48 +00:00
Jake Hillion
cbeafba9bb tbv2: fix type names for std::optional
Type names of optional elements were accidentally left as todo. Update
`std::optional` to use `make_field` and correctly name its elements.

Test Plan:
- CI
- Updated the integration tests to test the names.
2024-01-09 15:09:24 +00:00
Jake Hillion
db93feb180 incomplete: name type in compiler errors
Summary:

We have a good type representation in the Type Graph of an incomplete type and
the underlying type that represents. However, this incomplete type still ends
up in the generated code as `void` which loses information. For example, a
container that can't contain void may fail to compile because it was
initialised with `void` but really its because the type it was supposed to be
initialised with (say, `Foo`) had incomplete debug information.

This change identifies that a type is incomplete in the output by generating it
as an incomplete type `struct Incomplete<struct Foo>`. This allows us to name
the type correctly in the TreeBuilder output and filter for incomplete types,
as well as getting appropriate compiler errors if it mustn't be incomplete.

Test Plan:
- CI
- Added a unit test to namegen.
- Enabled and added an extra pointers_incomplete test.

This change is tricky to test because it isn't really user visible. The types
still use their `inputName` which is unchanged in any successful output - this
change is used so the compiler fails with a more detailed error.
2024-01-09 15:08:25 +00:00
Jake Hillion
5a2ca8b059 tbv2: implement folly::IOBuf
folly::IOBuf does not have TreeBuilder v2 container support. Add it.

The implementation is a direct clone of v1. It still lacks tests.

Test Plan:
- It codegens on a prod type.
- No runtime testing... Bad form, I know.
- Issue created to add integration tests: https://github.com/facebookexperimental/object-introspection/issues/436
2023-12-20 16:13:50 +00:00
Jake Hillion
55989a9156 oilgen: migrate to source parsing (#421)
Summary:
oilgen: migrate to source parsing

Using debug information generated from partial source (that is, not the final
binary) has been insufficient to generally generate OIL code.

A particular example is pointers to templates:
```cpp
#include <oi/oi.h>
template <typename T>
struct Foo {
  T t;
};
template <typename T>
struct Bar {
  Foo<T>& f;
};
void foo(const Bar<int>& b) {
  oi::introspect(b);
}
```

The pointer/reference to `Foo<int>` appears in DWARF with
`DW_AT_declaration(true)` because it could be specialised before its usage.
However, with OIL, we are creating an implicit usage site in the
`oi::introspect` call that the compiler is unable to see.

This change reworks OILGen to work from a Clang command line rather than debug
information. We setup and run a compiler on the source, giving us access to an
AST and Semantic Analyser. We then:
- Find the `oi::introspect` template.
- Iterate through each of its callsites for their type.
- Run `ClangTypeParser::parse` on each type.
- Run codegen.
- Compile into an object file.

Having access to the semantic analyser allows us to forcefully complete a type,
as it would be if it was used in the initial code.


Test Plan:
hope

`buck2 run fbcode//mode/opt fbcode//object-introspection/oil/examples/compile-time:compile-time`

Reviewed By: tyroguru

Differential Revision: D51854477

Pulled By: JakeHillion
2023-12-19 13:26:25 -08:00
Jake Hillion
b117150f83 tbv2: add dynamic context passed through all functions (#410)
Summary:
tbv2: add dynamic context passed through all functions

Previously for we had some shared state between all requests, noticeably the
pointers set. This change adds a by reference value to all requests which can
hold additional mutable state. The pointers set is moved into this mutable
state for OIL, which means each concurrent request will have its own pointer
set. Doing things this way allows more features to be added in the future
without such a big code modification.

Closes https://github.com/facebookexperimental/object-introspection/issues/404

Pull Request resolved: https://github.com/facebookexperimental/object-introspection/pull/410

Test Plan: - CI

Differential Revision: D51394035

Pulled By: JakeHillion

fbshipit-source-id: 55d2ba9b5e056148a29dc821020cfc3d94e5175a
2023-11-16 08:03:32 -08:00
Jake Hillion
592e182e0f tbv2: replace DB template param with Ctx (#409)
Summary:
tbv2: replace DB template param with Ctx

TreeBuilder v2 adds a DB template parameter to every function. This is used as
part of the static type to decide what type of DataBuffer is being used:
currently `BackInserterDataBuffer<std::vector<uint8_t>>` for OIL and it would
be `DataSegmentDataBuffer` for OID.

This change replaces the `DB` template parameter with a more general `Ctx`. Due
to issues with dependent naming it also adds a `using DB` to each `TypeHandler`
which has the same function as before. This allows us to add more "static
context" (typedefs and constants) to functions without changing this signature
again, because changing the signature of everything is a massive pain.

Currently this change achieves nothing because Ctx contains only DB in a static
wrapper. In the next change I'm going to pass a reference of type Ctx around to
add a "dynamic context" to invocations which will contain the pointer array. In
future we'll then be able to add either static or dynamic context without any
signature adjustments.


Test Plan:
- CI

---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebookexperimental/object-introspection/pull/409).
* https://github.com/facebookexperimental/object-introspection/issues/410
* __->__ https://github.com/facebookexperimental/object-introspection/issues/409

Reviewed By: ajor

Differential Revision: D51352092

Pulled By: JakeHillion
2023-11-15 11:52:17 -08:00
Jake Hillion
f9cb0115e1 types: remove now unused handlers
Summary:

Handlers were added in an intermediate form of tbv2 but those intermediate
forms have now been removed. Remove all the handlers to make grepping/find and
replaces easier across the types.

Test Plan:

- CI
2023-11-15 14:27:07 +00:00
Jake Hillion
e9d8df0ca4 oil: add support for std::list
Summary:

Remove the now useless `handler` and adds the `traversal_func` and `processor`
entries for `std::list`. This type is a bit weird as most of our sequential
containers don't have any overhead on storing the element. I went for the same
approach we take for maps where we have a shared `[]` element covering the map
overhead and below that a `key` & `value`. As we only have a single element
under it which doesn't have a logical name I went for `*`.

Closes #315.

Test Plan:
- CI
- Copied the relevant `std::vector` tests and updated the existing one.
2023-11-15 14:14:02 +00:00
Jake Hillion
3871d92abb collapse TreeBuilderV2 features
Summary:

Currently there are two features between CodeGen v2 (TypeGraph) and TreeBuilder
v2. These are TypedDataSegment and TreeBuilderTypeChecking. Each of these
features currently has a full set of tests run in the CI and each have specific
exclusions.

Collapse these features into TreeBuilder v2. This allows for significantly
simplified testing as any OIL tests run under TreeBuilder v2 and any OID tests
run under TreeBuilder v1.

The reasoning behind this is I no longer intend to partially roll out this
feature. Full TreeBuilder v2 applies different conditions to containers than
the intermediate states, and writing these only to have them never deployed is
a waste of time.

Test Plan:
- it builds
- CI
2023-11-13 19:43:03 +00:00
Jake Hillion
5daed4cc72 tbv2: add a repeat instruction 2023-10-25 16:05:12 +01:00
Jake Hillion
4c6f232766 containers: add required features (#374)
Summary:
Adds the option for required features to container definitions. These cause the container not to be passed to `DrgnParser` if that feature is not enabled during code generation. The thrift isset type does not currently work with `tree-builder-v2` and only provides benefit with `capture-thrift-isset`. This change makes sure the container is ignored if it won't be useful, allowing code generation under `tree-builder-v2`.


Test Plan: - CI

Differential Revision: D49960512

Pulled By: JakeHillion
2023-10-09 17:50:39 -04:00
Thierry Treyer
593a1412d8 Fix exclusive_size for F14NodeMap 2023-10-09 10:41:34 -06:00
Thierry Treyer
7b5baca3b0 Fix exclusive_size for F14FastMap 2023-10-09 10:41:34 -06:00
Thierry Treyer
9ce7bd5fad Fix exclusive_size for F14VectorMap 2023-10-09 10:41:34 -06:00
Thierry Treyer
2d3709da65 Fix exclusive_size for F14ValueMap 2023-10-09 10:41:34 -06:00
Thierry Treyer
37b25bb278 Fix exclusive_size for F14NodeSet 2023-10-09 10:41:34 -06:00
Thierry Treyer
7836134343 Fix exclusive_size for F14VectorSet 2023-10-09 10:41:34 -06:00
Thierry Treyer
cc12264847 Fix exclusive_size for F14FastSet 2023-10-09 10:41:34 -06:00
Thierry Treyer
76adc64723 Fix exclusive_size for F14ValueSet 2023-10-09 10:41:34 -06:00
Alastair Robertson
35d45c2b4f folly::sorted_vector_map: Add key-capture support 2023-09-29 11:21:41 -07:00
Alastair Robertson
5572e294f7 CaptureKeys: Add support for std::map and std::unordered_map 2023-09-28 19:42:49 -07:00
Alastair Robertson
4afa2ff190 CodeGen: Add support for key capture 2023-09-28 19:42:49 -07:00
Alastair Robertson
1f66ef064a ContainerInfo: Read "extra" codegen field 2023-09-28 19:42:49 -07:00
Jake Hillion
edb7bf5f3f Implement Container V2 for folly::sorted_vector_set 2023-09-27 19:06:20 -07:00
Jake Hillion
8bad704d9c Implement Container V2 for fbstring 2023-09-27 18:16:36 -07:00
Jake Hillion
d71307cb43 oil: change std::stack reference to a std::function (#345)
Summary:

Previously on large types OIL would have problems with corrupting the `std::stack<exporter::inst::Inst>` that is passed to the processors. This change hides the implementation of the stack from the processors by wrapping the call to emplace in a `std::function` written by the non-generated code, which solves the test case I've seen for this crashing. It also allows us to easily change the stack implementation in future - I plan to change it to a `std::stack<T, std::vector<T>>` in a follow up.

Reviewed By: tyroguru

Differential Revision: D49273116
2023-09-14 16:57:45 +01:00
Thierry Treyer
331c47705c Implement Container V2 for folly::small_vector 2023-09-08 16:34:34 +02:00
Thierry Treyer
465303cb99 Implement new container for F14ValueMap 2023-09-08 16:31:36 +02:00
Thierry Treyer
5d3aeec86d Implement new container for F14ValueSet 2023-09-08 16:31:36 +02:00
Thierry Treyer
0df6879f66 Implement Container V2 for F14FastSet 2023-09-08 16:31:36 +02:00
Thierry Treyer
a3d14d227f Implement Container V2 for F14NodeSet 2023-09-08 16:31:36 +02:00
Thierry Treyer
65fecf314a Implement Container V2 for F14VectorSet 2023-09-08 16:31:36 +02:00
Thierry Treyer
f8a28b41ef Document lack of byte-accurate reporting for F14Map 2023-09-07 18:47:09 +02:00
Thierry Treyer
b327cf956b Implement Container V2 for F14FastMap 2023-09-07 18:47:09 +02:00
Thierry Treyer
b8e2770e2c Implement Container V2 for F14NodeMap 2023-09-07 18:47:09 +02:00
Thierry Treyer
d2a40b4f45 Implement Container V2 for F14VectorMap 2023-09-07 18:47:09 +02:00
Thierry Treyer
a5284549da Implement Container V2 for std::shared_ptr 2023-08-31 20:07:32 +02:00
Thierry Treyer
344023239b Add OILv2 tests for std::optional 2023-08-31 20:07:32 +02:00
Jake Hillion
a95d6f2187 add support for std::optional in treebuilder v2 2023-08-31 20:07:32 +02:00
Thierry Treyer
2699b4a665 Implement Container V2 for std::unique_ptr 2023-08-31 20:07:32 +02:00
Thierry Treyer
deb7dcdf1c Implement Container V2 for std::unordered_multiset 2023-08-30 18:45:00 +02:00
Thierry Treyer
54e01cfe27 Implement Container V2 for std::unordered_multimap 2023-08-30 18:45:00 +02:00
Thierry Treyer
7bdf72ec58 Implement Container V2 for std::unordered_map 2023-08-30 18:45:00 +02:00
Thierry Treyer
c1f322dc42 Implement Container V2 for std::unordered_set 2023-08-30 18:45:00 +02:00