Commit Graph

25 Commits

Author SHA1 Message Date
Jake Hillion
40af807d8b tbv2: support capture-thrift-isset
Support the capture-thrift-isset feature with TreeBuilder-v2. Fairly minor
changes here except the type of the Enum in a template parameter now matters.

We follow the previous behaviour of capturing a value for each field in a
struct that has an `isset_bitset`. This value is a VarInt captured before the
C++ contents of the member. It has 3 values: 0 (not set), 1 (set), and 2
(unavailable). These are handled by the processor and represented in the output
as `false`, `true`, and `std::nullopt_t` respectively.

Changes:
- Add a simple Thrift isset processor before any fields that have Thrift isset.
- Store the fully qualified names of enum types in DrgnParser - it already
  worked out this information anyway for naming the values and this is
  consistent with classes.
- Forward all enum template parameters under their input name under the
  assumption that they will all be policy type things like `IssetBitsetOption`.
  This could turn out to be wrong.

Test plan:
- CI (doesn't test thrift changes but covers other regressions)
- Updated Thrift enum tests for new format.
- `FILTER='OilIntegration.*' make test` - Thrift tests failed before, succeed
  after.
2024-01-16 19:09:46 +00:00
Jake Hillion
4c047b5f91 tbv2: add is_primitive to output
C++ has a concept of Primitive which holds in the type graph. However we don't
currently expose this information to the end user. Expose this from the OIL
iterator to allow future features like primitive rollups.

This affects containers like maps which have a fake `[]` element with no type.
They use this to group together the key/value in a map and to account for any
per element storage overhead. Currently the decision is to make the fake `[]`
element a primitive if all of its children are primitives. This allows for more
effective primitive rollups if that is implemented. This implementation detail
may be changed in future.

Test Plan:
- CI
- Updated simple tests.
2024-01-16 11:14:13 +00:00
Jake Hillion
71e734b120 tbv2: calculate total memory footprint
Add the option to calculate total size (inclusive size) by wrapping the
existing iterator. This change provides a new iterator, `SizedIterator`, which
wraps an existing iterator and adds a new field `size` to the output element.

This is achieved with a two pass algorithm on the existing iterator:
1. Gather metadata for each element. This includes the total size up until that
   element and the range of elements that should be included in the size.
2. Return the result from the underlying iterator with the additional
   field.

This algorithm is `O(N)` time on the number of elements in the iterator and
`O(N)` time, storing 16 bytes per element. This isn't super expensive but is a
lot more than the current algorithm which requires close to constant space.
Because of this I've implemented it as a wrapper on the iterator rather than on
by default, though it is now on in every one of our integration test cases.

Test plan:
- Added to the integration tests for full coverage.
2024-01-04 09:21:35 +00:00
Jake Hillion
35afd15ee4 capture_keys: store dynamic type path components more efficiently 2023-12-14 16:05:33 +00:00
Jake Hillion
393f8aab42 clang-format: disable bin packing
Bin packing often makes code hard to read. Disable it entirely.

Test plan:
- CI
2023-11-13 18:19:53 +00:00
Jake Hillion
4563cbe713 tbv2: improve equality for iterator 2023-10-25 17:05:42 +01:00
Jake Hillion
5daed4cc72 tbv2: add a repeat instruction 2023-10-25 16:05:12 +01:00
Jake Hillion
6f3580143e tbv2: change instruction stack from deque to vector 2023-10-25 15:56:01 +01:00
Jake Hillion
fcb50df611 oil: make AoT introspect noinline 2023-10-25 13:23:47 +01:00
Jake Hillion
57c1a76228 oil: add fallible build option
Summary: Add a second form for OIL AoT compilation which returns a `std::optional` instead of throwing. This is preferred if it's possible for the symbol to not be defined and you must not throw, instead preferring to handle the failure a different way. This still calls the throwing interface as that's what oilgen depends on, but makes sure the precondition is true first such that it won't throw.

Differential Revision: D50363926
2023-10-17 08:31:26 -07:00
Jake Hillion
e867178ebd capture_keys: include data in type path 2023-10-11 16:32:35 -07:00
Jake Hillion
37991140da support 0 to many config files (#371)
Summary:
Previously OID/OIL required exactly one configuration file. This change makes it so you can supply 0 or more configuration files. 0 is useful if you have pre-generated the cache or use some sort of remote generation system. 1 is useful for the common case, where you have a configuration file that describes your entire source and use just that. More are useful if you have supplemental bits of config you wish to apply/override - see the changes to the integration test framework where we do exactly this.


Test Plan:
This isn't super well tested. It works for the test cases which add features via the config or enable `codegen.ignore`.

- CI

Reviewed By: ajor

Differential Revision: D49758032

Pulled By: JakeHillion
2023-10-02 14:06:39 -06:00
Alastair Robertson
7361d8fa29 TreeBuilder v2: Extend Element with a data member 2023-09-28 19:42:49 -07:00
Thierry Treyer
c657a41d79 Add CSV Exporter 2023-09-29 03:17:45 +02:00
Jake Hillion
8dd1182323 static types: add consume function similar to delegate 2023-09-15 14:43:09 +01:00
Jake Hillion
d71307cb43 oil: change std::stack reference to a std::function (#345)
Summary:

Previously on large types OIL would have problems with corrupting the `std::stack<exporter::inst::Inst>` that is passed to the processors. This change hides the implementation of the stack from the processors by wrapping the call to emplace in a `std::function` written by the non-generated code, which solves the test case I've seen for this crashing. It also allows us to easily change the stack implementation in future - I plan to change it to a `std::stack<T, std::vector<T>>` in a follow up.

Reviewed By: tyroguru

Differential Revision: D49273116
2023-09-14 16:57:45 +01:00
Jake Hillion
1d387ca880 oil: update iterator for range based for loops 2023-09-14 11:49:18 +01:00
Jake Hillion
a9cffbe22d Compile Time OIL for OIL v2
Summary:
Update `OIGenerator` and out BUCK stuff for compile time OIL with OIL v2. Main things:
- Switch `OIGenerator` from the `getObjectSize` call to the new `introspect` call.
- Switch from looking at template parameters to looking at function parameters, as this was exposing a bug in our elfutils/drgn and this way it's the same as OID.
- Migrate `OIGenerator` to CodeGen v2 and update CodeGen v2 to accept a linkage name.
- Update the compile time example to be the same as the JIT example, using the new interface and the JSON exporter.
- Clean up the `ObjectIntrospection.h` header.

Differential Revision: D48687728

fbshipit-source-id: 2c3c041fd1b6499c5e02eb5e2082a977bfa529d7
2023-08-30 04:28:02 -07:00
Jake Hillion
2bf1dd8b60 Update internal build for oil v2 2023-08-24 15:16:52 +01:00
Jake Hillion
5071519e45 oil v2 2023-08-23 15:59:53 +01:00
Jake Hillion
d7cfcca30d move all internal logic to oi::detail namespace 2023-07-26 18:01:38 +01:00
Jake Hillion
bd948152b7 add exporters::TypeCheckingWalker 2023-07-13 16:05:24 +01:00
Jake Hillion
032c28c0ea type checking: add description of data segment type 2023-07-13 16:05:24 +01:00
Jake Hillion
02defdb859 comment existing typed data segment work 2023-07-10 21:09:43 +01:00
Jake Hillion
6aead62652 static types: place in own header for testing 2023-06-30 12:54:02 +01:00