JakeHillion/object-introspection
1
0
Fork 0
mirror of https://github.com/JakeHillion/object-introspection.git synced 2024-11-10 05:26:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
fe9b4b232a
object-introspection/.circleci/config.yml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

233 lines
7.6 KiB
YAML
Raw Normal View History

Initial commit
2022-12-19 14:37:51 +00:00
version: 2.1
workflows:
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
object-introspection:
Initial commit
2022-12-19 14:37:51 +00:00
jobs:
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
- build:
name: build-gcc
Initial commit
2022-12-19 14:37:51 +00:00
cc: /usr/bin/gcc
cxx: /usr/bin/g++
Enable -Werror for Clang builds in CI Don't enable for local dev builds or for GCC CI builds, as every compiler version has a different set of warnings and trying to make them all pass is a neverending task.
2023-05-22 11:02:47 +01:00
warnings_as_errors: "OFF"
Integration tests: Set up CI testing for TypeGraph CTest can't forward command line arguments to the test runner, so add the option to using an environment variable to enable features instead. CMake issue tracking the feature that would have been needed: https://gitlab.kitware.com/cmake/cmake/-/issues/20470 Tests which aren't passing yet have been disabled in CI.
2023-06-26 16:34:52 +01:00
- test:
OID: Make CodeGen v2 (TypeGraph) the default
2023-12-14 16:39:16 +00:00
name: test-gcc
Integration tests: Set up CI testing for TypeGraph CTest can't forward command line arguments to the test runner, so add the option to using an environment variable to enable features instead. CMake issue tracking the feature that would have been needed: https://gitlab.kitware.com/cmake/cmake/-/issues/20470 Tests which aren't passing yet have been disabled in CI.
2023-06-26 16:34:52 +01:00
requires:
- build-gcc
tests: add ClangTypeParserTest Currently there is no testing for ClangTypeParser even though it's used in production. This is because adding integration tests is very hard: they require testing the build time behaviour at runtime, or else they'd be build failures intead of test failures. There's a PR available for integration tests but it's incomplete. In contrast ClangTypeParser can be sort of unit tested. This follows the structure of `test/test_drgn_parser.cpp` with some differences. There is a tonne of boilerplate for setting up the Clang tool, and this set of testing operates on type names instead of OID functions. The new tests are also incredibly slow as they compile the entire `integration_test_target.cpp` (which is huge) for every test case. I don't think this is avoidable without compromising the separation of the tests somewhat due to the way Clang tooling forces the code to be structured. Currently I can't run these tests locally on a Meta devserver due to some weirdness with the internal build and the `compile_commands.json` file. They run in the CI and on any other open source machine though so I'm happy to merge it - it's still useful. I'm going to close the PR to change the devserver build given I'll be unable to follow up if it ends up being bad. Test plan: - CI
2024-02-23 16:26:03 +00:00
exclude_regex: ".*inheritance_polymorphic.*|.*arrays_member_int0|ClangTypeParserTest.*"
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
- coverage:
CI: Fix code coverage collection Multiple jobs are not allowed to persist the same files to a workspace. This commit takes the lazy approach of splitting the type-graph and non-type-graph jobs into different workspaces and creating a second coverage job to avoid the conflict.
2023-06-28 09:37:24 +01:00
name: coverage
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
requires:
- test-gcc
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
- build:
name: build-clang
Initial commit
2022-12-19 14:37:51 +00:00
cc: /usr/bin/clang-12
cxx: /usr/bin/clang++-12
Enable -Werror for Clang builds in CI Don't enable for local dev builds or for GCC CI builds, as every compiler version has a different set of warnings and trying to make them all pass is a neverending task.
2023-05-22 11:02:47 +01:00
warnings_as_errors: "ON"
Integration tests: Set up CI testing for TypeGraph CTest can't forward command line arguments to the test runner, so add the option to using an environment variable to enable features instead. CMake issue tracking the feature that would have been needed: https://gitlab.kitware.com/cmake/cmake/-/issues/20470 Tests which aren't passing yet have been disabled in CI.
2023-06-26 16:34:52 +01:00
- test:
OID: Make CodeGen v2 (TypeGraph) the default
2023-12-14 16:39:16 +00:00
name: test-clang
Integration tests: Set up CI testing for TypeGraph CTest can't forward command line arguments to the test runner, so add the option to using an environment variable to enable features instead. CMake issue tracking the feature that would have been needed: https://gitlab.kitware.com/cmake/cmake/-/issues/20470 Tests which aren't passing yet have been disabled in CI.
2023-06-26 16:34:52 +01:00
requires:
- build-clang
TypeGraph: Create dummy containers These represent types which don't store any interesting data for us to measure, but which are required by a real container so can not be replaced with our own generated class types. std::allocator often has bad DWARF, so it must be replaced after the DWARF is fixed up in Flattener. The others could be replaced earlier in the transformation process if desired, but I've left them all together for simplicity for now. This fixes the folly::fbstring tests.
2023-06-26 19:00:51 +01:00
# Tests disabled due to bad DWARF generated by the old clang compiler in CI
ignore fully qualified types
2024-02-07 15:41:52 +00:00
exclude_regex: ".*inheritance_polymorphic.*|.*arrays_member_int0|.*fbstring.*|.*std_string_*|.*multi_arg_tb_.*|.*ignored_member|OilIntegration.fbstring_.*|OilIntegration.capture_keys_string|OilIntegration.capture_keys_multi_level"
Initial commit
2022-12-19 14:37:51 +00:00
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
executors:
ubuntu-docker:
Initial commit
2022-12-19 14:37:51 +00:00
docker:
- image: ubuntu:jammy
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
resource_class: small
big-boy:
machine:
image: ubuntu-2204:2022.10.2
resource_class: 2xlarge
jobs:
build:
# TODO this job could be run in Docker
executor: big-boy
Initial commit
2022-12-19 14:37:51 +00:00
parameters:
cc:
type: string
cxx:
type: string
Enable -Werror for Clang builds in CI Don't enable for local dev builds or for GCC CI builds, as every compiler version has a different set of warnings and trying to make them all pass is a neverending task.
2023-05-22 11:02:47 +01:00
warnings_as_errors:
type: string
Initial commit
2022-12-19 14:37:51 +00:00
environment:
CC: << parameters.cc >>
CXX: << parameters.cxx >>
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
working_directory:
/tmp/object-introspection
Initial commit
2022-12-19 14:37:51 +00:00
steps:
- run:
name: Install dependencies
command: |
sudo apt-get update
sudo apt-get install -y \
bison \
build-essential \
clang-12 \
Update to clang/llvm 15 (#342) Summary: Update to clang-15 compiler and libraries as clang-12 is ancient. The changes to oilgen are necessary because the new internal toolchain is being more picky about linking PIC to PIC. In certain modes we build with PIC, but try to link a non-PIC oilgen artifact. Add the ability to build the oilgen artifacts with PIC which sorts this. Reviewed By: ttreyer Differential Revision: D46220858
2023-09-14 11:40:11 +01:00
clang-15 \
Initial commit
2022-12-19 14:37:51 +00:00
cmake \
flex \
gawk \
libboost-all-dev \
libbz2-dev \
libcap2-bin \
Update to clang/llvm 15 (#342) Summary: Update to clang-15 compiler and libraries as clang-12 is ancient. The changes to oilgen are necessary because the new internal toolchain is being more picky about linking PIC to PIC. In certain modes we build with PIC, but try to link a non-PIC oilgen artifact. Add the ability to build the oilgen artifacts with PIC which sorts this. Reviewed By: ttreyer Differential Revision: D46220858
2023-09-14 11:40:11 +01:00
libclang-15-dev \
Initial commit
2022-12-19 14:37:51 +00:00
libcurl4-gnutls-dev \
libdouble-conversion-dev \
libdw-dev \
libfmt-dev \
libgflags-dev \
libgmock-dev \
libgoogle-glog-dev \
libgtest-dev \
libjemalloc-dev \
libmsgpack-dev \
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
libomp-12-dev \
liburing-dev \
Initial commit
2022-12-19 14:37:51 +00:00
libzstd-dev \
Update to clang/llvm 15 (#342) Summary: Update to clang-15 compiler and libraries as clang-12 is ancient. The changes to oilgen are necessary because the new internal toolchain is being more picky about linking PIC to PIC. In certain modes we build with PIC, but try to link a non-PIC oilgen artifact. Add the ability to build the oilgen artifacts with PIC which sorts this. Reviewed By: ttreyer Differential Revision: D46220858
2023-09-14 11:40:11 +01:00
llvm-15-dev \
Initial commit
2022-12-19 14:37:51 +00:00
ninja-build \
pkg-config \
CI: Output junit result format directly from CTest Remove the conversion script we were previously using.
2023-02-01 18:06:31 +00:00
python3-setuptools
Initial commit
2022-12-19 14:37:51 +00:00
pip3 install toml
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
# Ubuntu 22.04 CMake is too old and we don't have a newer image yet
git clone --depth 1 --branch v3.30.2 https://github.com/Kitware/CMake.git /tmp/cmake
(cd /tmp/cmake && cmake -B build/ -G Ninja && cmake --build build/)
Initial commit
2022-12-19 14:37:51 +00:00
environment:
DEBIAN_FRONTEND: noninteractive
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
- checkout
Initial commit
2022-12-19 14:37:51 +00:00
- run:
name: Build
command: |
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
/tmp/cmake/build/bin/cmake -G Ninja -B build/ -DWITH_FLAKY_TESTS=Off -DCODE_COVERAGE=On -DWARNINGS_AS_ERRORS=<< parameters.warnings_as_errors >>
ninja -C build/
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
# Testing rubbish:
cp test/ci.oid.toml build/testing.oid.toml
- persist_to_workspace:
root: .
paths:
- build/*
- extern/*
tests: add ClangTypeParserTest Currently there is no testing for ClangTypeParser even though it's used in production. This is because adding integration tests is very hard: they require testing the build time behaviour at runtime, or else they'd be build failures intead of test failures. There's a PR available for integration tests but it's incomplete. In contrast ClangTypeParser can be sort of unit tested. This follows the structure of `test/test_drgn_parser.cpp` with some differences. There is a tonne of boilerplate for setting up the Clang tool, and this set of testing operates on type names instead of OID functions. The new tests are also incredibly slow as they compile the entire `integration_test_target.cpp` (which is huge) for every test case. I don't think this is avoidable without compromising the separation of the tests somewhat due to the way Clang tooling forces the code to be structured. Currently I can't run these tests locally on a Meta devserver due to some weirdness with the internal build and the `compile_commands.json` file. They run in the CI and on any other open source machine though so I'm happy to merge it - it's still useful. I'm going to close the PR to change the devserver build given I'll be unable to follow up if it ends up being bad. Test plan: - CI
2024-02-23 16:26:03 +00:00
- include/*
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
- types/*
Initial commit
2022-12-19 14:37:51 +00:00
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
test:
executor: big-boy
Integration tests: Set up CI testing for TypeGraph CTest can't forward command line arguments to the test runner, so add the option to using an environment variable to enable features instead. CMake issue tracking the feature that would have been needed: https://gitlab.kitware.com/cmake/cmake/-/issues/20470 Tests which aren't passing yet have been disabled in CI.
2023-06-26 16:34:52 +01:00
parameters:
oid_test_args:
type: string
default: ""
tests_regex:
type: string
default: ".*"
exclude_regex:
type: string
default: ""
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
working_directory:
/tmp/object-introspection
CI: Split build and test into separate jobs This will give us more flexibility later, e.g. allowing different steps for GCC and Clang (as needed for code coverage), or letting us parallelise our tests across multiple machines.
2023-02-01 18:03:42 +00:00
steps:
- attach_workspace:
at: .
- run:
name: Install dependencies
command: |
sudo apt-get update
sudo apt-get install -y \
Update to clang/llvm 15 (#342) Summary: Update to clang-15 compiler and libraries as clang-12 is ancient. The changes to oilgen are necessary because the new internal toolchain is being more picky about linking PIC to PIC. In certain modes we build with PIC, but try to link a non-PIC oilgen artifact. Add the ability to build the oilgen artifacts with PIC which sorts this. Reviewed By: ttreyer Differential Revision: D46220858
2023-09-14 11:40:11 +01:00
clang-15 \
update folly and enable ci usage
2023-02-23 23:32:15 +00:00
libboost-all-dev \
add test for folly::fbstring
2023-04-05 21:56:55 +01:00
libfmt-dev \
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
libgflags-dev \
libgoogle-glog-dev \
libjemalloc-dev \
libomp-12-dev \
llvm-15-dev
CI: Set installs to noninteractive so they don't hang waiting for user input
2023-08-01 16:58:06 +01:00
environment:
DEBIAN_FRONTEND: noninteractive
Initial commit
2022-12-19 14:37:51 +00:00
- run:
name: Test
environment:
# disable drgn multithreading as tests are already run in parallel
OMP_NUM_THREADS: 1
command: |
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
Integration tests: Set up CI testing for TypeGraph CTest can't forward command line arguments to the test runner, so add the option to using an environment variable to enable features instead. CMake issue tracking the feature that would have been needed: https://gitlab.kitware.com/cmake/cmake/-/issues/20470 Tests which aren't passing yet have been disabled in CI.
2023-06-26 16:34:52 +01:00
OID_TEST_ARGS='<< parameters.oid_test_args >>' ctest \
--test-dir build/test/ \
--test-action Test \
-j 16 \
--tests-regex '<< parameters.tests_regex >>' \
--exclude-regex '<< parameters.exclude_regex >>' \
--no-compress-output \
--output-on-failure \
--schedule-random \
--timeout 60 \
ci: retry tests once on failure
2023-06-28 16:08:07 +01:00
--repeat until-pass:2 \
CI: Output junit result format directly from CTest Remove the conversion script we were previously using.
2023-02-01 18:06:31 +00:00
--output-junit results.xml
Initial commit
2022-12-19 14:37:51 +00:00
- store_test_results:
CI: Output junit result format directly from CTest Remove the conversion script we were previously using.
2023-02-01 18:06:31 +00:00
path: build/test/results.xml
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
- persist_to_workspace:
# Save code coverage data
root: .
paths:
- build/*
coverage:
executor: ubuntu-docker
working_directory:
/tmp/object-introspection
steps:
- run:
name: Install dependencies
command: |
apt-get update
apt-get install -y \
Code Coverage: Reduce irrelevant partial branch matches The C++ compiler generates implicit branches for exception handling, which is particularly common when calling standard library functions. It's not generally useful to have tests for the case when std::vector::push_back fails due to an out-of-memory condition, so in this PR we use a feature in the latest lcov release to filter out these compiler-generated branches. Our code coverage numbers consequently go way up!
2023-06-21 21:03:48 +01:00
build-essential \
cpanminus \
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
curl \
git \
Code Coverage: Reduce irrelevant partial branch matches The C++ compiler generates implicit branches for exception handling, which is particularly common when calling standard library functions. It's not generally useful to have tests for the case when std::vector::push_back fails due to an out-of-memory condition, so in this PR we use a feature in the latest lcov release to filter out these compiler-generated branches. Our code coverage numbers consequently go way up!
2023-06-21 21:03:48 +01:00
gpg
# Install lcov 2.0 - required for the "--filter branch" option.
# This improves C++ branch coverage by excluding compiler-generated
# branches, which primarily come from exception handling in
# standard library functions.
cpanm --notest Capture::Tiny DateTime
pushd /tmp
curl -sLO https://github.com/linux-test-project/lcov/releases/download/v2.0/lcov-2.0.tar.gz
tar -xf lcov-2.0.tar.gz
cd lcov-2.0
make install
popd
CI: Set installs to noninteractive so they don't hang waiting for user input
2023-08-01 16:58:06 +01:00
environment:
DEBIAN_FRONTEND: noninteractive
Code coverage: Checkout repo before running Apparently this is needed for CodeCov.io to work.
2023-02-03 16:26:22 +00:00
- checkout
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
- attach_workspace:
at: .
- run:
name: Code Coverage
when: always
command: |
Code Coverage: Reduce irrelevant partial branch matches The C++ compiler generates implicit branches for exception handling, which is particularly common when calling standard library functions. It's not generally useful to have tests for the case when std::vector::push_back fails due to an out-of-memory condition, so in this PR we use a feature in the latest lcov release to filter out these compiler-generated branches. Our code coverage numbers consequently go way up!
2023-06-21 21:03:48 +01:00
lcov --capture --directory . --filter branch --no-external --ignore-errors mismatch --ignore-errors source --rc lcov_branch_coverage=1 --output-file coverage.info
CodeCov: Collect coverage from all source files The "src" directory got renamed to "oi" and the previous rule stopped matching our source code. By changing to collect coverage for everything except for "build" and "extern", we should avoid this problem in the future.
2023-05-24 16:14:19 +01:00
# Empirically, extract-then-remove is faster than remove-then-extract
lcov --extract coverage.info '/tmp/object-introspection/*' --rc lcov_branch_coverage=1 --output-file coverage.info
lcov --remove coverage.info '/tmp/object-introspection/build/*' '/tmp/object-introspection/extern/*' --rc lcov_branch_coverage=1 --output-file coverage.info
CI: Add code coverage reporting Add CODE_COVERAGE option to enable coverage instrumentation in builds. Add new CI job to upload coverage reports to CodeCov.io. Reports can be found by navigating the codecov.io UI, or by going to the URL printed at the end of the "Code Coverage" CI job. More CodeCov integration could be set up later once we have coverage reporting on the main branch.
2023-02-03 09:47:57 +00:00
lcov --list --rc lcov_branch_coverage=1 coverage.info
curl https://keybase.io/codecovsecurity/pgp_keys.asc | gpg --no-default-keyring --keyring trustedkeys.gpg --import
curl -Os https://uploader.codecov.io/latest/linux/codecov
curl -Os https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
curl -Os https://uploader.codecov.io/latest/linux/codecov.SHA256SUM.sig
gpgv codecov.SHA256SUM.sig codecov.SHA256SUM
shasum -a 256 -c codecov.SHA256SUM
chmod +x codecov
# It appears that codecov wants to scan through all directories
# other than "build", looking for files to upload, even if we
# specify a file name on the command line.
#
# "extern" is huge and makes uploading the coverage report take
# forever. Delete it for a speedup.
rm -rf extern
Pass in CodeCov.io token through CI env var CodeCov's docs say that a token isn't required for a public repository, but our uploads have been broken for a few weeks at this point and passing in the token fixes them...
2023-04-21 13:50:29 +01:00
./codecov -Z -f coverage.info -t $CODECOV_TOKEN
Reference in New Issue Copy Permalink