object-introspection/flake.nix

133 lines
3.1 KiB
Nix
Raw Permalink Normal View History

2023-12-14 15:18:24 +00:00
{
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
description = "Object level memory profiler for C++";
2023-12-14 15:18:24 +00:00
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
treefmt-nix.url = "github:numtide/treefmt-nix";
treefmt-nix.inputs.nixpkgs.follows = "nixpkgs";
};
outputs =
{
self,
nixpkgs,
flake-utils,
treefmt-nix,
...
}@inputs:
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
flake-utils.lib.eachSystem [ flake-utils.lib.system.x86_64-linux ] (
system:
let
pkgs = import nixpkgs { inherit system; };
drgnSrc = pkgs.fetchFromGitHub {
owner = "JakeHillion";
repo = "drgn";
rev = "b1f8c3e8526611b6720800250ba858a713dd9e4f";
hash = "sha256-5WhMHgx/RKtqjxGx4AyiqVKMot5xulr+6c8i2E9IxiA=";
fetchSubmodules = true;
};
mkOidPackage =
llvmPackages:
with pkgs;
llvmPackages.stdenv.mkDerivation rec {
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
name = "oid";
src = self;
nativeBuildInputs = [
autoconf
automake
bison
cmake
flex
gettext
git
hexdump
libtool
ninja
pkgconf
python312
python312Packages.setuptools
python312Packages.toml
glibcLocales
];
buildInputs = [
llvmPackages.libclang
llvmPackages.llvm
llvmPackages.openmp
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
boost
bzip2
curl
double-conversion
elfutils
flex
folly
folly.fmt
gflags
glog
gtest
icu
jemalloc
libarchive
libmicrohttpd
liburing
libxml2
lzma
msgpack
range-v3
rocksdb_8_11
sqlite
tomlplusplus
zstd
];
cmakeFlags = [
"-Ddrgn_SOURCE_DIR=${drgnSrc}"
"-DFORCE_BOOST_STATIC=Off"
];
outputs = [ "out" ];
};
in
{
packages = rec {
default = oid-llvm16;
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
oid-llvm15 = mkOidPackage pkgs.llvmPackages_15;
oid-llvm16 = mkOidPackage pkgs.llvmPackages_16;
nix: add building oid to the flake OI's build is challenging and has often been a problem for the Open Source community. It requires an extremely specific set of dependencies that are very hard to achieve on most systems. There are frequent breakages, like when updating to CentOS Stream 9, or when trying to update the CI's clang from clang-12 to clang-15 - OI requires the clang libraries to be version 15 but can't be compiled with it on the CI! This changes provides a mostly working build environment with `nix`. This environment is pinned to a specific nixpkgs revision using `flake.lock`, and only updates when we explicitly tell it to. Summary of changes: - Update CMakeLists.txt required version to 3.24. This allows specifying `FIND_PACKAGE_ARGS` in `FetchContent`, meaning we can use system packages. This is available on most up to date distros (3.30.2 is current). - Extends `flake.nix` to be able to build OI. Adds instructions for building and developing OI using `nix`. - Partially runs the tests in GitHub Actions. A huge amount must be excluded because of incompatibilites between the clangStdenv compiler and drgn. We have similar, though fewer, issues when building with the clang-12/libstdcxx mix on the Ubuntu 22.04 CircleCI, though this is at least reproducible. - Updates CircleCI to build CMake from source as we don't have a newer image available. Also add some newly found dependencies (not sure how it was working without them before). Test plan: This change requires less testing than previous build related changes because it deprecates most of the build types. - The internal BUCK build is unaffected. No special testing. - The semi-internal CMake build is gone. Use Nix. - The Nix build for clang-15 and some tests are continuously tested in GitHub actions. - Tested the set of Nix commands in the README. All work except the one that points to GitHub as this must be merged first. - The existing CircleCI runs on Ubuntu 20.04 are maintained. - Unable to test the new `test-report.yml` as it must be merged due to the permissions it needs. Will follow up with testing after this is merged. See: https://github.com/dorny/test-reporter?tab=readme-ov-file#recommended-setup-for-public-repositories The list of exclusions for GitHub Actions/nix testing is currently very long, I think 29% of the tests. This should be stable and reproducible though, and likely needs deep changes to OI to fix. That's why fixes are excluded from this PR. It's all to do with the forked drgn not being able to parse clang's newer DWARF output, and can't be fixed by rolling back as we required a relatively new libcxx.
2024-08-15 15:48:08 +01:00
};
apps.default = {
type = "app";
program = "${self.packages.${system}.default}/bin/oid";
};
}
)
// flake-utils.lib.eachDefaultSystem (
system:
2023-12-14 15:18:24 +00:00
let
pkgs = nixpkgs.legacyPackages.${system};
treefmtEval = treefmt-nix.lib.evalModule pkgs (pkgs: {
projectRootFile = "flake.nix";
settings.global.excludes = [ "./extern/**" ];
programs.nixfmt.enable = true;
programs.clang-format.enable = true;
programs.black.enable = true;
programs.isort.enable = true;
});
in
{
2023-12-14 15:18:24 +00:00
formatter = treefmtEval.config.build.wrapper;
checks.formatting = treefmtEval.config.build.check self;
}
);
2023-12-14 15:18:24 +00:00
}