JakeHillion/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3668d993a3
nixpkgs/pkgs/build-support/rust
History
Benjamin Hipple 2115a2037c fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir 
This has several advantages:

1. It takes up less space on disk in-between builds in the nix store.
2. It uses less space in the binary cache for vendor derivation packages.
3. It uses less network traffic downloading from the binary cache.
4. It plays nicely with hashed mirrors like tarballs.nixos.org, which only
   substitute --flat hashes on single files (not recursive directory hashes).
5. It's consistent with how simple `fetchurl` src derivations work.
6. It provides a stronger abstraction between input src-package and output
   package, e.g., it's harder to accidentally depend on the src derivation at
   runtime by referencing something like `${src}/etc/index.html`. Likewise, in
   the store it's harder to get confused with something that is just there as a
   build-time dependency vs. a runtime dependency, since the build-time
   src dependencies are tarred up.

Disadvantages are:
1. It takes slightly longer to untar at the start of a build.

As currently implemented, this attaches the compacted vendor.tar.gz feature as a
rider on `verifyCargoDeps`, since both of them are relatively newly implemented
behavior that change the `cargoSha256`.

If this PR is accepted, I will push forward the remaining rust packages with a
series of treewide PRs to update the `cargoSha256`s.
2020-02-10 10:17:29 -05:00
..
build-rust-crate buildRustCrate: fix #78412 2020-01-28 14:07:58 +01:00
patch-registry-deps
cargo-vendor-normalise.py fetchcargo: add type checking to cargo-vendor-normalise.py 2018-09-11 23:44:14 +02:00
carnix.nix Carnix: 0.9 -> 0.10 2019-03-26 18:54:28 +00:00
crates-io.nix treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
default-crate-overrides.nix cargo-vendor: drop 2019-09-22 20:20:20 -05:00
default.nix fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir 2020-02-10 10:17:29 -05:00
fetchcargo-default-config.toml fectchcargo: don't break old sha256 2018-09-11 23:44:14 +02:00
fetchcargo.nix rustPlatform.fetchcargo: expose 2019-12-23 18:27:56 +00:00
fetchCargoTarball.nix fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir 2020-02-10 10:17:29 -05:00
fetchcrate.nix add fetchCrate function to fetch rust crates 2017-12-12 04:58:45 -06:00
README.md fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir 2020-02-10 10:17:29 -05:00

README.md

Updated fetchCargo behavior

Changes to the fetchcargo.nix behavior that cause changes to the cargoSha256 are somewhat disruptive, so historically we've added conditionals to provide backwards compatibility. We've now accumulated enough of these that it makes sense to do a clean sweep updating hashes, and delete the conditionals in the fetcher to simplify maintenance and implementation complexity. These conditionals are:

  1. When cargo vendors dependencies, it generates a config. Previously, we were hard-coding our own config, but this fails if there are git dependencies. We have conditional logic to sometimes copy the vendored cargo config in, and sometimes not.

  2. When a user updates the src package, they may forget to update the cargoSha256. We have an opt-in conditional flag to add the Cargo.lock into the vendor dir for inspection and compare at build-time, but it defaults to false.

  3. We were previously vendoring into a directory with a recursive hash, but would like to vendor into a compressed tar.gz file instead, for the reasons specified in the git commit message adding this feature.

Migration plan

  1. (DONE in this PR) Implement fetchCargoTarball as a separate, clean fetcher implementation along-side fetchcargo. Rename verifyCargoDeps (default false) to legacyCargoFetcher (default true), which switches the fetcher implementation used. Replace verifyCargoDeps = true; with legacyCargoFetcher = false; in Rust applications.

  2. Send a treewide Rust PR that sets legacyCargoFetcher = true; in all Rust applications not using this (which is ~200 of them), with a note to maintainers to delete if updating the package. Change the default in buildRustPackage to false.

  3. Go through all Rust src packages deleting the legacyCargoFetcher = false; line and re-computing the cargoSha256, merging as we go.

  4. Delete the fetchcargo.nix implementation entirely and also remove:

  • All overrides in application-level packages
  • The fetchcargo-default-config.toml and conditionals around using it when no $CARGO_CONFIG exists
  • This README.md file