nixpkgs/pkgs/development/ruby-modules/gem/default.nix

235 lines
6.6 KiB
Nix
Raw Normal View History

ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
# This builds gems in a way that is compatible with bundler.
#
# Bundler installs gems from git sources _very_ differently from how RubyGems
# installes gem packages, though they both install gem packages similarly.
#
# We monkey-patch Bundler to remove any impurities and then drive its internals
# to install git gems.
#
# For the sake of simplicity, gem packages are installed with the standard `gem`
# program.
#
# Note that bundler does not support multiple prefixes; it assumes that all
# gems are installed in a common prefix, and has no support for specifying
# otherwise. Therefore, if you want to be able to use the resulting derivations
# with bundler, you need to create a symlink forrest first, which is what
# `bundlerEnv` does for you.
#
# Normal gem packages can be used outside of bundler; a binstub is created in
# $out/bin.
2016-06-12 18:11:37 +01:00
{ lib, fetchurl, fetchgit, makeWrapper, git, darwin
, ruby, bundler
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
} @ defs:
lib.makeOverridable (
{ name ? null
, gemName
, version ? null
, type ? "gem"
, document ? [] # e.g. [ "ri" "rdoc" ]
, platform ? "ruby"
, ruby ? defs.ruby
, stdenv ? ruby.stdenv
, namePrefix ? (let
rubyName = builtins.parseDrvName ruby.name;
in "${rubyName.name}${rubyName.version}-")
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
, buildInputs ? []
, meta ? {}
, patches ? []
, gemPath ? []
, dontStrip ? true
# Assume we don't have to build unless strictly necessary (e.g. the source is a
# git checkout).
# If you need to apply patches, make sure to set `dontBuild = false`;
, dontBuild ? true
, dontInstallManpages ? false
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
, propagatedBuildInputs ? []
, propagatedUserEnvPkgs ? []
, buildFlags ? []
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
, passthru ? {}
, ...} @ attrs:
let
src = attrs.src or (
if type == "gem" then
fetchurl {
urls = map (
remote: "${remote}/gems/${gemName}-${version}.gem"
) (attrs.source.remotes or [ "https://rubygems.org" ]);
inherit (attrs.source) sha256;
}
else if type == "git" then
fetchgit {
inherit (attrs.source) url rev sha256 fetchSubmodules;
}
else if type == "url" then
fetchurl attrs.source
else
throw "buildRubyGem: don't know how to build a gem of type \"${type}\""
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
);
documentFlag =
if document == []
then "-N"
else "--document ${lib.concatStringsSep "," document}";
in
stdenv.mkDerivation ((builtins.removeAttrs attrs ["source"]) // {
inherit ruby;
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
inherit dontBuild;
inherit dontStrip;
inherit type;
buildInputs = [
ruby makeWrapper
] ++ lib.optionals (type == "git") [ git ]
++ lib.optionals (type != "gem") [ bundler ]
++ lib.optional stdenv.isDarwin darwin.libobjc
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
++ buildInputs;
#name = builtins.trace (attrs.name or "no attr.name" ) "${namePrefix}${gemName}-${version}";
name = attrs.name or "${namePrefix}${gemName}-${version}";
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
inherit src;
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
unpackPhase = attrs.unpackPhase or ''
runHook preUnpack
if [[ -f $src && $src == *.gem ]]; then
if [[ -z "$dontBuild" ]]; then
# we won't know the name of the directory that RubyGems creates,
# so we'll just use a glob to find it and move it over.
gempkg="$src"
sourceRoot=source
gem unpack $gempkg --target=container
cp -r container/* $sourceRoot
rm -r container
# copy out the original gemspec, for convenience during patching /
# overrides.
gem specification $gempkg --ruby > original.gemspec
gemspec=$(readlink -f .)/original.gemspec
else
gempkg="$src"
fi
else
# Fall back to the original thing for everything else.
dontBuild=""
preUnpack="" postUnpack="" unpackPhase
fi
runHook postUnpack
'';
buildPhase = attrs.buildPhase or ''
runHook preBuild
if [[ "$type" == "gem" ]]; then
if [[ -z "$gemspec" ]]; then
gemspec="$(find . -name '*.gemspec')"
echo "found the following gemspecs:"
echo "$gemspec"
gemspec="$(echo "$gemspec" | head -n1)"
fi
exec 3>&1
output="$(gem build $gemspec | tee >(cat - >&3))"
exec 3>&-
gempkg=$(echo "$output" | grep -oP 'File: \K(.*)')
echo "gem package built: $gempkg"
elif [[ "$type" == "git" ]]; then
git init
# remove variations to improve the likelihood of a bit-reproducible output
rm -rf .git/logs/ .git/hooks/ .git/index .git/FETCH_HEAD .git/ORIG_HEAD .git/refs/remotes/origin/HEAD .git/config
# support `git ls-files`
git add .
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
fi
runHook postBuild
'';
# Note:
# We really do need to keep the $out/${ruby.gemPath}/cache.
# This is very important in order for many parts of RubyGems/Bundler to not blow up.
# See https://github.com/bundler/bundler/issues/3327
installPhase = attrs.installPhase or ''
runHook preInstall
export GEM_HOME=$out/${ruby.gemPath}
mkdir -p $GEM_HOME
echo "buildFlags: $buildFlags"
${lib.optionalString (type == "url") ''
ruby ${./nix-bundle-install.rb} \
"path" \
'${gemName}' \
'${version}' \
'${lib.escapeShellArgs buildFlags}'
''}
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
${lib.optionalString (type == "git") ''
ruby ${./nix-bundle-install.rb} \
"git" \
'${gemName}' \
'${version}' \
'${lib.escapeShellArgs buildFlags}' \
'${attrs.source.url}' \
'.' \
'${attrs.source.rev}'
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
''}
${lib.optionalString (type == "gem") ''
if [[ -z "$gempkg" ]]; then
echo "failure: \$gempkg path unspecified" 1>&2
exit 1
elif [[ ! -f "$gempkg" ]]; then
echo "failure: \$gempkg path invalid" 1>&2
exit 1
fi
gem install \
--local \
--force \
--http-proxy 'http://nodtd.invalid' \
--ignore-dependencies \
--install-dir "$GEM_HOME" \
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
--build-root '/' \
--backtrace \
--no-env-shebang \
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
${documentFlag} \
$gempkg $gemFlags -- $buildFlags
# looks like useless files which break build repeatability and consume space
rm -fv $out/${ruby.gemPath}/doc/*/*/created.rid || true
rm -fv $out/${ruby.gemPath}/gems/*/ext/*/mkmf.log || true
# write out metadata and binstubs
spec=$(echo $out/${ruby.gemPath}/specifications/*.gemspec)
ruby ${./gem-post-build.rb} "$spec"
''}
${lib.optionalString (!dontInstallManpages) ''
for section in {1..9}; do
mandir="$out/share/man/man$section"
find $out/lib \( -wholename "*/man/*.$section" -o -wholename "*/man/man$section/*.$section" \) \
-execdir mkdir -p $mandir \; -execdir cp '{}' $mandir \;
done
''}
ruby: new bundler infrastructure This improves our Bundler integration (i.e. `bundlerEnv`). Before describing the implementation differences, I'd like to point a breaking change: buildRubyGem now expects `gemName` and `version` as arguments, rather than a `name` attribute in the form of "<gem-name>-<version>". Now for the differences in implementation. The previous implementation installed all gems at once in a single derivation. This was made possible by using a set of monkey-patches to prevent Bundler from downloading gems impurely, and to help Bundler find and activate all required gems prior to installation. This had several downsides: * The patches were really hard to understand, and required subtle interaction with the rest of the build environment. * A single install failure would cause the entire derivation to fail. The new implementation takes a different approach: we install gems into separate derivations, and then present Bundler with a symlink forest thereof. This has a couple benefits over the existing approach: * Fewer patches are required, with less interplay with the rest of the build environment. * Changes to one gem no longer cause a rebuild of the entire dependency graph. * Builds take 20% less time (using gitlab as a reference). It's unfortunate that we still have to muck with Bundler's internals, though it's unavoidable with the way that Bundler is currently designed. There are a number improvements that could be made in Bundler that would simplify our packaging story: * Bundler requires all installed gems reside within the same prefix (GEM_HOME), unlike RubyGems which allows for multiple prefixes to be specified through GEM_PATH. It would be ideal if Bundler allowed for packages to be installed and sourced from multiple prefixes. * Bundler installs git sources very differently from how RubyGems installs gem packages, and, unlike RubyGems, it doesn't provide a public interface (CLI or programmatic) to guide the installation of a single gem. We are presented with the options of either reimplementing a considerable portion Bundler, or patch and use parts of its internals; I choose the latter. Ideally, there would be a way to install gems from git sources in a manner similar to how we drive `gem` to install gem packages. * When a bundled program is executed (via `bundle exec` or a binstub that does `require 'bundler/setup'`), the setup process reads the Gemfile.lock, activates the dependencies, re-serializes the lock file it read earlier, and then attempts to overwrite the Gemfile.lock if the contents aren't bit-identical. I think the reasoning is that by merely running an application with a newer version of Bundler, you'll automatically keep the Gemfile.lock up-to-date with any changes in the format. Unfortunately, that doesn't play well with any form of packaging, because bundler will immediately cause the application to abort when it attempts to write to the read-only Gemfile.lock in the store. We work around this by normalizing the Gemfile.lock with the version of Bundler that we'll use at runtime before we copy it into the store. This feels fragile, but it's the best we can do without changes upstream, or resorting to more delicate hacks. With all of the challenges in using Bundler, one might wonder why we can't just cut Bundler out of the picture and use RubyGems. After all, Nix provides most of the isolation that Bundler is used for anyway. The problem, however, is that almost every Rails application calls `Bundler::require` at startup (by way of the default project templates). Because bundler will then, by default, `require` each gem listed in the Gemfile, Rails applications are almost always written such that none of the source files explicitly require their dependencies. That leaves us with two options: support and use Bundler, or maintain massive patches for every Rails application that we package. Closes #8612
2015-11-15 02:17:29 +00:00
runHook postInstall
'';
propagatedBuildInputs = gemPath ++ propagatedBuildInputs;
propagatedUserEnvPkgs = gemPath ++ propagatedUserEnvPkgs;
passthru = passthru // { isRubyGem = true; };
inherit meta;
})
)