ruby: new bundler infrastructure

This improves our Bundler integration (i.e. `bundlerEnv`).

Before describing the implementation differences, I'd like to point a
breaking change: buildRubyGem now expects `gemName` and `version` as
arguments, rather than a `name` attribute in the form of
"<gem-name>-<version>".

Now for the differences in implementation.

The previous implementation installed all gems at once in a single
derivation. This was made possible by using a set of monkey-patches to
prevent Bundler from downloading gems impurely, and to help Bundler
find and activate all required gems prior to installation. This had
several downsides:

* The patches were really hard to understand, and required subtle
  interaction with the rest of the build environment.
* A single install failure would cause the entire derivation to fail.

The new implementation takes a different approach: we install gems into
separate derivations, and then present Bundler with a symlink forest
thereof. This has a couple benefits over the existing approach:

* Fewer patches are required, with less interplay with the rest of the
  build environment.
* Changes to one gem no longer cause a rebuild of the entire dependency
  graph.
* Builds take 20% less time (using gitlab as a reference).

It's unfortunate that we still have to muck with Bundler's internals,
though it's unavoidable with the way that Bundler is currently designed.
There are a number improvements that could be made in Bundler that would
simplify our packaging story:

* Bundler requires all installed gems reside within the same prefix
  (GEM_HOME), unlike RubyGems which allows for multiple prefixes to
  be specified through GEM_PATH. It would be ideal if Bundler allowed
  for packages to be installed and sourced from multiple prefixes.
* Bundler installs git sources very differently from how RubyGems
  installs gem packages, and, unlike RubyGems, it doesn't provide a
  public interface (CLI or programmatic) to guide the installation of a
  single gem. We are presented with the options of either
  reimplementing a considerable portion Bundler, or patch and use parts
  of its internals; I choose the latter. Ideally, there would be a way
  to install gems from git sources in a manner similar to how we drive
  `gem` to install gem packages.
* When a bundled program is executed (via `bundle exec` or a
  binstub that does `require 'bundler/setup'`), the setup process reads
  the Gemfile.lock, activates the dependencies, re-serializes the lock
  file it read earlier, and then attempts to overwrite the Gemfile.lock
  if the contents aren't bit-identical. I think the reasoning is that
  by merely running an application with a newer version of Bundler, you'll
  automatically keep the Gemfile.lock up-to-date with any changes in the
  format. Unfortunately, that doesn't play well with any form of
  packaging, because bundler will immediately cause the application to
  abort when it attempts to write to the read-only Gemfile.lock in the
  store. We work around this by normalizing the Gemfile.lock with the
  version of Bundler that we'll use at runtime before we copy it into
  the store. This feels fragile, but it's the best we can do without
  changes upstream, or resorting to more delicate hacks.

With all of the challenges in using Bundler, one might wonder why we
can't just cut Bundler out of the picture and use RubyGems. After all,
Nix provides most of the isolation that Bundler is used for anyway.

The problem, however, is that almost every Rails application calls
`Bundler::require` at startup (by way of the default project templates).
Because bundler will then, by default, `require` each gem listed in the
Gemfile, Rails applications are almost always written such that none of
the source files explicitly require their dependencies. That leaves us
with two options: support and use Bundler, or maintain massive patches
for every Rails application that we package.

Closes #8612
This commit is contained in:
Charles Strahan 2015-11-14 21:17:29 -05:00
parent 0e07172c6d
commit b6c06e216b
26 changed files with 646 additions and 887 deletions

View File

@ -8,8 +8,6 @@ bundlerEnv {
lockfile = ./Gemfile.lock; lockfile = ./Gemfile.lock;
gemset = ./gemset.nix; gemset = ./gemset.nix;
buildInputs = [ curl ];
meta = with lib; { meta = with lib; {
description = "Simple, blog aware, static site generator"; description = "Simple, blog aware, static site generator";
homepage = http://jekyllrb.com/; homepage = http://jekyllrb.com/;

View File

@ -1,6 +1,6 @@
{ stdenv, buildEnv, fetchgit, fetchurl, makeWrapper, bundlerEnv, bundler_HEAD { stdenv, buildEnv, fetchgit, fetchurl, makeWrapper, bundlerEnv, bundler
, ruby, libxslt, libxml2, sqlite, openssl, docker , ruby, libxslt, libxml2, sqlite, openssl, docker
, dataDir ? "/var/lib/panamax-api" }: , dataDir ? "/var/lib/panamax-api" }@args:
with stdenv.lib; with stdenv.lib;
@ -14,9 +14,9 @@ stdenv.mkDerivation rec {
gemset = ./gemset.nix; gemset = ./gemset.nix;
gemfile = ./Gemfile; gemfile = ./Gemfile;
lockfile = ./Gemfile.lock; lockfile = ./Gemfile.lock;
buildInputs = [ openssl ];
}; };
bundler = bundler_HEAD.override { inherit ruby; };
bundler = args.bundler.override { inherit ruby; };
database_yml = builtins.toFile "database.yml" '' database_yml = builtins.toFile "database.yml" ''
production: production:

View File

@ -1,5 +1,5 @@
{ stdenv, fetchgit, fetchurl, makeWrapper, bundlerEnv, bundler_HEAD { stdenv, fetchgit, fetchurl, makeWrapper, bundlerEnv, bundler
, ruby, rubygemsFun, openssl, sqlite, dataDir ? "/var/lib/panamax-ui"}: , ruby, openssl, sqlite, dataDir ? "/var/lib/panamax-ui"}@args:
with stdenv.lib; with stdenv.lib;
@ -13,10 +13,9 @@ stdenv.mkDerivation rec {
gemset = ./gemset.nix; gemset = ./gemset.nix;
gemfile = ./Gemfile; gemfile = ./Gemfile;
lockfile = ./Gemfile.lock; lockfile = ./Gemfile.lock;
buildInputs = [ openssl ];
}; };
bundler = bundler_HEAD.override { inherit ruby; }; bundler = args.bundler.override { inherit ruby; };
src = fetchgit { src = fetchgit {
rev = "refs/tags/v${version}"; rev = "refs/tags/v${version}";

View File

@ -0,0 +1,208 @@
# This builds gems in a way that is compatible with bundler.
#
# Bundler installs gems from git sources _very_ differently from how RubyGems
# installes gem packages, though they both install gem packages similarly.
#
# We monkey-patch Bundler to remove any impurities and then drive its internals
# to install git gems.
#
# For the sake of simplicity, gem packages are installed with the standard `gem`
# program.
#
# Note that bundler does not support multiple prefixes; it assumes that all
# gems are installed in a common prefix, and has no support for specifying
# otherwise. Therefore, if you want to be able to use the resulting derivations
# with bundler, you need to create a symlink forrest first, which is what
# `bundlerEnv` does for you.
#
# Normal gem packages can be used outside of bundler; a binstub is created in
# $out/bin.
{ lib, ruby, rubygems, bundler, fetchurl, fetchgit, makeWrapper, git, buildRubyGem
} @ defs:
lib.makeOverridable (
{ name ? null
, gemName
, version ? null
, type ? "gem"
, document ? [] # e.g. [ "ri" "rdoc" ]
, platform ? "ruby"
, ruby ? defs.ruby
, stdenv ? ruby.stdenv
, namePrefix ? "${ruby.name}" + "-"
, buildInputs ? []
, doCheck ? false
, meta ? {}
, patches ? []
, gemPath ? []
, dontStrip ? true
, remotes ? ["https://rubygems.org"]
# Assume we don't have to build unless strictly necessary (e.g. the source is a
# git checkout).
# If you need to apply patches, make sure to set `dontBuild = false`;
, dontBuild ? true
, propagatedBuildInputs ? []
, propagatedUserEnvPkgs ? []
, buildFlags ? null
, passthru ? {}
, ...} @ attrs:
if ! builtins.elem type [ "git" "gem" ]
then throw "buildRubyGem: don't know how to build a gem of type \"${type}\""
else
let
shellEscape = x: "'${lib.replaceChars ["'"] [("'\\'" + "'")] x}'";
rubygems = (attrs.rubygems or defs.rubygems).override {
inherit ruby;
};
src = attrs.src or (
if type == "gem"
then fetchurl {
urls = map (remote: "${remote}/gems/${gemName}-${version}.gem") remotes;
inherit (attrs) sha256;
} else fetchgit {
inherit (attrs) url rev sha256 fetchSubmodules;
leaveDotGit = true;
}
);
documentFlag =
if document == []
then "-N"
else "--document ${lib.concatStringsSep "," document}";
in
stdenv.mkDerivation (attrs // {
inherit ruby rubygems;
inherit doCheck;
inherit dontBuild;
inherit dontStrip;
inherit type;
buildInputs = [
ruby rubygems makeWrapper
] ++ lib.optionals (type == "git") [ git bundler ]
++ buildInputs;
name = attrs.name or (namePrefix + gemName);
inherit src;
phases = attrs.phases or [ "unpackPhase" "patchPhase" "buildPhase" "installPhase" "fixupPhase" ];
unpackPhase = attrs.unpackPhase or ''
runHook preUnpack
if [[ -f $src && $src == *.gem ]]; then
if [[ -z "$dontBuild" ]]; then
# we won't know the name of the directory that RubyGems creates,
# so we'll just use a glob to find it and move it over.
gempkg="$src"
sourceRoot=source
gem unpack $gempkg --target=container
cp -r container/* $sourceRoot
rm -r container
# copy out the original gemspec, for convenience during patching /
# overrides.
gem specification $gempkg --ruby > original.gemspec
gemspec=$(readlink -f .)/original.gemspec
else
gempkg="$src"
fi
else
# Fall back to the original thing for everything else.
dontBuild=""
preUnpack="" postUnpack="" unpackPhase
fi
runHook postUnpack
'';
buildPhase = attrs.buildPhase or ''
runHook preBuild
if [[ "$type" == "gem" ]]; then
if [[ -z "$gemspec" ]]; then
gemspec="$(find . -name '*.gemspec')"
echo "found the following gemspecs:"
echo "$gemspec"
gemspec="$(echo "$gemspec" | head -n1)"
fi
exec 3>&1
output="$(gem build $gemspec | tee >(cat - >&3))"
exec 3>&-
gempkg=$(echo "$output" | grep -oP 'File: \K(.*)')
echo "gem package built: $gempkg"
fi
runHook postBuild
'';
# Note:
# We really do need to keep the $out/${ruby.gemPath}/cache.
# This is very important in order for many parts of RubyGems/Bundler to not blow up.
# See https://github.com/bundler/bundler/issues/3327
installPhase = attrs.installPhase or ''
runHook preInstall
export GEM_HOME=$out/${ruby.gemPath}
mkdir -p $GEM_HOME
echo "buildFlags: $buildFlags"
${lib.optionalString (type == "git") ''
ruby ${./nix-bundle-install.rb} \
${gemName} \
${attrs.url} \
${src} \
${attrs.rev} \
${version} \
${shellEscape (toString buildFlags)}
''}
${lib.optionalString (type == "gem") ''
if [[ -z "$gempkg" ]]; then
echo "failure: \$gempkg path unspecified" 1>&2
exit 1
elif [[ ! -f "$gempkg" ]]; then
echo "failure: \$gempkg path invalid" 1>&2
exit 1
fi
gem install \
--local \
--force \
--http-proxy 'http://nodtd.invalid' \
--ignore-dependencies \
--build-root '/' \
--backtrace \
${documentFlag} \
$gempkg $gemFlags -- $buildFlags
# looks like useless files which break build repeatability and consume space
rm -fv $out/${ruby.gemPath}/doc/*/*/created.rid || true
rm -fv $out/${ruby.gemPath}/gems/*/ext/*/mkmf.log || true
# write out metadata and binstubs
spec=$(echo $out/${ruby.gemPath}/specifications/*.gemspec)
ruby ${./gem-post-build.rb} "$spec"
''}
runHook postInstall
'';
propagatedBuildInputs = gemPath ++ propagatedBuildInputs;
propagatedUserEnvPkgs = gemPath ++ propagatedUserEnvPkgs;
passthru = passthru // { isRubyGem = true; };
inherit meta;
})
)

View File

@ -0,0 +1,77 @@
require 'rbconfig'
require 'rubygems'
require 'rubygems/specification'
require 'fileutils'
ruby = File.join(ENV["ruby"], "bin", RbConfig::CONFIG['ruby_install_name'])
out = ENV["out"]
bin_path = File.join(ENV["out"], "bin")
gem_home = ENV["GEM_HOME"]
gem_path = ENV["GEM_PATH"].split(":")
install_path = Dir.glob("#{gem_home}/gems/*").first
gemspec_path = ARGV[0]
if defined?(Encoding.default_internal)
Encoding.default_internal = Encoding::UTF_8
Encoding.default_external = Encoding::UTF_8
end
gemspec_content = File.read(gemspec_path)
spec = nil
if gemspec_content[0..2] == "---" # YAML header
spec = Gem::Specification.from_yaml(gemspec_content)
else
spec = Gem::Specification.load(gemspec_path)
end
FileUtils.mkdir_p("#{out}/nix-support")
# write meta-data
meta = "#{out}/nix-support/gem-meta"
FileUtils.mkdir_p(meta)
FileUtils.ln_s(gemspec_path, "#{meta}/spec")
File.open("#{meta}/name", "w") do |f|
f.write(spec.name)
end
File.open("#{meta}/install-path", "w") do |f|
f.write(install_path)
end
File.open("#{meta}/require-paths", "w") do |f|
f.write(spec.require_paths.join(" "))
end
File.open("#{meta}/executables", "w") do |f|
f.write(spec.executables.join(" "))
end
# add this gem to the GEM_PATH for dependencies
File.open("#{out}/nix-support/setup-hook", "a") do |f|
f.puts("addToSearchPath GEM_PATH #{gem_home}")
spec.require_paths.each do |dir|
f.puts("addToSearchPath RUBYLIB #{install_path}/#{dir}")
end
end
# create regular rubygems binstubs
FileUtils.mkdir_p(bin_path)
spec.executables.each do |exe|
File.open("#{bin_path}/#{exe}", "w") do |f|
f.write(<<-EOF)
#!#{ruby}
#
# This file was generated by Nix.
#
# The application '#{exe}' is installed as part of a gem, and
# this file is here to facilitate running it.
#
gem_path = ENV["GEM_PATH"]
ENV["GEM_PATH"] = "\#{gem_path}\#{":" unless gem_path.nil? || gem_path.empty?}#{(gem_path+[gem_home]).join(":")}"
require 'rubygems'
load Gem.bin_path(#{spec.name.inspect}, #{exe.inspect})
EOF
end
FileUtils.chmod("+x", "#{bin_path}/#{exe}")
end

View File

@ -0,0 +1,153 @@
require 'rbconfig'
require 'bundler/vendored_thor'
require 'bundler'
require 'rubygems/command'
require 'fileutils'
require 'pathname'
require 'tmpdir'
# Options:
#
# name - the gem name
# uri - git repo uri
# repo - path to local checkout
# ref - the commit hash
# version - gem version
# build-flags - build arguments
ruby = File.join(ENV["ruby"], "bin", RbConfig::CONFIG['ruby_install_name'])
out = ENV["out"]
bin_dir = File.join(ENV["out"], "bin")
name = ARGV[0]
uri = ARGV[1]
REPO = ARGV[2]
ref = ARGV[3]
version = ARGV[4]
build_flags = ARGV[5]
# options to pass to bundler
options = {
"name" => name,
"uri" => uri,
"ref" => ref,
"version" => version,
}
# Monkey-patch Bundler to use our local checkout.
# I wish we didn't have to do this, but bundler does not expose an API to do
# these kinds of things.
Bundler.module_eval do
def self.requires_sudo?
false
end
def self.root
# we don't have a Gemfile, so it doesn't make sense to try to make paths
# relative to the (non existent) parent directory thereof, so we give a
# nonsense path here.
Pathname.new("/no-root-path")
end
def self.bundle_path
Pathname.new(ENV["GEM_HOME"])
end
def self.locked_gems
nil
end
end
Bundler::Source::Git.class_eval do
def allow_git_ops?
true
end
end
Bundler::Source::Git::GitProxy.class_eval do
def checkout
unless path.exist?
FileUtils.mkdir_p(path.dirname)
FileUtils.cp_r(File.join(REPO, ".git"), path)
system("chmod -R +w #{path}")
end
end
def copy_to(destination, submodules=false)
unless File.exist?(destination.join(".git"))
FileUtils.mkdir_p(destination.dirname)
FileUtils.cp_r(REPO, destination)
system("chmod -R +w #{destination}")
end
end
end
# UI
verbose = false
no_color = false
Bundler.ui = Bundler::UI::Shell.new({"no-color" => no_color})
Bundler.ui.level = "debug" if verbose
# Install
source = Bundler::Source::Git.new(options)
spec = source.specs.search_all(name).first
Bundler.rubygems.with_build_args [build_flags] do
source.install(spec)
end
msg = spec.post_install_message
if msg
Bundler.ui.confirm "Post-install message from #{name}:"
Bundler.ui.info msg
end
# Write out the binstubs
if spec.executables.any?
FileUtils.mkdir_p(bin_dir)
spec.executables.each do |exe|
wrapper = File.join(bin_dir, exe)
File.open(wrapper, "w") do |f|
stub = generate_stub(spec.name, exe)
f.write(<<-EOF)
#!#{ruby}
#
# This file was generated by Nix.
#
# The application '#{exe}' is installed as part of a gem, and
# this file is here to facilitate running it.
#
require 'rubygems'
require 'bundler/setup'
load Gem.bin_path(#{spec.name.inspect}, #{exe.inspect})
EOF
end
FileUtils.chmod("+x", wrapper)
end
end
# Write out metadata
meta = "#{out}/nix-support/gem-meta"
FileUtils.mkdir_p(meta)
FileUtils.ln_s(spec.loaded_from.to_s, "#{meta}/spec")
File.open("#{meta}/name", "w") do |f|
f.write spec.name
end
File.open("#{meta}/install-path", "w") do |f|
f.write source.install_path.to_s
end
File.open("#{meta}/require-paths", "w") do |f|
f.write spec.require_paths.join(" ")
end
File.open("#{meta}/executables", "w") do |f|
f.write spec.executables.join(" ")
end
# make the lib available during bundler/git installs
File.open("#{out}/nix-support/setup-hook", "a") do |f|
spec.require_paths.each do |dir|
f.puts("addToSearchPath RUBYLIB #{source.install_path}/#{dir}")
end
end

View File

@ -1,4 +0,0 @@
source "http://rubygems.org"
gem "bundix",
:git => "https://github.com/cstrahan/bundix.git",
:ref => "v1.0.3"

View File

@ -1,18 +0,0 @@
GIT
remote: https://github.com/cstrahan/bundix.git
revision: c879cf901ff8084b4c97aaacfb5ecbdb0f95cc03
ref: v1.0.3
specs:
bundix (1.0.2)
thor (~> 0.19.1)
GEM
remote: http://rubygems.org/
specs:
thor (0.19.1)
PLATFORMS
ruby
DEPENDENCIES
bundix!

View File

@ -1,9 +1,20 @@
{ ruby, bundlerEnv }: { ruby, fetchgit, buildRubyGem, bundler }:
bundlerEnv { let
name = "bundix"; thor = buildRubyGem {
inherit ruby; gemName = "thor";
gemset = ./gemset.nix; version = "0.19.1";
gemfile = ./Gemfile; type = "gem";
lockfile = ./Gemfile.lock; sha256 = "08p5gx18yrbnwc6xc0mxvsfaxzgy2y9i78xq7ds0qmdm67q39y4z";
};
in buildRubyGem {
gemName = "bundix";
version = "1.0.4";
gemPath = [ thor bundler ];
src = fetchgit {
url = "https://github.com/cstrahan/bundix.git";
rev = "6dcf1f71c61584f5c9b919ee9df7b0c554862076";
sha256 = "1w17bvc9srcgr4ry81ispcj35g9kxihbyknmqp8rnd4h5090b7b2";
};
} }

View File

@ -1,22 +0,0 @@
{
"bundix" = {
version = "1.0.2";
source = {
type = "git";
url = "https://github.com/cstrahan/bundix.git";
rev = "c879cf901ff8084b4c97aaacfb5ecbdb0f95cc03";
sha256 = "05kmdnq4qa5h8l3asv05cjpnyplnqqx6hrqybj2cjlzmdxnkkgyj";
fetchSubmodules = false;
};
dependencies = [
"thor"
];
};
"thor" = {
version = "0.19.1";
source = {
type = "gem";
sha256 = "08p5gx18yrbnwc6xc0mxvsfaxzgy2y9i78xq7ds0qmdm67q39y4z";
};
};
}

View File

@ -1,312 +1,75 @@
{ stdenv, runCommand, writeText, writeScript, writeScriptBin, ruby, lib { stdenv, runCommand, writeText, writeScript, writeScriptBin, ruby, lib
, callPackage, defaultGemConfig, fetchurl, fetchgit, buildRubyGem , bundler_HEAD , callPackage, defaultGemConfig, fetchurl, fetchgit, buildRubyGem, buildEnv
, rubygems
, git , git
, makeWrapper
, bundler
, tree
}@defs: }@defs:
# This is a work-in-progress.
# The idea is that his will replace load-ruby-env.nix.
{ name, gemset, gemfile, lockfile, ruby ? defs.ruby, gemConfig ? defaultGemConfig { name, gemset, gemfile, lockfile, ruby ? defs.ruby, gemConfig ? defaultGemConfig
, enableParallelBuilding ? false # TODO: this might not work, given the env-var shinanigans. , postBuild ? null
, postInstall ? null , document ? []
, documentation ? false
, meta ? {} , meta ? {}
, ignoreCollisions ? false
, ... , ...
}@args: }@args:
let let
shellEscape = x: "'${lib.replaceChars ["'"] [("'\\'" + "'")] x}'"; shellEscape = x: "'${lib.replaceChars ["'"] [("'\\'" + "'")] x}'";
const = x: y: x; importedGemset = import gemset;
bundler = bundler_HEAD.override { inherit ruby; };
inherit (builtins) attrValues;
gemName = attrs: "${attrs.name}-${attrs.version}.gem";
fetchers.path = attrs: attrs.source.path;
fetchers.gem = attrs: fetchurl {
url = "${attrs.source.source or "https://rubygems.org"}/downloads/${gemName attrs}";
inherit (attrs.source) sha256;
};
fetchers.git = attrs: fetchgit {
inherit (attrs.source) url rev sha256 fetchSubmodules;
leaveDotGit = true;
};
applySrc = attrs:
attrs // {
src = (fetchers."${attrs.source.type}" attrs);
};
applyGemConfigs = attrs: applyGemConfigs = attrs:
if gemConfig ? "${attrs.name}" (if gemConfig ? "${attrs.gemName}"
then attrs // gemConfig."${attrs.name}" attrs then attrs // gemConfig."${attrs.gemName}" attrs
else attrs; else attrs);
configuredGemset = lib.flip lib.mapAttrs importedGemset (name: attrs:
needsPatch = attrs: applyGemConfigs (attrs // { gemName = name; })
(attrs ? patches) || (attrs ? prePatch) || (attrs ? postPatch);
# patch a gem or source tree.
# for gems, the gem is unpacked, patched, and then repacked.
# see: https://github.com/fedora-ruby/gem-patch/blob/master/lib/rubygems/patcher.rb
applyPatches = attrs:
if !needsPatch attrs
then attrs
else attrs // { src =
stdenv.mkDerivation {
name = gemName attrs;
phases = [ "unpackPhase" "patchPhase" "installPhase" ];
buildInputs = [ ruby ] ++ attrs.buildInputs or [];
patches = attrs.patches or [ ];
prePatch = attrs.prePatch or "true";
postPatch = attrs.postPatch or "true";
unpackPhase = ''
runHook preUnpack
if [[ -f ${attrs.src} ]]; then
isGem=1
# we won't know the name of the directory that RubyGems creates,
# so we'll just use a glob to find it and move it over.
gem unpack ${attrs.src} --target=container
cp -r container/* contents
rm -r container
else
cp -r ${attrs.src} contents
chmod -R +w contents
fi
cd contents
runHook postUnpack
'';
installPhase = ''
runHook preInstall
if [[ -n "$isGem" ]]; then
${writeScript "repack.rb" ''
#!${ruby}/bin/ruby
require 'rubygems'
require 'rubygems/package'
require 'fileutils'
Encoding.default_internal = Encoding::UTF_8
Encoding.default_external = Encoding::UTF_8
if Gem::VERSION < '2.0'
load "${./package-1.8.rb}"
end
out = ENV['out']
files = Dir['**/{.[^\.]*,*}']
package = Gem::Package.new("${attrs.src}")
patched_package = Gem::Package.new(package.spec.file_name)
patched_package.spec = package.spec.clone
patched_package.spec.files = files
patched_package.build(false)
FileUtils.cp(patched_package.spec.file_name, out)
''}
else
cp -r . $out
fi
runHook postInstall
'';
};
};
instantiate = (attrs:
applyPatches (applyGemConfigs (applySrc attrs))
); );
hasBundler = builtins.hasAttr "bundler" importedGemset;
bundler = if hasBundler then gems.bundler else defs.bundler.override (attrs: { inherit ruby; });
rubygems = defs.rubygems.override (attrs: { inherit ruby; });
gems = lib.flip lib.mapAttrs configuredGemset (name: attrs:
buildRubyGem ((removeAttrs attrs ["source"]) // attrs.source // {
inherit ruby rubygems;
gemName = name;
gemPath = map (gemName: gems."${gemName}") (attrs.dependencies or []);
}));
# We have to normalize the Gemfile.lock, otherwise bundler tries to be
# helpful by doing so at run time, causing executables to immediately bail
# out. Yes, I'm serious.
confFiles = runCommand "gemfile-and-lockfile" {} ''
mkdir -p $out
cp ${gemfile} $out/Gemfile
cp ${lockfile} $out/Gemfile.lock
instantiated = lib.flip lib.mapAttrs (import gemset) (name: attrs: cd $out
instantiate (attrs // { inherit name; }) chmod +w Gemfile.lock
); source ${rubygems}/nix-support/setup-hook
export GEM_PATH=${bundler}/${ruby.gemPath}
needsPreInstall = attrs: ${ruby}/bin/ruby -rubygems -e \
(attrs ? preInstall) || (attrs ? buildInputs) || (attrs ? nativeBuildInputs); "require 'bundler'; Bundler.definition.lock('Gemfile.lock')"
# TODO: support cross compilation? look at stdenv/generic/default.nix.
runPreInstallers = lib.fold (next: acc:
if !needsPreInstall next
then acc
else acc + ''
${writeScript "${next.name}-pre-install" ''
#!${stdenv.shell}
export nativeBuildInputs="${toString ((next.nativeBuildInputs or []) ++ (next.buildInputs or []))}"
source ${stdenv}/setup
header "running pre-install script for ${next.name}"
${next.preInstall or ""}
${ruby}/bin/ruby -e 'print ENV.inspect' > env/${next.name}
stopNest
''}
''
) "" (attrValues instantiated);
# copy *.gem to ./gems
copyGems = lib.fold (next: acc:
if next.source.type == "gem"
then acc + "cp ${next.src} gems/${gemName next}\n"
else acc
) "" (attrValues instantiated);
runRuby = name: env: command:
runCommand name env ''
${ruby}/bin/ruby ${writeText name command}
''; '';
envPaths = lib.attrValues gems ++ lib.optional (!hasBundler) bundler;
bundlerEnv = buildEnv {
inherit name ignoreCollisions;
paths = envPaths;
pathsToLink = [ "/lib" ];
postBuild = ''
source ${rubygems}/nix-support/setup-hook
# TODO: include json_pure, so the version of ruby doesn't matter. ${ruby}/bin/ruby ${./gen-bin-stubs.rb} \
# not all rubies have support for JSON built-in, "${ruby}/bin/ruby" \
# so we'll convert JSON to ruby expressions. "${confFiles}/Gemfile" \
json2rb = writeScript "json2rb" '' "$out/${ruby.gemPath}" \
#!${ruby}/bin/ruby "${bundler}/${ruby.gemPath}" \
begin ${shellEscape (toString envPaths)}
require 'json' '' + lib.optionalString (postBuild != null) postBuild;
rescue LoadError => ex
require 'json_pure'
end
puts JSON.parse(STDIN.read).inspect
'';
# dump the instantiated gemset as a ruby expression.
serializedGemset = runCommand "gemset.rb" { json = builtins.toJSON instantiated; } ''
printf '%s' "$json" | ${json2rb} > $out
'';
# this is a mapping from a source type and identifier (uri/path/etc)
# to the pure store path.
# we'll use this from the patched bundler to make fetching sources pure.
sources = runRuby "sources.rb" { gemset = serializedGemset; } ''
out = ENV['out']
gemset = eval(File.read(ENV['gemset']))
sources = {
"git" => { },
"path" => { },
"gem" => { },
"svn" => { }
}
gemset.each_value do |spec|
type = spec["source"]["type"]
val = spec["src"]
key =
case type
when "gem"
spec["name"]
when "git"
spec["source"]["url"]
when "path"
spec["source"]["originalPath"]
when "svn"
nil # TODO
end
sources[type][key] = val if key
end
File.open(out, "wb") do |f|
f.print sources.inspect
end
'';
# rewrite PATH sources to point into the nix store.
purifiedLockfile = runRuby "purifiedLockfile" {} ''
out = ENV['out']
sources = eval(File.read("${sources}"))
paths = sources["path"]
lockfile = File.read("${lockfile}")
paths.each_pair do |impure, pure|
lockfile.gsub!(/^ remote: #{Regexp.escape(impure)}/, " remote: #{pure}")
end
File.open(out, "wb") do |f|
f.print lockfile
end
'';
needsBuildFlags = attrs: attrs ? buildFlags;
mkBuildFlags = spec:
"export BUNDLE_BUILD__${lib.toUpper spec.name}='${lib.concatStringsSep " " (map shellEscape spec.buildFlags)}'";
allBuildFlags =
lib.concatStringsSep "\n"
(map mkBuildFlags
(lib.filter needsBuildFlags (attrValues instantiated)));
derivation = stdenv.mkDerivation {
inherit name;
buildInputs = [
ruby
bundler
git
] ++ args.buildInputs or [];
phases = [ "installPhase" "fixupPhase" ];
outputs = [
"out" # the installed libs/bins
"bundle" # supporting files for bundler
];
installPhase = ''
mkdir -p $bundle
export BUNDLE_GEMFILE=$bundle/Gemfile
cp ${gemfile} $BUNDLE_GEMFILE
cp ${purifiedLockfile} $BUNDLE_GEMFILE.lock
export NIX_GEM_SOURCES=${sources}
export NIX_BUNDLER_GEMPATH=${bundler}/${ruby.gemPath}
export GEM_HOME=$out/${ruby.gemPath}
export GEM_PATH=$NIX_BUNDLER_GEMPATH:$GEM_HOME
mkdir -p $GEM_HOME
${allBuildFlags}
mkdir gems
cp ${bundler}/${bundler.ruby.gemPath}/cache/bundler-*.gem gems
${copyGems}
${lib.optionalString (!documentation) ''
mkdir home
HOME="$(pwd -P)/home"
echo "gem: --no-rdoc --no-ri" > $HOME/.gemrc
''}
mkdir env
${runPreInstallers}
mkdir $out/bin
cp ${./monkey_patches.rb} monkey_patches.rb
export RUBYOPT="-rmonkey_patches.rb -I $(pwd -P)"
bundler install --frozen --binstubs ${lib.optionalString enableParallelBuilding "--jobs $NIX_BUILD_CORES"}
RUBYOPT=""
runHook postInstall
'';
inherit postInstall;
passthru = { passthru = {
inherit ruby; inherit ruby bundler meta gems;
inherit bundler;
env = let env = let
irbrc = builtins.toFile "irbrc" '' irbrc = builtins.toFile "irbrc" ''
if not ENV["OLD_IRBRC"].empty? if !(ENV["OLD_IRBRC"].nil? || ENV["OLD_IRBRC"].empty?)
require ENV["OLD_IRBRC"] require ENV["OLD_IRBRC"]
end end
require 'rubygems' require 'rubygems'
@ -314,12 +77,12 @@ let
''; '';
in stdenv.mkDerivation { in stdenv.mkDerivation {
name = "interactive-${name}-environment"; name = "interactive-${name}-environment";
nativeBuildInputs = [ ruby derivation ]; nativeBuildInputs = [ ruby bundlerEnv ];
shellHook = '' shellHook = ''
export BUNDLE_GEMFILE=${derivation.bundle}/Gemfile export BUNDLE_GEMFILE=${confFiles}/Gemfile
export GEM_HOME=${derivation}/${ruby.gemPath} export BUNDLE_PATH=${bundlerEnv}/${ruby.gemPath}
export NIX_BUNDLER_GEMPATH=${bundler}/${ruby.gemPath} export GEM_HOME=${bundlerEnv}/${ruby.gemPath}
export GEM_PATH=$NIX_BUNDLER_GEMPATH:$GEM_HOME export GEM_PATH=${bundlerEnv}/${ruby.gemPath}
export OLD_IRBRC="$IRBRC" export OLD_IRBRC="$IRBRC"
export IRBRC=${irbrc} export IRBRC=${irbrc}
''; '';
@ -331,8 +94,8 @@ let
''; '';
}; };
}; };
inherit meta;
}; };
in derivation in
bundlerEnv

View File

@ -0,0 +1,46 @@
require 'rbconfig'
require 'rubygems'
require 'rubygems/specification'
require 'fileutils'
# args/settings
out = ENV["out"]
ruby = ARGV[0]
gemfile = ARGV[1]
bundle_path = ARGV[2]
bundler_gem_path = ARGV[3]
paths = ARGV[4].split
# generate binstubs
FileUtils.mkdir_p("#{out}/bin")
paths.each do |path|
next unless File.directory?("#{path}/nix-support/gem-meta")
name = File.read("#{path}/nix-support/gem-meta/name")
executables = File.read("#{path}/nix-support/gem-meta/executables").split
executables.each do |exe|
File.open("#{out}/bin/#{exe}", "w") do |f|
f.write(<<-EOF)
#!#{ruby}
#
# This file was generated by Nix.
#
# The application '#{exe}' is installed as part of a gem, and
# this file is here to facilitate running it.
#
ENV["BUNDLE_GEMFILE"] = "#{gemfile}"
ENV["BUNDLE_PATH"] = "#{bundle_path}"
gem_path = ENV["GEM_PATH"]
ENV["GEM_PATH"] = "\#{gem_path}\#{":" unless gem_path.nil? || gem_path.empty?}#{bundler_gem_path}"
require 'rubygems'
require 'bundler/setup'
load Gem.bin_path(#{name.inspect}, #{exe.inspect})
EOF
FileUtils.chmod("+x", "#{out}/bin/#{exe}")
end
end
end

View File

@ -1,246 +0,0 @@
require 'bundler'
# Undo the RUBYOPT trickery.
opt = ENV['RUBYOPT'].dup
opt.gsub!(/-rmonkey_patches.rb -I [^ ]*/, '')
ENV['RUBYOPT'] = opt
Bundler.module_eval do
class << self
# mappings from original uris to store paths.
def nix_gem_sources
@nix_gem_sources ||=
begin
src = ENV['NIX_GEM_SOURCES']
eval(Bundler.read_file(src))
end
end
# extract the gemspecs from the gems pulled from Rubygems.
def nix_gemspecs
@nix_gemspecs ||= Dir.glob("gems/*.gem").map do |path|
Bundler.rubygems.spec_from_gem(path)
end
end
# swap out ENV
def nix_with_env(env, &block)
if env
old_env = ENV.to_hash
begin
ENV.replace(env)
block.call
ensure
ENV.replace(old_env)
end
else
block.call
end
end
# map a git uri to a fetchgit store path.
def nix_git(uri)
Pathname.new(nix_gem_sources["git"][uri])
end
end
end
Bundler::Source::Git::GitProxy.class_eval do
def checkout
unless path.exist?
FileUtils.mkdir_p(path.dirname)
FileUtils.cp_r(Bundler.nix_git(@uri).join(".git"), path)
system("chmod -R +w #{path}")
end
end
def copy_to(destination, submodules=false)
unless File.exist?(destination.join(".git"))
FileUtils.mkdir_p(destination.dirname)
FileUtils.cp_r(Bundler.nix_git(@uri), destination)
system("chmod -R +w #{destination}")
end
end
end
Bundler::Fetcher.class_eval do
def use_api
true
end
def fetch_dependency_remote_specs(gem_names)
Bundler.ui.debug "Query Gemcutter Dependency Endpoint API: #{gem_names.join(',')}"
deps_list = []
spec_list = gem_names.map do |name|
spec = Bundler.nix_gemspecs.detect {|spec| spec.name == name }
if spec.nil?
msg = "WARNING: Could not find gemspec for '#{name}'"
Bundler.ui.warn msg
nil
else
dependencies = spec.dependencies.
select {|dep| dep.type != :development}.
map do |dep|
deps_list << dep.name
dep
end
[spec.name, spec.version, spec.platform, dependencies]
end
end
spec_list.compact!
[spec_list, deps_list.uniq]
end
end
Bundler::Source::Rubygems.class_eval do
# We copy all gems into $PWD/gems, and this allows RubyGems to find those
# gems during installation.
def fetchers
@fetchers ||= [
Bundler::Fetcher.new(URI.parse("file://#{File.expand_path(Dir.pwd)}"))
]
end
# Look-up gems that were originally from RubyGems.
def remote_specs
@remote_specs ||=
begin
lockfile = Bundler::LockfileParser.new(Bundler.read_file(Bundler.default_lockfile))
gem_names = lockfile.specs.
select {|spec| spec.source.is_a?(Bundler::Source::Rubygems)}.
map {|spec| spec.name}
idx = Bundler::Index.new
api_fetchers.each do |f|
Bundler.ui.info "Fetching source index from #{f.uri}"
idx.use f.specs(gem_names, self)
end
idx
end
end
end
Bundler::Installer.class_eval do
# WHY:
# This allows us to provide a typical Nix experience, where
# `buildInputs` and/or `preInstall` may set up the $PATH and other env-vars
# as needed. By swapping out the environment per install, we can have finer
# grained control than we would have otherwise.
#
# HOW:
# This is a wrapper around the original `install_gem_from_spec`.
# We expect that a "pre-installer" might exist at `pre-installers/<gem-name>`,
# and if it does, we execute it.
# The pre-installer is expected to dump its environment variables as a Ruby
# hash to `env/<gem-name>`.
# We then swap out the environment for the duration of the install,
# and then set it back to what it was originally.
alias original_install_gem_from_spec install_gem_from_spec
def install_gem_from_spec(spec, standalone = false, worker = 0)
env_dump = "env/#{spec.name}"
if File.exist?(env_dump)
env = eval(Bundler.read_file(env_dump))
unless env
Bundler.ui.error "The environment variables for #{spec.name} could not be loaded!"
exit 1
end
Bundler.nix_with_env(env) do
original_install_gem_from_spec(spec, standalone, worker)
end
else
original_install_gem_from_spec(spec, standalone, worker)
end
end
def generate_bundler_executable_stubs(spec, options = {})
return if spec.executables.empty?
out = ENV['out']
spec.executables.each do |executable|
next if executable == "bundle" || executable == "bundler"
binstub_path = "#{out}/bin/#{executable}"
File.open(binstub_path, 'w', 0777 & ~File.umask) do |f|
f.print <<-TEXT
#!#{RbConfig.ruby}
old_gemfile = ENV["BUNDLE_GEMFILE"]
old_gem_home = ENV["GEM_HOME"]
old_gem_path = ENV["GEM_PATH"]
ENV["BUNDLE_GEMFILE"] =
"#{ENV["BUNDLE_GEMFILE"]}"
ENV["GEM_HOME"] =
"#{ENV["GEM_HOME"]}"
ENV["GEM_PATH"] =
"#{ENV["NIX_BUNDLER_GEMPATH"]}:\#{ENV["GEM_HOME"]}\#{old_gem_path ? ":\#{old_gem_path}" : ""}}"
require 'rubygems'
require 'bundler/setup'
ENV["BUNDLE_GEMFILE"] = old_gemfile
ENV["GEM_HOME"] = old_gem_home
ENV["GEM_PATH"] = old_gem_path
load Gem.bin_path('#{spec.name}', '#{executable}')
TEXT
end
end
end
end
Gem::Installer.class_eval do
# Make the wrappers automagically use bundler.
#
# Stage 1.
# Set $BUNDLE_GEMFILE so bundler knows what gems to load.
# Set $GEM_HOME to the installed gems, because bundler looks there for
# non-Rubygems installed gems (e.g. git/svn/path sources).
# Set $GEM_PATH to include both bundler and installed gems.
#
# Stage 2.
# Setup bundler, locking down the gem versions.
#
# Stage 3.
# Reset $BUNDLE_GEMFILE, $GEM_HOME, $GEM_PATH.
#
# Stage 4.
# Run the actual executable.
def app_script_text(bin_file_name)
return <<-TEXT
#!#{RbConfig.ruby}
#
# This file was generated by Nix's RubyGems.
#
# The application '#{spec.name}' is installed as part of a gem, and
# this file is here to facilitate running it.
#
old_gemfile = ENV["BUNDLE_GEMFILE"]
old_gem_home = ENV["GEM_HOME"]
old_gem_path = ENV["GEM_PATH"]
ENV["BUNDLE_GEMFILE"] =
"#{ENV["BUNDLE_GEMFILE"]}"
ENV["GEM_HOME"] =
"#{ENV["GEM_HOME"]}"
ENV["GEM_PATH"] =
"#{ENV["NIX_BUNDLER_GEMPATH"]}:\#{ENV["GEM_HOME"]}\#{old_gem_path ? ":\#{old_gem_path}" : ""}}"
require 'rubygems'
require 'bundler/setup'
ENV["BUNDLE_GEMFILE"] = old_gemfile
ENV["GEM_HOME"] = old_gem_home
ENV["GEM_PATH"] = old_gem_path
load Gem.bin_path('#{spec.name}', '#{bin_file_name}')
TEXT
end
end

View File

@ -1,29 +0,0 @@
require 'rubygems/installer'
require 'rubygems/builder'
# Simulate RubyGems 2.0 behavior.
module Gem::Package
def self.new(gem)
@gem = gem
self
end
def self.extract_files(dir)
installer = Gem::Installer.new @gem
installer.unpack(dir)
end
def self.build(skip_validation=false)
builder = Gem::Builder.new(spec)
builder.build
end
def self.spec=(spec)
@spec = spec
end
def self.spec
@spec ||= Gem::Installer.new(@gem).spec
end
end

View File

@ -1,17 +1,10 @@
{ buildRubyGem, makeWrapper, ruby, coreutils }: { buildRubyGem, makeWrapper, ruby, coreutils }:
buildRubyGem { buildRubyGem rec {
name = "bundler-1.10.6"; inherit ruby;
namePrefix = ""; name = "${gemName}-${version}";
gemName = "bundler";
version = "1.10.6";
sha256 = "1vlzfq0bkkj4jyq6av0y55mh5nj5n0f3mfbmmifwgkh44g8k6agv"; sha256 = "1vlzfq0bkkj4jyq6av0y55mh5nj5n0f3mfbmmifwgkh44g8k6agv";
dontPatchShebangs = true; dontPatchShebangs = true;
postInstall = ''
find $out -type f -perm -0100 | while read f; do
substituteInPlace $f \
--replace "/usr/bin/env" "${coreutils}/bin/env"
done
wrapProgram $out/bin/bundler \
--prefix PATH ":" ${ruby}/bin
'';
} }

View File

@ -1,136 +0,0 @@
{ lib, ruby, rubygemsFun, fetchurl, makeWrapper, git } @ defs:
lib.makeOverridable (
{ name
, ruby ? defs.ruby
, rubygems ? (rubygemsFun ruby)
, stdenv ? ruby.stdenv
, namePrefix ? "${lib.replaceStrings ["-"] ["_"] ruby.name}" + "-"
, buildInputs ? []
, doCheck ? false
, dontBuild ? true
, meta ? {}
, gemPath ? []
, ...} @ attrs:
stdenv.mkDerivation (attrs // {
inherit ruby rubygems;
inherit doCheck;
buildInputs = [ ruby rubygems makeWrapper git ] ++ buildInputs;
name = namePrefix + name;
src = if attrs ? src
then attrs.src
else fetchurl {
url = "http://rubygems.org/downloads/${attrs.name}.gem";
inherit (attrs) sha256;
};
phases = [ "unpackPhase" "patchPhase" "buildPhase" "checkPhase" "installPhase" "fixupPhase" ];
# The source is expected to either be a gem package or a directory.
#
# - Gem packages are already built, so they don't even need to be unpacked.
# They will skip the buildPhase.
# - A directory containing the sources will need to go through all of the
# usual phases.
unpackPhase= ''
gemRegex="\.gem"
if [[ $src =~ $gemRegex ]]
then
runHook preUnpack
echo "source is a gem package, won't unpack"
gempkg=$src
dontBuild=1
runHook postUnpack
else
# Fall back to the original thing for everything else.
unpackPhase
fi
'';
checkPhase = "true";
buildPhase = ''
runHook preBuild
# TODO: Investigate. The complete working tree is touched by fetchgit.
if [ -d .git ]; then
git reset
fi
gemspec=$(find . -name '*.gemspec')
echo "found the following gemspecs:"
echo "$gemspec"
gemspec=$(echo "$gemspec" | head -n1)
echo "building $gemspec"
exec 3>&1
output=$(gem build $gemspec | tee >(cat - >&3))
exec 3>&-
gempkg=$(echo "$output" | grep -oP 'File: \K(.*)')
echo "gem package built: $gempkg"
runHook postBuild
'';
installPhase = ''
runHook preInstall
# NOTE: This does NOT build the unpacked gem, but installs $src directly.
# Gems that have not been downloaded from rubygems.org may need a
# separate buildPhase.
# --ignore-dependencies is necessary as rubygems otherwise always
# connects to the repository, thus breaking pure builds.
GEM_HOME=$out/${ruby.gemPath} \
gem install \
--local \
--force \
--http-proxy "http://nodtd.invalid" \
--ignore-dependencies \
--build-root "/" \
--backtrace \
$gempkg $gemFlags -- $buildFlags
# Yes, we really do need the $out/${ruby.gemPath}/cache.
# This is very important in order for many parts of RubyGems/Bundler to not blow up.
# See https://github.com/bundler/bundler/issues/3327
mkdir -p $out/bin
for prog in $out/${ruby.gemPath}/gems/*/bin/*; do
makeWrapper $prog $out/bin/$(basename $prog) \
--prefix GEM_PATH : "$out/${ruby.gemPath}:$GEM_PATH" \
--prefix RUBYLIB : "${rubygems}/lib" \
$extraWrapperFlags ''${extraWrapperFlagsArray[@]}
done
#--prefix RUBYOPT rubygems \
# looks like useless files which break build repeatability and consume space
rm -fv $out/${ruby.gemPath}/doc/*/*/created.rid || true
rm -fv $out/${ruby.gemPath}/gems/*/ext/*/mkmf.log || true
mkdir -p $out/nix-support
cat > $out/nix-support/setup-hook <<EOF
if [[ "\$GEM_PATH" != *$out* ]]; then
addToSearchPath GEM_PATH $out/${ruby.gemPath}
fi
EOF
runHook postInstall
'';
propagatedBuildInputs = gemPath;
propagatedUserEnvPkgs = gemPath;
passthru.isRubyGem = true;
inherit meta;
})
)

View File

@ -20,7 +20,7 @@
{ lib, fetchurl, writeScript, ruby, kerberos, libxml2, libxslt, python, stdenv, which { lib, fetchurl, writeScript, ruby, kerberos, libxml2, libxslt, python, stdenv, which
, libiconv, postgresql, v8_3_16_14, clang, sqlite, zlib, imagemagick , libiconv, postgresql, v8_3_16_14, clang, sqlite, zlib, imagemagick
, pkgconfig , ncurses, xapian, gpgme, utillinux, fetchpatch, tzdata, icu, libffi , pkgconfig , ncurses, xapian, gpgme, utillinux, fetchpatch, tzdata, icu, libffi
, cmake, libssh2, openssl, mysql, darwin , cmake, libssh2, openssl, mysql, darwin, git, perl, gecode_3, curl
}: }:
let let
@ -32,6 +32,14 @@ in
buildInputs = [ which icu zlib ]; buildInputs = [ which icu zlib ];
}; };
dep-selector-libgecode = attrs: {
USE_SYSTEM_GECODE = true;
postInstall = ''
installPath=$(cat $out/nix-support/gem-meta/install-path)
sed -i $installPath/lib/dep-selector-libgecode.rb -e 's@VENDORED_GECODE_DIR =.*@VENDORED_GECODE_DIR = "${gecode_3}"@'
'';
};
ffi = attrs: { ffi = attrs: {
buildInputs = [ libffi pkgconfig ]; buildInputs = [ libffi pkgconfig ];
}; };
@ -40,11 +48,12 @@ in
buildInputs = [ gpgme ]; buildInputs = [ gpgme ];
}; };
# note that you need version >= v3.16.14.8,
# otherwise the gem will fail to link to the libv8 binary.
# see: https://github.com/cowboyd/libv8/pull/161
libv8 = attrs: { libv8 = attrs: {
buildInputs = [ which v8 python ]; buildInputs = [ which v8 python ];
buildFlags = [ buildFlags = [ "--with-system-v8=true" ];
"--with-system-v8=true"
];
}; };
mysql2 = attrs: { mysql2 = attrs: {
@ -73,12 +82,20 @@ in
buildInputs = lib.optional stdenv.isDarwin darwin.libobjc; buildInputs = lib.optional stdenv.isDarwin darwin.libobjc;
}; };
patron = attrs: {
buildInputs = [ curl ];
};
pg = attrs: { pg = attrs: {
buildFlags = [ buildFlags = [
"--with-pg-config=${postgresql}/bin/pg_config" "--with-pg-config=${postgresql}/bin/pg_config"
]; ];
}; };
puma = attrs: {
buildInputs = [ openssl ];
};
rmagick = attrs: { rmagick = attrs: {
buildInputs = [ imagemagick pkgconfig ]; buildInputs = [ imagemagick pkgconfig ];
}; };
@ -95,6 +112,7 @@ in
}; };
sup = attrs: { sup = attrs: {
dontBuild = false;
# prevent sup from trying to dynamically install `xapian-ruby`. # prevent sup from trying to dynamically install `xapian-ruby`.
postPatch = '' postPatch = ''
cp ${./mkrf_conf_xapian.rb} ext/mkrf_conf_xapian.rb cp ${./mkrf_conf_xapian.rb} ext/mkrf_conf_xapian.rb
@ -118,6 +136,7 @@ in
}; };
tzinfo = attrs: { tzinfo = attrs: {
dontBuild = false;
postPatch = '' postPatch = ''
substituteInPlace lib/tzinfo/zoneinfo_data_source.rb \ substituteInPlace lib/tzinfo/zoneinfo_data_source.rb \
--replace "/usr/share/zoneinfo" "${tzdata}/share/zoneinfo" --replace "/usr/share/zoneinfo" "${tzdata}/share/zoneinfo"
@ -130,6 +149,7 @@ in
xapian-ruby = attrs: { xapian-ruby = attrs: {
# use the system xapian # use the system xapian
dontBuild = false;
buildInputs = [ xapian pkgconfig zlib ]; buildInputs = [ xapian pkgconfig zlib ];
postPatch = '' postPatch = ''
cp ${./xapian-Rakefile} Rakefile cp ${./xapian-Rakefile} Rakefile

View File

@ -1,69 +0,0 @@
{ ruby, lib, callPackage, gemFixes, fetchurl, fetchgit, buildRubyGem }@defs:
# This function builds a set of gems. You first convert your Gemfile to an attrset
# called a "gemset", and then use this function to build the gemset.
#
# A gemset looks like the following:
#
# {
# libv8 = {
# version = "3.16.14.7";
# src = {
# type = "gem";
# sha256 = "...";
# };
# };
# therubyracer = {
# version = "0.12.1";
# dependencies = [ "libv8" ];
# src = {
# type = "gem";
# sha256 = "...";
# };
# };
# }
#
# If you use these gems as build inputs, the GEM_PATH will be updated
# appropriately, and command like `bundle exec` should work out of the box.
{ gemset, ruby ? defs.ruby, fixes ? gemFixes }@args:
let
const = x: y: x;
fetchers.path = attrs: attrs.src.path;
fetchers.gem = attrs: fetchurl {
url = "${attrs.src.source or "https://rubygems.org"}/downloads/${attrs.name}-${attrs.version}.gem";
inherit (attrs.src) sha256;
};
fetchers.git = attrs: fetchgit {
inherit (attrs.src) url rev sha256 fetchSubmodules;
leaveDotGit = true;
};
instantiate = (attrs:
let
defaultAttrs = {
name = "${attrs.name}-${attrs.version}";
inherit ruby gemPath;
};
gemPath = map (name: gemset''."${name}") (attrs.dependencies or []);
fixedAttrs = attrs // (fixes."${attrs.name}" or (const {})) attrs;
withSource = fixedAttrs //
(if (lib.isDerivation fixedAttrs.src || builtins.isString fixedAttrs.src)
then {}
else { src = (fetchers."${fixedAttrs.src.type}" fixedAttrs); });
in
buildRubyGem (withSource // defaultAttrs)
);
gemset' = if builtins.isAttrs gemset then gemset else import gemset;
gemset'' = lib.flip lib.mapAttrs gemset' (name: attrs:
if (lib.isDerivation attrs)
then attrs
else instantiate (attrs // { inherit name; })
);
in gemset''

View File

@ -1,37 +1,35 @@
args @ { makeWrapper, ruby, ... }: with args; { stdenv, lib, fetchurl, makeWrapper, ruby }:
rec { stdenv.mkDerivation rec {
name = "rubygems-" + version; name = "rubygems-${version}";
version = "2.4.1"; version = "2.4.1";
src = fetchurl { src = fetchurl {
url = "http://production.cf.rubygems.org/rubygems/${name}.tgz"; url = "http://production.cf.rubygems.org/rubygems/${name}.tgz";
sha256 = "0cpr6cx3h74ykpb0cp4p4xg7a8j0bhz3sk271jq69l4mm4zy4h4f"; sha256 = "0cpr6cx3h74ykpb0cp4p4xg7a8j0bhz3sk271jq69l4mm4zy4h4f";
}; };
buildInputs = [ruby makeWrapper]; patches = [ ./gem_hook.patch ];
configureFlags = [];
doInstall = fullDepEntry ('' buildInputs = [ruby makeWrapper];
buildPhase = ":";
installPhase = ''
ruby setup.rb --prefix=$out/ ruby setup.rb --prefix=$out/
wrapProgram $out/bin/gem --prefix RUBYLIB : $out/lib wrapProgram $out/bin/gem --prefix RUBYLIB : $out/lib
find $out -type f -name "*.rb" | xargs sed -i "s@/usr/bin/env@$(type -p env)@g"
find $out -type f -name "*.rb" |
xargs sed -i "s@/usr/bin/env@$(type -p env)@g"
mkdir -pv $out/nix-support mkdir -pv $out/nix-support
cat > $out/nix-support/setup-hook <<EOF cat > $out/nix-support/setup-hook <<EOF
export RUBYOPT=rubygems export RUBYOPT=rubygems
addToSearchPath RUBYLIB $out/lib addToSearchPath RUBYLIB $out/lib
EOF'') ["minInit" "addInputs" "doUnpack" "defEnsureDir"]; EOF
'';
/* doConfigure should be specified separately */
phaseNames = ["doPatch" "doInstall"];
meta = { meta = {
description = "Ruby gems package collection"; description = "A package management framework for Ruby";
longDescription = ''
Nix can create nix packages from gems.
To use it by installing gem-nix package.
'';
}; };
patches = [ ./gem_hook.patch ];
} }

View File

@ -0,0 +1,21 @@
{ stdenv, fetchurl, perl }:
stdenv.mkDerivation rec {
name = "gecode-${version}";
version = "3.7.3";
src = fetchurl {
url = "http://www.gecode.org/download/${name}.tar.gz";
sha256 = "0k45jas6p3cyldgyir1314ja3174sayn2h2ly3z9b4dl3368pk77";
};
buildInputs = [ perl ];
meta = with stdenv.lib; {
license = licenses.mit;
homepage = http://www.gecode.org;
description = "Toolkit for developing constraint-based systems";
platforms = platforms.all;
maintainers = [ maintainers.manveru ];
};
}

View File

@ -7,7 +7,8 @@ let
version = "1.8.0"; version = "1.8.0";
rake = buildRubyGem { rake = buildRubyGem {
inherit ruby; inherit ruby;
name = "rake-10.4.2"; gemName = "rake";
version = "10.4.2";
sha256 = "1rn03rqlf1iv6n87a78hkda2yqparhhaivfjpizblmxvlw2hk5r8"; sha256 = "1rn03rqlf1iv6n87a78hkda2yqparhhaivfjpizblmxvlw2hk5r8";
}; };

View File

@ -1,4 +1,4 @@
{ stdenv, fetchgit, autoconf, automake, libtool, pkgconfig, glib, libdaemon, buildRubyGem { stdenv, fetchgit, autoconf, automake, libtool, pkgconfig, glib, libdaemon
, mpd_clientlib, curl, sqlite, ruby, bundlerEnv, libnotify, pandoc }: , mpd_clientlib, curl, sqlite, ruby, bundlerEnv, libnotify, pandoc }:
let let

View File

@ -8,8 +8,6 @@ bundlerEnv {
lockfile = ./Gemfile.lock; lockfile = ./Gemfile.lock;
gemset = ./gemset.nix; gemset = ./gemset.nix;
buildInputs = [ curl ];
meta = with lib; { meta = with lib; {
description = "A data collector"; description = "A data collector";
homepage = http://www.fluentd.org/; homepage = http://www.fluentd.org/;

View File

@ -5278,11 +5278,9 @@ let
ruby = ruby_2_1_3; ruby = ruby_2_1_3;
}; };
bundler = callPackage ../development/interpreters/ruby/bundler.nix { }; bundler = callPackage ../development/interpreters/ruby/bundler.nix { };
bundler_HEAD = import ../development/interpreters/ruby/bundler-head.nix { bundler_HEAD = bundler;
inherit buildRubyGem coreutils fetchgit; defaultGemConfig = callPackage ../development/interpreters/ruby/gemconfig/default.nix { };
}; buildRubyGem = callPackage ../development/interpreters/ruby/build-ruby-gem { };
defaultGemConfig = callPackage ../development/interpreters/ruby/bundler-env/default-gem-config.nix { };
buildRubyGem = callPackage ../development/interpreters/ruby/gem.nix { };
bundlerEnv = callPackage ../development/interpreters/ruby/bundler-env { }; bundlerEnv = callPackage ../development/interpreters/ruby/bundler-env { };
inherit (callPackage ../development/interpreters/ruby {}) inherit (callPackage ../development/interpreters/ruby {})
@ -5298,10 +5296,7 @@ let
ruby_2_1 = ruby_2_1_7; ruby_2_1 = ruby_2_1_7;
ruby_2_2 = ruby_2_2_3; ruby_2_2 = ruby_2_2_3;
rubygemsFun = ruby: builderDefsPackage (callPackage ../development/interpreters/ruby/rubygems.nix) { rubygems = hiPrio (callPackage ../development/interpreters/ruby/rubygems.nix {});
inherit ruby;
};
rubygems = hiPrio (rubygemsFun ruby);
rq = callPackage ../applications/networking/cluster/rq { }; rq = callPackage ../applications/networking/cluster/rq { };
@ -6449,7 +6444,9 @@ let
gdbm = callPackage ../development/libraries/gdbm { }; gdbm = callPackage ../development/libraries/gdbm { };
gecode = callPackage ../development/libraries/gecode { }; gecode_3 = callPackage ../development/libraries/gecode/3.nix { };
gecode_4 = callPackage ../development/libraries/gecode { };
gecode = gecode_4;
gegl = callPackage ../development/libraries/gegl { }; gegl = callPackage ../development/libraries/gegl { };