The Bazel build part of tensorflow builds can really use as many cores
as is available - to avoid blocking a small machine for literally hours
while it churns away, mark the build as big-parallel so we can schedule
it to run quickly on a machine with lots of cores.
Use the attribute mpi to provide a system wide default MPI
implementation. The default is openmpi (as before).
This now allows for overriding the MPI implentation by using
the overlay mechanism. Build all packages with mpich instead
of the default openmpi can now be achived like this:
self: super:
{
mpi = super.mpich;
}
All derivations that have been using "mpi ? null" to provide optional
building with MPI have been change in the following way to allow for
optional builds with MPI:
{ ...
, mpi
, useMpi ? false
}
Also the following related changes:
* Removed Python 2 support because it's not supported by TF and related packages for a long time.
* Upgraded tensorboard and estimator packages to the required versions.
* Added extra plugins for tensorboard to support profiling.
* In the previous derivation versions, TF_SYSTEM_LIBS didn't have any effect because it was reset at repo fetching stage, so TF always used its own dependencies. Made TF_SYSTEM_LIBS actually work and fixed the errors caused by enabling it.
* Enabled tensorboard by default (but still keeping an option to disable it if needed).
This allows us to get rid of the compatibility hacks that we had to add
(tf-1.15-bazel-1.0.patch) and also fixes#77626.
(cherry picked from commit c7adb4ee72)
They always can be regenerated during the actual build, and they are sometimes
random, e.g in Tensorflow;
platforms -> NIX_BUILD_TOP/tmp/install/35282f5123611afa742331368e9ae529/_embedded_binaries/platforms
This merges work done by yorickvP and timokau in #63208 and #63616 respectively.
Now the derivation builds both libtensorflow and the Python package and puts them into
different outputs.
Quite a bit of improvements were done on the top, including:
* Use official tag revision as source, not a branch;
* Use all system libraries possible (before only one was actually used);
* Move various environment variables to the derivation itself from hooks;
* Use source Python build instead of wheel build to ensure fixup hooks do their important jobs on libraries;
* And more that I forgot!
The `buildPython*` function computes name from `pname` and `version`.
This change removes `name` attribute from all expressions in
`pkgs/development/python-modules`.
While at it, some other minor changes were made as well, such as
replacing `fetchurl` calls with `fetchPypi`.
