Commit Graph

810 Commits

Author SHA1 Message Date
Frederik Rietdijk
76966f8ddd pythonPackages.pip-install-hook: remove --build option
Pip had the option --build to build in a custom or temporary directory.
Nowadays, pip just listens to TMPDIR, which we already set.

This option was deprecated and is removed in pip 20.3.
2020-12-01 14:44:26 +01:00
Frederik Rietdijk
9a63b3d3d6
Merge pull request #104781 from NixOS/staging-next
Staging next
2020-11-30 18:27:29 +01:00
adisbladis
49b8738a5f
python: Propagate packageOverrides to pythonForBuild
When overriding versions of build tools injected via hooks
`packageOverrides` was not taken into account and 2 incompatible
versions of the same package (`wheel` in this case) ended up in the
closure, causing the builds to fail.
2020-11-30 11:12:59 +01:00
Frederik Rietdijk
0d8491cb2b Merge master into staging-next 2020-11-29 13:51:10 +01:00
Frederik Rietdijk
455774e546 Python: add todo note regarding pythonForBuild 2020-11-28 17:36:23 +01:00
Frederik Rietdijk
5804c8a7f5 Python splicing: splice package sets, not Python derivation
In the original commit the various builds of Python were added to
`otherSplices`, instead of the intended Python package sets.
2020-11-28 17:36:23 +01:00
Frederik Rietdijk
6cf25f9dbd Python: rename parameters and arguments passed to passthru
As part of the splicing the build/host/target combinations of the interpreter
need to be passed around internally. The chosen names were not very clear,
implying they were package sets whereas actually there were derivations.
2020-11-28 17:36:23 +01:00
Frederik Rietdijk
8220b0449c python.tests: use self.callPackage instead of super.callPackage
super was incorrectly possible until https://github.com/NixOS/nixpkgs/pull/104201
got merged.
2020-11-28 16:57:07 +01:00
Frederik Rietdijk
cce2fd547b Python: use pythonPackagesBuildHost instead of pythonForBuild
Follow-up to #104201, related to #105113.
2020-11-28 16:36:03 +01:00
Jonathan Ringer
0a4575a1da python/hooks/pythonNamespaces: fix __pycache__ being empty, or not existing 2020-11-26 13:51:05 -08:00
Frederik Rietdijk
0a12b8a5e9 Revert "Revert "Revert "cpython: fix finding headers when cross-compiling extension modules"""
This still does not function without issues. E.g., bootstrapped-pip fails with Python 3.6 and 3.7
as well as 3.8 on 32-bit.

Because this is a stdenv-rebuild it needs to be tested significantly better

This reverts commit 6100bc29f7.
2020-11-25 10:34:23 +01:00
Jonathan Ringer
f5d9dd3050 python/hooks/pythonNamespaces: fix __pycache__ logic 2020-11-23 08:33:04 -08:00
Frederik Rietdijk
95d9ff16f3 python.tests: not when cross-compiling
Running the tests is not possible when cross-compiling.
2020-11-23 15:29:23 +01:00
Frederik Rietdijk
6100bc29f7 Revert "Revert "cpython: fix finding headers when cross-compiling extension modules""
Moving the sysconfig file caused spidermonkey_78 to fail to build. We now symlink it instead.

This reverts commit 6a23dde75b.
2020-11-23 15:29:23 +01:00
Frederik Rietdijk
7726f81b13 python.tests: use self.callPackage instead of super.callPackage
super was incorrectly possible until https://github.com/NixOS/nixpkgs/pull/104201
got merged.
2020-11-22 13:17:24 +01:00
Frederik Rietdijk
6a23dde75b Revert "cpython: fix finding headers when cross-compiling extension modules"
This breaks virtualenv https://github.com/NixOS/nixpkgs/issues/104483.
We should probably not move `_sysconfigdata` but just copy it for cross to
another place or output.

This reverts commit 11806b6ede.
2020-11-22 13:13:23 +01:00
Jonathan Ringer
8d210e2ea4 python3.pkgs.pipInstallHook: don't warn on script installation 2020-11-20 12:25:52 -08:00
Frederik Rietdijk
27d3a18ac7 buildPython*: don't catch conflicts when cross-compiling
Hack until we fix PYTHONPATH.
2020-11-19 20:59:16 +01:00
Ben Wolsieffer
11806b6ede cpython: fix finding headers when cross-compiling extension modules 2020-11-19 20:42:58 +01:00
Frederik Rietdijk
ea7b8978ef Merge master into staging-next 2020-11-19 20:08:15 +01:00
John Ericson
b57c5d4456 python: Use makeScopeWithSplicing
Now non-`buildInputs` that are python packages should be resolved
correctly.
2020-11-19 11:58:07 -05:00
John Ericson
330d468002 python: Use packages from previous stage in setup-hooks
This makes sense as these are tools we want to run at build time.
2020-11-19 11:58:03 -05:00
Frederik Rietdijk
da12fc6838 Merge staging-next into staging 2020-11-18 15:36:56 +01:00
Bernardo Meurer
7edf1ced82 pypy: enable aarch64-linux platform 2020-11-18 09:55:33 +01:00
Frederik Rietdijk
9fc4a4db5a Merge staging-next into staging 2020-11-13 20:59:08 +01:00
Frederik Rietdijk
3fceafce1e buildPython*: remove pythonRecompileBytecodeHook as dependency
This hook was added to get reproducible bytecode. Because it was causing
issues it was disabled, but still kept as a dependency. Now the main
issue with bytecode reproducibility has been resolved by updating pip to
20.2.4, we remove this hook as a dependency.

If a package with Python code is not yet reproducible, one could add
this hook to `nativeBuildInputs`.
2020-11-12 19:32:30 +01:00
Jonathan Ringer
5a5122418a python310: 3.10.0a1 -> 3.10.0a2 2020-11-11 23:13:44 -08:00
Frederik Rietdijk
20f001c01e Merge master into staging-next 2020-11-09 14:33:52 +01:00
Samuel Gräfenstein
22b73d1282 pkgs/*: add final newline to .nix files 2020-11-09 11:39:28 +00:00
Frederik Rietdijk
61a04f735a python3.pkgs.pip: 20.1.1 -> 20.2.4
Reproducible builds of pyproject projects using pip is resolved.

Fixes https://github.com/pypa/pip/issues/7808
Fixes https://github.com/NixOS/nixpkgs/issues/81441

The more recent c409f69480 caused trouble
with pyproject troubles and had to be reverted anyway.
https://github.com/NixOS/nixpkgs/pull/102222#issuecomment-722380794

Revert "pythonPackages.pip: make reproducible (#102222)"

This reverts commit c409f69480.

Revert "python3Packages.pip: allow setting reproducible temporary directory via NIX_PIP_INSTALL_TMPDIR"

This reverts commit aedbade43e.
2020-11-05 15:07:21 +01:00
Frederik Rietdijk
6ddd6b4914 Revert "Merge pull request #93083 from risicle/ris-cpython-debug"
This increase the closure size by 30 MB.

https://github.com/NixOS/nixpkgs/pull/93083#issuecomment-718025366

This reverts commit 4b340cbbb4, reversing
changes made to 47e499f7a4.
2020-11-03 12:50:05 +01:00
zimbatm
c409f69480
pythonPackages.pip: make reproducible (#102222)
The previous attempt wasn't covering all of the bases. It relied on
invoking that pip-install-hook, and didn't apply to pip itself.

The core issue is that the generated .pyc files embed some of the
temporary paths, which are randomly generated. See
https://r13y.com/diff/bf8c3ca3148ebff9ecf41f294cc60b9f209c006d49699e356969ff32d736f1c6-8806a7cca91fdd300e48736bfcd57c4d0b54c1cc2fd61609f35143170862b59c.html

In this new attempt, the approach is to patch the TempFile
implementation directly, so that it creates stable temporary
directories. We also assume that if SOURCE_DATE_EPOCH is set, we are in
a scenario where reproducible builds are desirable and enter that
branch.

See also https://github.com/pypa/pip/issues/7808
2020-10-31 21:24:36 +01:00
Jonathan Ringer
67d2de5ffa python310: init at 3.10a1 2020-10-19 09:31:27 -07:00
Jonathan Ringer
9c9418151d update-python-libraries: allow for top-level attrs to be updated with github fetcher 2020-10-15 12:10:08 -07:00
Frederik Rietdijk
9e1943edc0 Merge master into staging-next 2020-10-13 19:34:34 +02:00
Jonathan Ringer
2a9c3ec083
update-python-libraries: make script updateScript friendly 2020-10-11 12:03:18 -07:00
Frederik Rietdijk
692d219a93 Merge staging-next into staging 2020-10-06 10:25:58 +02:00
Jonathan Ringer
4eaf246f24 update-python-libraries: add github fetcher 2020-10-06 00:55:43 -07:00
Raphael Borun Das Gupta
3696bc0e89 python39: 3.9.0rc2 -> 3.9.0 2020-10-06 08:32:37 +02:00
Jan Tojnar
32b4375f10
Merge branch 'staging-next' into staging 2020-09-29 00:12:29 +02:00
Frederik Rietdijk
7c411b3729 python38: 3.8.5 -> 3.8.6 2020-09-27 13:07:49 +02:00
Frederik Rietdijk
74f3e9cce1 python39: 3.9.0rc1 -> 3.9.0rc2 2020-09-27 11:48:00 +02:00
Johannes Rosenberger
7abb57c7b5 fix passing qt5 version to pythonInterpreters
fixes c88f3adb17, which resulted in
qt 5.15 being used in pythonPackages, despite 5.14 being
declared, and adapts qutebrowser accordingly.

'callPackage { pkgs = pkgs // { … }; }' does not work, because
it does not take into account the recursive evaluation of nixpkgs:

`pkgs/development/interpreters/python/default.nix` calls
`pkgs/top-level/python-packages.nix` with `callPackage`.
Thus, even if the former gets passed the updated `pkgs`,
the latter always gets passed `pkgs.pkgs`.

For the change in the qt5 version to apply consistently, 'pkgs.extend'
must be used.

qutebrowser only used the right qt5 version (5.15) because all
pythonPackages used it anyway.
2020-09-22 21:03:43 +02:00
Drew Risinger
60df61b67e mk-python-derivation: document pyproject format
Optional setting of format == "pyproject", "egg" had been documented
in the manual, but they weren't listed in the function header for
``mk-python-derivation.nix``.
2020-09-16 17:38:41 +02:00
Frederik Rietdijk
af81d39b87 Merge staging-next into staging 2020-09-04 20:03:30 +02:00
Jonathan Ringer
762434fb02 python39: 3.9.0b5 -> 3.9.0rc1 2020-09-03 23:36:28 -07:00
Frederik Rietdijk
e29c1e42e0 Merge staging-next into staging 2020-08-31 19:46:33 +02:00
Frederik Rietdijk
5d8dd5c259 python.tests: don't mix the two types of tests
The integration tests code was included in the environment tests. After
this commit it is hopefully clearer what belongs together.
2020-08-30 12:59:24 +02:00
Frederik Rietdijk
2248a7422f python36: 3.6.11 -> 3.6.12 2020-08-30 08:59:42 +02:00
Spencer Baugh
780caf6690 python37: 3.7.8 -> 3.7.9
Release notes:
https://docs.python.org/release/3.7.9/whatsnew/changelog.html

Most notable is the fixed behavior of asynchronous generators; without
this fix, they fail to work in some scenarios.
2020-08-30 08:52:06 +02:00