Daniël de Kok
|
c67382b02f
|
python3Packages.tokenizers: 0.9.2 -> 0.9.4
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.3
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.4
|
2020-11-10 14:52:03 -08:00 |
|
Daniël de Kok
|
d2e918cc12
|
python3Packages.tokenizers: 0.8.1 -> 0.9.2
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.0
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.1
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.2
Changes in the derivation:
* Tokenizers uses a newer version of pyo3 that does not require Rust
nightly anymore. So, we do not have to use any cheat codes anymore.
* Tokenizers is now a mixed Rust/Python project. The way it is set
up does not work with Maturin, so switch to setuptools-rust instead.
* Add additional data files needed for tests.
* Use `pytestCheckHook`.
|
2020-10-28 10:41:27 -07:00 |
|
Daniël de Kok
|
403e2506af
|
python3Packages.tokenizers: 0.8.1rc1 -> 0.8.1
|
2020-08-05 08:23:11 +02:00 |
|
Daniël de Kok
|
b0a6bd4281
|
python3Packages.tokenizers: fix build
Update the parking_lot dependency to be compatible with recent Rust
versions that replace asm! by llvm_asm!:
https://github.com/Amanieu/parking_lot/pull/223
Fixes #94682
|
2020-08-05 08:19:01 +02:00 |
|
Daniël de Kok
|
918d5cfb33
|
python3Packages.tokenizers: 0.8.0 -> 0.8.1.rc1
Changelog:
https://github.com/huggingface/tokenizers/releases/tag/python-v0.8.1.rc1
|
2020-07-07 07:25:13 +02:00 |
|
Daniël de Kok
|
66d0b2a0d0
|
python3Packages.tokenizers: init at 0.8.0
|
2020-07-02 21:53:51 +02:00 |
|