d2e918cc12
Changelog: https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.0 https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.1 https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.2 Changes in the derivation: * Tokenizers uses a newer version of pyo3 that does not require Rust nightly anymore. So, we do not have to use any cheat codes anymore. * Tokenizers is now a mixed Rust/Python project. The way it is set up does not work with Maturin, so switch to setuptools-rust instead. * Add additional data files needed for tests. * Use `pytestCheckHook`. |
||
---|---|---|
.. | ||
applications | ||
build-support | ||
common-updater | ||
data | ||
desktops | ||
development | ||
games | ||
misc | ||
os-specific | ||
pkgs-lib | ||
servers | ||
shells | ||
stdenv | ||
test | ||
tools | ||
top-level |