|
|
162b0ca |
pyproject RPM macros
|
|
|
162b0ca |
====================
|
|
|
162b0ca |
|
|
|
fb4066b |
These macros allow projects that follow the Python [packaging specifications]
|
|
|
fb4066b |
to be packaged as RPMs.
|
|
|
162b0ca |
|
|
|
fb4066b |
They are still *provisional*: we can make non-backwards-compatible changes to
|
|
|
fb4066b |
the API.
|
|
|
fb4066b |
Please subscribe to Fedora's [python-devel list] if you use the macros.
|
|
|
fb4066b |
|
|
|
fb4066b |
They work for:
|
|
|
fb4066b |
|
|
|
fb4066b |
* traditional Setuptools-based projects that use the `setup.py` file,
|
|
|
fb4066b |
* newer Setuptools-based projects that have a `setup.cfg` file,
|
|
|
fb4066b |
* general Python projects that use the [PEP 517] `pyproject.toml` file (which allows using any build system, such as setuptools, flit or poetry).
|
|
|
fb4066b |
|
|
|
fb4066b |
These macros replace `%py3_build` and `%py3_install`, which only work with `setup.py`.
|
|
|
fb4066b |
|
|
|
fb4066b |
[packaging specifications]: https://packaging.python.org/specifications/
|
|
|
fb4066b |
[python-devel list]: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/
|
|
|
162b0ca |
|
|
|
162b0ca |
|
|
|
162b0ca |
Usage
|
|
|
162b0ca |
-----
|
|
|
162b0ca |
|
|
|
fb4066b |
To use these macros, first BuildRequire them:
|
|
|
162b0ca |
|
|
|
162b0ca |
BuildRequires: pyproject-rpm-macros
|
|
|
162b0ca |
|
|
|
fb4066b |
Also BuildRequire the devel package for the Python you are building against.
|
|
|
fb4066b |
In Fedora, that's `python3-devel`.
|
|
|
fb4066b |
(In the future, we plan to make `python3-devel` itself require
|
|
|
fb4066b |
`pyproject-rpm-macros`.)
|
|
|
162b0ca |
|
|
|
fb4066b |
Next, you need to generate more build dependencies (of your projects and
|
|
|
fb4066b |
the macros themselves) by running `%pyproject_buildrequires` in the
|
|
|
fb4066b |
`%generate_buildrequires` section:
|
|
|
fdf5116 |
|
|
|
fdf5116 |
%generate_buildrequires
|
|
|
fdf5116 |
%pyproject_buildrequires
|
|
|
fdf5116 |
|
|
|
fb4066b |
This will add build dependencies according to [PEP 517] and [PEP 518].
|
|
|
fb4066b |
To also add run-time and test-time dependencies, see the section below.
|
|
|
fb4066b |
If you need more dependencies, such as non-Python libraries, BuildRequire
|
|
|
fb4066b |
them manually.
|
|
|
fb4066b |
|
|
|
fb4066b |
Note that `%generate_buildrequires` may produce error messages `(exit 11)` in
|
|
|
fb4066b |
the build log. This is expected behavior of BuildRequires generators; see
|
|
|
fb4066b |
[the Fedora change] for details.
|
|
|
fb4066b |
|
|
|
fb4066b |
[the Fedora change]: https://fedoraproject.org/wiki/Changes/DynamicBuildRequires
|
|
|
fdf5116 |
|
|
|
fdf5116 |
Then, build a wheel in `%build` with `%pyproject_wheel`:
|
|
|
162b0ca |
|
|
|
162b0ca |
%build
|
|
|
162b0ca |
%pyproject_wheel
|
|
|
162b0ca |
|
|
|
fdf5116 |
And install the wheel in `%install` with `%pyproject_install`:
|
|
|
162b0ca |
|
|
|
162b0ca |
%install
|
|
|
162b0ca |
%pyproject_install
|
|
|
162b0ca |
|
|
|
fb4066b |
`%pyproject_install` installs all wheels in `$PWD/pyproject-wheeldir/`.
|
|
|
49a323e |
|
|
|
162b0ca |
|
|
|
03316d8 |
Adding run-time and test-time dependencies
|
|
|
03316d8 |
------------------------------------------
|
|
|
03316d8 |
|
|
|
03316d8 |
To run tests in the `%check` section, the package's runtime dependencies
|
|
|
03316d8 |
often need to also be included as build requirements.
|
|
|
fb4066b |
This can be done using the `-r` flag:
|
|
|
03316d8 |
|
|
|
03316d8 |
%generate_buildrequires
|
|
|
03316d8 |
%pyproject_buildrequires -r
|
|
|
03316d8 |
|
|
|
fb4066b |
For this to work, the project's build system must support the
|
|
|
fb4066b |
[`prepare-metadata-for-build-wheel` hook](https://www.python.org/dev/peps/pep-0517/#prepare-metadata-for-build-wheel).
|
|
|
fb4066b |
The popular buildsystems (setuptools, flit, poetry) do support it.
|
|
|
fb4066b |
|
|
|
03316d8 |
For projects that specify test requirements using an [`extra`
|
|
|
03316d8 |
provide](https://packaging.python.org/specifications/core-metadata/#provides-extra-multiple-use),
|
|
|
03316d8 |
these can be added using the `-x` flag.
|
|
|
6a8d86e |
Multiple extras can be supplied as a comma separated list.
|
|
|
03316d8 |
For example, if upstream suggests installing test dependencies with
|
|
|
03316d8 |
`pip install mypackage[testing]`, the test deps would be generated by:
|
|
|
03316d8 |
|
|
|
03316d8 |
%generate_buildrequires
|
|
|
262f6d3 |
%pyproject_buildrequires -x testing
|
|
|
03316d8 |
|
|
|
8a60635 |
For projects that specify test requirements in their [tox] configuration,
|
|
|
fda0a23 |
these can be added using the `-t` flag (default tox environment)
|
|
|
fda0a23 |
or the `-e` flag followed by the tox environment.
|
|
|
fda0a23 |
The default tox environment (such as `py37` assuming the Fedora's Python version is 3.7)
|
|
|
ec07317 |
is available in the `%{toxenv}` macro.
|
|
|
8a60635 |
For example, if upstream suggests running the tests on Python 3.7 with `tox -e py37`,
|
|
|
8a60635 |
the test deps would be generated by:
|
|
|
8a60635 |
|
|
|
8a60635 |
%generate_buildrequires
|
|
|
fda0a23 |
%pyproject_buildrequires -t
|
|
|
ec07317 |
|
|
|
ec07317 |
If upstream uses a custom derived environment, such as `py37-unit`, use:
|
|
|
ec07317 |
|
|
|
fda0a23 |
%pyproject_buildrequires -e %{toxenv}-unit
|
|
|
ec07317 |
|
|
|
ec07317 |
Or specify more environments if needed:
|
|
|
ec07317 |
|
|
|
fda0a23 |
%pyproject_buildrequires -e %{toxenv}-unit,%{toxenv}-integration
|
|
|
ec07317 |
|
|
|
fda0a23 |
The `-e` option redefines `%{toxenv}` for further reuse.
|
|
|
ec07317 |
Use `%{default_toxenv}` to get the default value.
|
|
|
8a60635 |
|
|
|
fda0a23 |
The `-t`/`-e` option uses [tox-current-env]'s `--print-deps-to-file` behind the scenes.
|
|
|
8a60635 |
|
|
|
262f6d3 |
Note that both `-x` and `-t` imply `-r`,
|
|
|
262f6d3 |
because runtime dependencies are always required for testing.
|
|
|
262f6d3 |
|
|
|
8a60635 |
[tox]: https://tox.readthedocs.io/
|
|
|
8a60635 |
[tox-current-env]: https://github.com/fedora-python/tox-current-env/
|
|
|
03316d8 |
|
|
|
ec07317 |
|
|
|
ec07317 |
Running tox based tests
|
|
|
ec07317 |
-----------------------
|
|
|
ec07317 |
|
|
|
ec07317 |
In case you want to run the tests as specified in [tox] configuration,
|
|
|
fb4066b |
you must use `%pyproject_buildrequires` with `-t` or `-e` as explained above.
|
|
|
fb4066b |
Then, use the `%tox` macro in `%check`:
|
|
|
ec07317 |
|
|
|
ec07317 |
%check
|
|
|
ec07317 |
%tox
|
|
|
ec07317 |
|
|
|
ec07317 |
The macro:
|
|
|
ec07317 |
|
|
|
ec07317 |
- Always prepends `$PATH` with `%{buildroot}%{_bindir}`
|
|
|
ec07317 |
- If not defined, sets `$PYTHONPATH` to `%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}`
|
|
|
ec07317 |
- If not defined, sets `$TOX_TESTENV_PASSENV` to `*`
|
|
|
ec07317 |
- Runs `tox` with `-q` (quiet), `--recreate` and `--current-env` (from [tox-current-env]) flags
|
|
|
ec07317 |
- Implicitly uses the tox environment name stored in `%{toxenv}` - as overridden by `%pyproject_buildrequires -t`
|
|
|
ec07317 |
|
|
|
ec07317 |
By using the `-e` flag, you can use a different tox environment(s):
|
|
|
ec07317 |
|
|
|
ec07317 |
%check
|
|
|
ec07317 |
%tox
|
|
|
ec07317 |
%if %{with integration_test}
|
|
|
11da7b7 |
%tox -e %{default_toxenv}-integration
|
|
|
ec07317 |
%endif
|
|
|
ec07317 |
|
|
|
ec07317 |
If you wish to provide custom `tox` flags or arguments, add them after `--`:
|
|
|
ec07317 |
|
|
|
ec07317 |
%tox -- --flag-for-tox
|
|
|
ec07317 |
|
|
|
ec07317 |
If you wish to pass custom `posargs` to tox, use another `--`:
|
|
|
ec07317 |
|
|
|
ec07317 |
%tox -- --flag-for-tox -- --flag-for-posargs
|
|
|
ec07317 |
|
|
|
ec07317 |
Or (note the two sequential `--`s):
|
|
|
ec07317 |
|
|
|
ec07317 |
%tox -- -- --flag-for-posargs
|
|
|
ec07317 |
|
|
|
ec07317 |
|
|
|
2800b49 |
|
|
|
2800b49 |
Generating the %files section
|
|
|
2800b49 |
-----------------------------
|
|
|
2800b49 |
|
|
|
2800b49 |
To generate the list of files in the `%files` section, you can use `%pyproject_save_files` after the `%pyproject_install` macro.
|
|
|
2800b49 |
It takes toplevel module names (i.e. the names used with `import` in Python) and stores paths for those modules and metadata for the package (dist-info directory) to a file stored at `%{pyproject_files}`.
|
|
|
2800b49 |
For example, if a package provides the modules `requests` and `_requests`, write:
|
|
|
2800b49 |
|
|
|
2800b49 |
%install
|
|
|
2800b49 |
%pyproject_install
|
|
|
2800b49 |
%pyproject_save_files requests _requests
|
|
|
2800b49 |
|
|
|
2800b49 |
To add listed files to the `%files` section, use `%files -f %{pyproject_files}`.
|
|
|
2800b49 |
Note that you still need to add any documentation and license manually (for now).
|
|
|
2800b49 |
|
|
|
2800b49 |
%files -n python3-requests -f %{pyproject_files}
|
|
|
2800b49 |
%doc README.rst
|
|
|
2800b49 |
%license LICENSE
|
|
|
2800b49 |
|
|
|
2800b49 |
You can use globs in the module names if listing them explicitly would be too tedious:
|
|
|
2800b49 |
|
|
|
2800b49 |
%install
|
|
|
2800b49 |
%pyproject_install
|
|
|
99d9596 |
%pyproject_save_files '*requests'
|
|
|
2800b49 |
|
|
|
09be4c1 |
In fully automated environments, you can use the `*` glob to include all modules (put it in single quotes to prevent Shell from expanding it). In Fedora however, you should always use a more specific glob to avoid accidentally packaging unwanted files (for example, a top level module named `test`).
|
|
|
2800b49 |
|
|
|
5809bbc |
Speaking about automated environments, some files cannot be classified with `%pyproject_save_files`, but it is possible to list all unclassified files by adding a special `+auto` argument.
|
|
|
2800b49 |
|
|
|
2800b49 |
%install
|
|
|
2800b49 |
%pyproject_install
|
|
|
99d9596 |
%pyproject_save_files '*' +auto
|
|
|
2800b49 |
|
|
|
2800b49 |
%files -n python3-requests -f %{pyproject_files}
|
|
|
2800b49 |
|
|
|
2800b49 |
However, in Fedora packages, always list executables explicitly to avoid unintended collisions with other packages or accidental missing executables:
|
|
|
2800b49 |
|
|
|
2800b49 |
%install
|
|
|
2800b49 |
%pyproject_install
|
|
|
2800b49 |
%pyproject_save_files requests _requests
|
|
|
2800b49 |
|
|
|
2800b49 |
%files -n python3-requests -f %{pyproject_files}
|
|
|
2800b49 |
%doc README.rst
|
|
|
2800b49 |
%license LICENSE
|
|
|
2800b49 |
%{_bindir}/downloader
|
|
|
2800b49 |
|
|
|
2800b49 |
|
|
|
162b0ca |
Limitations
|
|
|
162b0ca |
-----------
|
|
|
162b0ca |
|
|
|
dbb90f5 |
`%pyproject_install` changes shebang lines of every Python script in `%{buildroot}%{_bindir}` to `#!%{__python3} %{py3_shbang_opt}` (`#!/usr/bin/python3 -s`).
|
|
|
dbb90f5 |
Existing Python flags in shebangs are preserved.
|
|
|
dbb90f5 |
For example `#!/usr/bin/python3 -Ru` will be updated to `#!/usr/bin/python3 -sRu`.
|
|
|
dbb90f5 |
Sometimes, this can interfere with tests that run such scripts directly by name,
|
|
|
dbb90f5 |
because in tests we usually rely on `PYTHONPATH` (and `-s` ignores that).
|
|
|
dbb90f5 |
Would this behavior be undesired for any reason,
|
|
|
9bb7de7 |
undefine `%{py3_shbang_opt}` to turn it off.
|
|
|
fdf5116 |
|
|
|
50645e1 |
Some valid Python version specifiers are not supported.
|
|
|
50645e1 |
|
|
|
fdf5116 |
[PEP 517]: https://www.python.org/dev/peps/pep-0517/
|
|
|
fdf5116 |
[PEP 518]: https://www.python.org/dev/peps/pep-0518/
|
|
|
82a7579 |
|
|
|
82a7579 |
|
|
|
82a7579 |
Testing the macros
|
|
|
82a7579 |
------------------
|
|
|
82a7579 |
|
|
|
82a7579 |
This repository has two kinds of tests.
|
|
|
82a7579 |
First, there is RPM `%check` section, run when building the `python-rpm-macros`
|
|
|
82a7579 |
package.
|
|
|
82a7579 |
|
|
|
82a7579 |
Then there are CI tests.
|
|
|
82a7579 |
There is currently [no way to run Fedora CI tests locally][ci-rfe],
|
|
|
82a7579 |
but you can do what the tests do manually using mock.
|
|
|
82a7579 |
For each `$PKG.spec` in `tests/`:
|
|
|
82a7579 |
|
|
|
82a7579 |
- clean your mock environment:
|
|
|
82a7579 |
|
|
|
82a7579 |
mock -r fedora-rawhide-x86_64 clean
|
|
|
82a7579 |
|
|
|
82a7579 |
- install the version of `python-rpm-macros` you're testing, e.g.:
|
|
|
82a7579 |
|
|
|
82a7579 |
mock -r fedora-rawhide-x86_64 install .../python-rpm-macros-*.noarch.rpm
|
|
|
82a7579 |
|
|
|
82a7579 |
- download the sources:
|
|
|
82a7579 |
|
|
|
82a7579 |
spectool -g -R $PKG.spec
|
|
|
82a7579 |
|
|
|
82a7579 |
- build a SRPM:
|
|
|
82a7579 |
|
|
|
82a7579 |
rpmbuild -bs $PKG.spec
|
|
|
82a7579 |
|
|
|
82a7579 |
- build in mock, using the path from the command above as `$SRPM`:
|
|
|
82a7579 |
|
|
|
82a7579 |
mock -r fedora-rawhide-x86_64 -n -N $SRPM
|
|
|
82a7579 |
|
|
|
82a7579 |
[ci-rfe]: https://pagure.io/fedora-ci/general/issue/4
|