rpms / pyproject-rpm-macros

Clone
Source Code
GIT

#268 %pyproject_save_files: Support nested directories in dist-info
Merged 2 years ago by churchyard. Opened 2 years ago by churchyard.
rpms/ churchyard/pyproject-rpm-macros nested_distinfo  into  rawhide

file modified
+5 -1 
@@ -10,7 +10,7 @@ 
 

  #   Increment Y and reset Z when new macros or features are added
 

  #   Increment Z when this is a bugfix or a cosmetic change
 

  # Dropping support for EOL Fedoras is *not* considered a breaking change
 

- Version:        1.0.1
 

+ Version:        1.1.0
 

  Release:        1%{?dist}
 

  

  # Macro files
 
@@ -121,6 +121,10 @@ 
 

  %license LICENSE
 

  

  %changelog
 

+ * Tue Apr 12 2022 Miro Hrončok <mhroncok@redhat.com> - 1.1.0-1
 

+ - %%pyproject_save_files: Support nested directories in dist-info
 

+ - Fixes: rhbz#1985340
 

+ 

  * Tue Mar 22 2022 Miro Hrončok <mhroncok@redhat.com> - 1.0.1-1
 

  - Prefix paths of intermediate files (such as %%{pyproject_files}) with NVRA

file modified
+8 -3 
@@ -320,15 +320,20 @@ 
 

              # we handle bytecode separately
 

              continue
 

  

-         if path.parent == distinfo:
 

-             if path.name in ("RECORD", "REQUESTED"):
 

+         if distinfo in path.parents:
 

+             if path.parent == distinfo and path.name in ("RECORD", "REQUESTED"):
 

                  # RECORD and REQUESTED files are removed in %pyproject_install
 

                  # See PEP 627
 

                  continue
 

-             if license_files and path.name in license_files:
 

+             if license_files and str(path.relative_to(distinfo)) in license_files:
 

                  paths["metadata"]["licenses"].append(path)
 

              else:
 

                  paths["metadata"]["files"].append(path)
 

+             # nested directories within distinfo
 

+             index = path.parents.index(distinfo)
 

+             for parent in list(path.parents)[:index]:  # no direct slice until Python 3.10
 

+                 if parent not in paths["metadata"]["dirs"]:
 

+                     paths["metadata"]["dirs"].append(parent)
 

              continue
 

  

          for sitedir in sitedirs:

file modified
+10 -8 
@@ -413,6 +413,7 @@ 
 

      metadata:
 

        dirs:
 

        - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info
 

+       - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/licenses
 

        docs: []
 

        files:
 

        - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/AUTHORS
 
@@ -422,8 +423,8 @@ 
 

        - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/entry_points.txt
 

        - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/top_level.txt
 

        licenses:
 

-       - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/LICENSE
 

-       - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/LICENSE.python
 

+       - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/licenses/LICENSE
 

+       - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/licenses/LICENSE.python
 

      lang:
 

        django:
 

          af:
 
@@ -7766,6 +7767,7 @@ 
 

  - - django
 

    - django
 

    - - '%dir /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info'
 

+     - '%dir /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/licenses'
 

      - '%dir /usr/lib/python3.7/site-packages/django'
 

      - '%dir /usr/lib/python3.7/site-packages/django/__pycache__'
 

      - '%dir /usr/lib/python3.7/site-packages/django/apps'
 
@@ -11349,8 +11351,8 @@ 
 

      - '%lang(zh) /usr/lib/python3.7/site-packages/django/contrib/sessions/locale/zh_Hant/LC_MESSAGES/django.mo'
 

      - '%lang(zh) /usr/lib/python3.7/site-packages/django/contrib/sites/locale/zh_Hans/LC_MESSAGES/django.mo'
 

      - '%lang(zh) /usr/lib/python3.7/site-packages/django/contrib/sites/locale/zh_Hant/LC_MESSAGES/django.mo'
 

-     - '%license /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/LICENSE'
 

-     - '%license /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/LICENSE.python'
 

+     - '%license /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/licenses/LICENSE'
 

+     - '%license /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/licenses/LICENSE.python'
 

      - /usr/bin/django-admin
 

      - /usr/bin/django-admin.py
 

      - /usr/lib/python3.7/site-packages/Django-3.0.7.dist-info/AUTHORS
 
@@ -15456,8 +15458,8 @@ 
 

      content: |
 

        Name: Django
 

        Version: 3.0.7
 

-       License-File: LICENSE
 

-       License-File: LICENSE.python
 

+       License-File: licenses/LICENSE
 

+       License-File: licenses/LICENSE.python
 

        Whatever: False data
 

  

  records:
 
@@ -15751,8 +15753,8 @@ 
 

        ../../../bin/django-admin.py,sha256=OOv0QKYqhDD2O4X3HQx3gFFQ-CC7hSLnWuzZnQXeiiA,115
 

        Django-3.0.7.dist-info/AUTHORS,sha256=cV29hNQ1SpKhTmZuPff3LWHyQ7mHNBWP7_0JufEUHbs,36806
 

        Django-3.0.7.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4
 

-       Django-3.0.7.dist-info/LICENSE,sha256=uEZBXRtRTpwd_xSiLeuQbXlLxUbKYSn5UKGM0JHipmk,1552
 

-       Django-3.0.7.dist-info/LICENSE.python,sha256=Z-Pr3SuMPxOcaosqZSgr_NAjh2cFRcFyPZjP7nMeQrQ,13231
 

+       Django-3.0.7.dist-info/licenses/LICENSE,sha256=uEZBXRtRTpwd_xSiLeuQbXlLxUbKYSn5UKGM0JHipmk,1552
 

+       Django-3.0.7.dist-info/licenses/LICENSE.python,sha256=Z-Pr3SuMPxOcaosqZSgr_NAjh2cFRcFyPZjP7nMeQrQ,13231
 

        Django-3.0.7.dist-info/METADATA,sha256=0ZU0N0E-CHKarXMLp4oOYf7EMUHR8eJ79f2yqw2NwoM,3574
 

        Django-3.0.7.dist-info/RECORD,,
 

        Django-3.0.7.dist-info/WHEEL,sha256=g4nMs7d-Xl9-xC9XovUrsDHGXt-FT0E17Yqo92DEfvY,92

file added
+62 
@@ -0,0 +1,62 @@ 
 

+ Name:           python-userpath
 

+ Version:        1.8.0
 

+ Release:        1%{?dist}
 

+ Summary:        Cross-platform tool for adding locations to the user PATH
 

+ License:        MIT
 

+ URL:            https://github.com/ofek/userpath
 

+ Source:         %{pypi_source userpath}
 

+ BuildArch:      noarch
 

+ BuildRequires:  python3-devel
 

+ 

+ # Manually BuildRequire the runtime dependencies until we have a solution
 

+ # for build backends without prepare_metadata_for_build_wheel():
 

+ BuildRequires:  python3dist(click)
 

+ 

+ %description
 

+ This package uses hatchling as build backend.
 

+ This package is tested because:
 

+ 

+  - the prepare_metadata_for_build_wheel hook does not exist
 

+    https://github.com/ofek/hatch/issues/128
 

+  - the licenses are stored in a dist-info subdirectory
 

+    https://bugzilla.redhat.com/1985340
 

+    (as of hatchling 0.22.0, not yet marked as License-File)
 

+ 

+ 

+ %package -n     python3-userpath
 

+ Summary:        %{summary}
 

+ 

+ %description -n python3-userpath
 

+ ...
 

+ 

+ 

+ %if 0%{?fedora} > 35
 

+ # On Fedora 35 or EPEL, we don't have hatchling yet, so this entire spec file builds nothing
 

+ 

+ %prep
 

+ %autosetup -p1 -n userpath-%{version}
 

+ sed -Ei '/^(coverage)$/d' requirements-dev.txt
 

+ 

+ 

+ %generate_buildrequires
 

+ # Cannot use -r (the default) with hatchling, must use -R
 

+ %pyproject_buildrequires requirements-dev.txt -R
 

+ 

+ 

+ %build
 

+ %pyproject_wheel
 

+ 

+ 

+ %install
 

+ %pyproject_install
 

+ %pyproject_save_files userpath
 

+ 

+ 

+ %check
 

+ %pytest
 

+ 

+ 

+ %files -n python3-userpath -f %{pyproject_files}
 

+ %{_bindir}/userpath
 

+ 

+ %endif

file modified
+3 
@@ -79,6 +79,9 @@ 
 

      - getmac:
 

          dir: .
 

          run: ./mocktest.sh python-getmac
 

+     - userpath:
 

+         dir: .
 

+         run: ./mocktest.sh python-userpath
 

      - double_install:
 

          dir: .
 

          run: ./mocktest.sh double-install

no initial comment

Build succeeded.

I see you already added a spec file for python-userpath as a test case, but I can independently confirm that this does include the license_files subdirectory and its contents with the generated files list for both python-hatchling and python-userpath. Thanks!

On the other hand, the license files in that subdirectory are not marked with %license, so it would be necessary to manually handle them, e.g.

%license LICENSE.txt

I haven’t looked into whether that is because of something pyproject-rpm-macros is or isn’t doing, or whether it’s something hatchling is or isn’t doing. Either way, if we can identify it, we can probably get it fixed.

See here: as of hatchling 0.22.0, not yet marked as License-File

License-File is not yet standardized. From pyproject-rpm-macros README:

%pyproject_save_files can automatically mark license files with %license macro ... Only license files declared via PEP 639 License-Field field are detected. PEP 639 is still a draft and can be changed in the future.

AFAIK only setuptools currently does this. If you can talk hatchling into supporting it, cool.

AFAIK only setuptools currently does this. If you can talk hatchling into supporting it, cool.

It looks like there is already some support in the source code. I filed an issue to ask when the License-File field should be set in practice, and whether it can be set in more cases.

AFAIK only setuptools currently does this. If you can talk hatchling into supporting it, cool.

It looks like there is already some support in the source code. I filed an issue to ask when the License-File field should be set in practice, and whether it can be set in more cases.

Upstream reports:

https://ofek.dev/hatch/dev/plugins/builder/#options core-metadata-version needs to be set to 2.3 (which isn't yet the default b/c PyPI will reject such uploads)

rebased onto 9ac213ae1fb1f069353189fe4f50c28ccfe97e21
2 years ago

[citest]

Build succeeded.

I'm so confused by this string I'm not sure what to suggest instead. Maybe "Interesting takes" or "features"?

Rawhide in mock doesn't work today, so I ran the mockbuilds and tests on Fedora 36.

Initial state: license files in nested directories are not picked up correctly by %pyproject_save_files.
Reality: Building python-userpath against pyproject-rpm-macros 1.0.1 ends with failure with: 

error: Installed (but unpackaged) file(s) found:
   /usr/lib/python3.10/site-packages/userpath-1.8.0.dist-info/license_files/LICENSE.txt

Assumption: the same specfile should produce the correct output when built against pyproject-rpm-macros 1.1.0 (this change)
Reality: Package builds successfully. :heavy_check_mark: 

$ rpm -ql python3-userpath-1.8.0-1.fc36.noarch.rpm                                        
<snip>
/usr/lib/python3.10/site-packages/userpath-1.8.0.dist-info/license_files
/usr/lib/python3.10/site-packages/userpath-1.8.0.dist-info/license_files/LICENSE.txt

Assumption: License files should be marked as %license if metadata contains License-File
Reality:

  • N/A for userpath (hatchling doesn't provide support for License-File)
  • correctly marked for licenses in nested directory (Django in package tests) :heavy_check_mark:
  • correctly marked for package that has got license files in distinfo (no regression introduced) :heavy_check_mark: 
$ rpm -qL pyp2spec-0.4.0-1.fc36.noarch.rpm                                        
/usr/lib/python3.10/site-packages/pyp2spec-0.4.0.dist-info/LICENSE-CC0
/usr/lib/python3.10/site-packages/pyp2spec-0.4.0.dist-info/LICENSE-MIT

Assumption: The change should work even if license files were placed in multiple levels of nested directories (multiple directories are picked up, license files are correctly marked)
Reality: tweaking with Django metadata (distinfo/license_files/deeper/license in package tests) confirms this assumption. :heavy_check_mark:

Edited 2 years ago by ksurma
  • [+] PR solves the issue it claims to address (reported in the comment above)
  • [+] Resulting RPM package is installable on the destination Fedora release (Tested with Zuul on all releases)
  • [+] PR is tested sufficiently and the test results are OK (green CI, tested manually on F36)
  • [+] PR is open against all relevant Fedora releases
  • [+] Branches don't diverge unnecessarily (same commit hash)
  • [+] PR doesn't contain backwards incompatible changes (partially tested with CI; absence of hatchling is taken care of)
  • [+] Each commit's scope is sane (there are no irrelevant changes combined together)
  • [+] Each commit message is relevant
  • [+] The right (problem, product) BZ ticket is referenced
  • [+] BZ ticket reference is in the correct format in %changelog
  • [+] Version/Release is bumped

I tested about everything I could think of in this change.
Except for the trivial nitpick in the test data I'd say the change is good to merge.

I'm so confused by this string I'm not sure what to suggest instead. Maybe "Interesting takes" or "features"?

The description describes what is being tested here. What do you think about "This package is tested because:"

I'm so confused by this string I'm not sure what to suggest instead. Maybe "Interesting takes" or "features"?

The description describes what is being tested here. What do you think about "This package is tested because:"

I like that, go ahead.

rebased onto 6d0900f
2 years ago

Pull-Request has been merged by churchyard
2 years ago
Metadata
None
No Tags
Changes Summary 5
+5 -1
file changed
pyproject-rpm-macros.spec
+8 -3
file changed
pyproject_save_files.py
+10 -8
file changed
pyproject_save_files_test_data.yaml
+62
file added
tests/python-userpath.spec
+3 -0
file changed
tests/tests.yml
Powered by Pagure 5.14.1
Documentation • File an Issue • About this Instance • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.