#110 Set $RPM_BUILD_ROOT in %{python3_...} macros, for alternate sysconfig install scheme
Merged 21 days ago by churchyard. Opened 2 months ago by churchyard.
rpms/ churchyard/python-rpm-macros set-RPM_BUILD_ROOT-in-macros  into  rawhide

file modified
+9 -7
@@ -1,12 +1,14 @@ 

  # unversioned macros: used with user defined __python, no longer part of rpm >= 4.15

  # __python is defined to error by default in the srpm macros

- %python_sitelib %(%{__python} -Esc "import sysconfig; print(sysconfig.get_path('purelib'))")

- %python_sitearch %(%{__python} -Esc "import sysconfig; print(sysconfig.get_path('platlib'))")

- %python_version %(%{__python} -Esc "import sys; sys.stdout.write('{0.major}.{0.minor}'.format(sys.version_info))")

- %python_version_nodots %(%{__python} -Esc "import sys; sys.stdout.write('{0.major}{0.minor}'.format(sys.version_info))")

- %python_platform %(%{__python} -Esc "import sysconfig; print(sysconfig.get_platform())")

- %python_platform_triplet %(%{__python} -Esc "import sysconfig; print(sysconfig.get_config_var('MULTIARCH'))")

- %python_ext_suffix %(%{__python} -Esc "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))")

+ # nb: $RPM_BUILD_ROOT is not set when the macros are expanded (at spec parse time)

+ #     so we set it manually (to empty string), making our Python prefer the correct install scheme location

+ %python_sitelib %(RPM_BUILD_ROOT= %{__python} -Esc "import sysconfig; print(sysconfig.get_path('purelib'))")

+ %python_sitearch %(RPM_BUILD_ROOT= %{__python} -Esc "import sysconfig; print(sysconfig.get_path('platlib'))")

+ %python_version %(RPM_BUILD_ROOT= %{__python} -Esc "import sys; sys.stdout.write('{0.major}.{0.minor}'.format(sys.version_info))")

+ %python_version_nodots %(RPM_BUILD_ROOT= %{__python} -Esc "import sys; sys.stdout.write('{0.major}{0.minor}'.format(sys.version_info))")

+ %python_platform %(RPM_BUILD_ROOT= %{__python} -Esc "import sysconfig; print(sysconfig.get_platform())")

+ %python_platform_triplet %(RPM_BUILD_ROOT= %{__python} -Esc "import sysconfig; print(sysconfig.get_config_var('MULTIARCH'))")

+ %python_ext_suffix %(RPM_BUILD_ROOT= %{__python} -Esc "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))")

  

  %py_setup setup.py

  %py_shbang_opts -s

file modified
+9 -7
@@ -1,10 +1,12 @@ 

- %python3_sitelib %(%{__python3} -Ic "import sysconfig; print(sysconfig.get_path('purelib'))")

- %python3_sitearch %(%{__python3} -Ic "import sysconfig; print(sysconfig.get_path('platlib'))")

- %python3_version %(%{__python3} -Ic "import sys; sys.stdout.write('{0.major}.{0.minor}'.format(sys.version_info))")

- %python3_version_nodots %(%{__python3} -Ic "import sys; sys.stdout.write('{0.major}{0.minor}'.format(sys.version_info))")

- %python3_platform %(%{__python3} -Ic "import sysconfig; print(sysconfig.get_platform())")

- %python3_platform_triplet %(%{__python3} -Ic "import sysconfig; print(sysconfig.get_config_var('MULTIARCH'))")

- %python3_ext_suffix %(%{__python3} -Ic "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))")

+ # nb: $RPM_BUILD_ROOT is not set when the macros are expanded (at spec parse time)

+ #     so we set it manually (to empty string), making our Python prefer the correct install scheme location

+ %python3_sitelib %(RPM_BUILD_ROOT= %{__python3} -Ic "import sysconfig; print(sysconfig.get_path('purelib'))")

+ %python3_sitearch %(RPM_BUILD_ROOT= %{__python3} -Ic "import sysconfig; print(sysconfig.get_path('platlib'))")

+ %python3_version %(RPM_BUILD_ROOT= %{__python3} -Ic "import sys; sys.stdout.write('{0.major}.{0.minor}'.format(sys.version_info))")

+ %python3_version_nodots %(RPM_BUILD_ROOT= %{__python3} -Ic "import sys; sys.stdout.write('{0.major}{0.minor}'.format(sys.version_info))")

+ %python3_platform %(RPM_BUILD_ROOT= %{__python3} -Ic "import sysconfig; print(sysconfig.get_platform())")

+ %python3_platform_triplet %(RPM_BUILD_ROOT= %{__python3} -Ic "import sysconfig; print(sysconfig.get_config_var('MULTIARCH'))")

+ %python3_ext_suffix %(RPM_BUILD_ROOT= %{__python3} -Ic "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))")

  %py3dir %{_builddir}/python3-%{name}-%{version}-%{release}

  

  %py3_shbang_opts -s

file modified
+5 -1
@@ -47,7 +47,7 @@ 

  end

  }

  Version:        %{__default_python3_version}

- Release:        8%{?dist}

+ Release:        9%{?dist}

  

  BuildArch:      noarch

  
@@ -141,6 +141,10 @@ 

  

  

  %changelog

+ * Thu Sep 09 2021 Miro Hrončok <mhroncok@redhat.com> - 3.10-9

+ - Set $RPM_BUILD_ROOT in %%{python3_...} macros

+   to allow selecting alternate sysconfig install scheme based on that variable

+ 

  * Thu Sep 09 2021 Petr Viktorin <pviktori@redhat.com> - 3.10-8

  - Use --hardlink-dupes in %%py_byte_compile and brp-python-bytecompile

    (for Python 3)

Our Pythons currently patches distutils to install packages to
/usr/lib(64)/pythonX.Y/site-packages when the $RPM_BUILD_ROOT environment
variable is set (and to /usr/local/lib(64)/pythonX.Y/site-packages otherwise).
With the deprecation of distutils [1] we want to change the patch to create
and use a different sysconfig install scheme [2].

However, we have realized that macros defined as %(%{__python3} ...) don't
"see" the environment variables set by rpmbuild and hence
e.g. %{python3_sitelib} evaluates to
/usr/local/lib/python3.X/site-packages -- which is not desired.
To be able to reliably detect an RPM build environment by checking
the presence of the $RPM_BUILD_ROOT environment variable,
we manually set it in the macro definitions.

We set it to %{buildroot} which is not always fully populated
(e.g. it can expand literally to
/home/anna/rpmbuild/BUILDROOT/%{NAME}-%{VERSION}-%{RELEASE}.x86_64)
but that does not bother us at all:
It simply needs to be set to something
and setting it to an empty/placeholder value seems weird at least.

See also the analysis of the build failures when this is not done [3].

[1] https://www.python.org/dev/peps/pep-0632/
[2] https://bugs.python.org/issue43976
[3] https://src.fedoraproject.org/rpms/python3.10/pull-request/63#comment-79042

Build succeeded.

I'm curious as to the underlying reason why $RPM_BUILD_ROOT is unset when you expect it to be set. As far as I can tell from the default macro set, it's defined in %___build_pre which gets included in most of the scriptlet templates (prep, buildrequires, build, audodep, clean, check, install and some others).

It might be worth dumping the actual executed scriptlets just to double check. I wish that conditional call to set -x (gets called when %verbose is true) was up earlier in the sequence so you could see the variables being set.

Never mind; I think I understand it; the issue is not that RPM_BUILD_ROOT is unset when the scriptlets run, it's that it's not set when the macros are expanded (at spec parse time), which itself shells out to python.

Honestly I think that kind of odd magic is deserving of a bit of explanatory commenting, because just by looking at the patch you could easily pick up at least two incorrect ideas:

  • That %buildroot is useful at specfile parse time

  • That the actual value of $RPM_BUILD_ROOT is important to the actual python command lines you're running.

Maybe using a dummy value is actually less confusing, if the value really doesn't matter. Though even that would still bear commenting.

I also think it would make a lot more sense if this could be done via a flag that's passed to python instead of it keying off of an environment variable. But that's obviously out of scope.

Yeah, I'll add a comment.
Flags are... hard to pass somewhere else, and hard to propagate to the correct code. Checking an environment variable is simple.

...it's that it's not set when the macros are expanded (at spec parse time)...

I'll also incorporate this into the commit message to make it more understandable.

rebased onto 5b578a8

a month ago
  • rebased
  • added a comment
  • set to an empty value
  • amended the commit message

Build succeeded.

Thanks for that; this makes much more sense now.

Pull-Request has been merged by churchyard

21 days ago

Build succeeded.