#72 kmod.prov: fix and speed it up
Merged 2 months ago by ignatenkobrain. Opened 2 months ago by vda.
rpms/ vda/redhat-rpm-config master  into  master

file modified
+23 -12

@@ -1,17 +1,28 @@ 

  #!/bin/sh +x

+ # Kernel build can have many thousands of modules.

+ # kmod.prov is run for every one of them.

+ # Try to make this script run as fast as we can.

+ # For example, use shell string ops instead of external programs

+ # where possible.

  

  IFS=$'\n'

  

- for i in $(grep -E '(/lib/modules/.*\.ko|/lib/modules/.*/modules.builtin)');

- do

- 	kmod=$(basename $i | sed -e 's/.[xg]z//');

+ read -r fname || exit

  

- 	if [ $kmod == "modules.builtin" ]; then

- 		for j in $(cat $i); do

- 			j=$(basename $j);

- 			echo "kmod($j)"

- 		done

- 	else

- 		echo "kmod($kmod)"

- 	fi

- done

+ # Only process files from .../lib/modules/... subtree

+ [ "${fname#*/lib/modules/*}" != "$fname" ] || exit 0

+ 

+ kmod=${fname##*/}  # like basename, but faster

+ 

+ if [ "$kmod" = "modules.builtin" ]; then

+         for j in $(cat -- "$fname"); do

+                 echo "kmod(${j##*/})"

+         done

+         exit 0

+ fi

+ 

+ kmod=${kmod%.gz}

+ kmod=${kmod%.xz}

+ if [ "${kmod%.ko}" != "$kmod" ]; then

+         echo "kmod($kmod)"

+ fi

file modified
+4 -1

@@ -6,7 +6,7 @@ 

  

  Summary: Red Hat specific rpm configuration files

  Name: redhat-rpm-config

- Version: 145

+ Version: 146

  Release: 1%{?dist}

  # No version specified.

  License: GPL+

@@ -207,6 +207,9 @@ 

  %{_rpmconfigdir}/macros.d/macros.kmp

  

  %changelog

+ * Thu Dec 05 2019 Denys Vlasenko <dvlasenk@redhat.com> - 146-1

+ - kmod.prov: fix and speed it up

+ 

  * Tue Dec 03 15:48:18 CET 2019 Igor Gnatenko <ignatenkobrain@fedoraproject.org> - 145-1

  - %%set_build_flags: define LT_SYS_LIBRARY_PATH

  

For kernel builds, /usr/lib/rpm/kmod.prov is fork+execed by rpmbuild in "Processing files:" step about 8000 times, single-threaded, with cumulative run time of ~2 minutes.

Ideally rpmbuild should be changed to parallelize this, but it's not trivial to implement.

I propose meanwhile to speed up the script, by avoiding additional fork+execing.

Tested to work, observed speedup: almost exactly 2 times faster.

While verifying correctness, noticed that old script was buggy - it was generating a bogus "Provides:" item - kmod(modules.builtin.modinfo), because the logic in script was filtering for /.ko files and for /modules.builtin files, and wasn't prepared for the existence of */modules.builtin.modinfo file.

Pull-Request has been merged by ignatenkobrain

2 months ago