#263 Fix brp-llvm-compile-lto-elf parallelism with hardlinks (#2234024)
Merged 9 months ago by fweimer. Opened 10 months ago by nikic.
rpms/ nikic/redhat-rpm-config brp-llvm-hard-links  into  rawhide

file modified
+4 -2
@@ -48,5 +48,7 @@ 

  

  echo "Checking for LLVM bitcode artifacts"

  export -f check_convert_bitcode

- find "$RPM_BUILD_ROOT" -type f -name "*.[ao]" -print0 | \

-   xargs -0 -r -n1 -P$NCPUS sh -c "check_convert_bitcode \$@ $CLANG_FLAGS" ARG0

+ # Deduplicate by device:inode to avoid processing hardlinks in parallel.

+ find "$RPM_BUILD_ROOT" -type f -name "*.[ao]" -printf "%d:%i %p\n" | \

+   awk '!seen[$1]++' | cut -d" " -f2- | \

+   xargs -d"\n" -r -n1 -P$NCPUS sh -c "check_convert_bitcode \$@ $CLANG_FLAGS" ARG0

file modified
+5 -1
@@ -4,7 +4,7 @@ 

  # 2) When making changes, increment the version (in baserelease) by 1.

  #    rpmdev-bumpspec and other tools update the macro below, which is used

  #    in Version: to get the desired effect.

- %global baserelease 267

+ %global baserelease 268

  

  Summary: Red Hat specific rpm configuration files

  Name: redhat-rpm-config
@@ -131,6 +131,7 @@ 

  

  # for brp-llvm-compile-lto-elf

  Requires: (llvm if clang)

+ Requires: (gawk if clang)

  

  # -fstack-clash-protection and -fcf-protection require GCC 8.

  Conflicts: gcc < 8.0.1-0.22
@@ -254,6 +255,9 @@ 

  %doc buildflags.md

  

  %changelog

+ * Fri Sep 29 2023 Nikita Popov <npopov@redhat.com> - 268-1

+ - Fix brp-llvm-compile-lto-elf parallelism with hardlinks (#2234024)

+ 

  * Tue Sep 26 2023 Florian Weimer <fweimer@redhat.com> - 267-1

  - Switch %%build_type_safety_c to 1 (#2142177)

  

@@ -18,17 +18,19 @@ 

  

  %build

  

- clang ${CFLAGS} -c %{SOURCE0} -o lib.o

- ar cr %{name}.a lib.o

+ clang ${CFLAGS} -c %{SOURCE0} -o %{name}.o

+ ar cr %{name}.a %{name}.o

  ranlib %{name}.a

  

  %install

  mkdir -p %{buildroot}%{_libdir}

  mkdir -p %{buildroot}%{_includedir}

  

+ %{__install} -p -m 644 -t %{buildroot}%{_libdir} %{name}.o

  %{__install} -p -m 644 -t %{buildroot}%{_libdir} %{name}.a

  %{__install} -p -m 644 -t %{buildroot}%{_includedir} %{SOURCE1}

  

  %files

+ %{_libdir}/%{name}.o

  %{_libdir}/%{name}.a

  %{_includedir}/%{name}.h

@@ -18,7 +18,9 @@ 

  

  %build

  gcc ${CFLAGS} -c %{SOURCE0} -o %{name}.o

- gcc ${LDFLAGS} %{name}.o %{_libdir}/%{name}-lib.a -o %{name}

+ gcc ${LDFLAGS} %{name}.o %{_libdir}/%{name}-lib.a -o %{name}-ar

+ gcc ${LDFLAGS} %{name}.o %{_libdir}/%{name}-lib.o -o %{name}-obj

  

  %check

- ./%{name} | grep "Hello, world!"

+ ./%{name}-ar | grep "Hello, world!"

+ ./%{name}-obj | grep "Hello, world!"

Deduplicate the files by inode, so each hardlink is only processed
once, thus avoiding race conditions.

Thanks!

Inode numbers are only unique per device, so perhaps the find format should use %d:%i %p. The rpm-build package depends on awk, so we can assume it's always present. But perhaps still add

# for brp-llvm-compile-lto-elf
Requires: (llvm if clang)
Requires: (gawk if clang)

to the spec file for documentation purposes.

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/cf0f0be820c6422996b0263e85eedc04

rebased onto fcdd5974d225556f6b0a1413f70652e78a57f241

10 months ago

rpm-build depending on awk is an implementation detail only, it's indeed better to be explicit about the dependency here.

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/04188cd6888c4bce88b511858f3e7bf4

I've switched to using %d:%i for deduplication and added the explicit gawk dependency.

This version looks good to me, thanks.

rebased onto 88a5069

9 months ago

1 new commit added

  • Also test brp-llvm-compile-lto-elf on object file
9 months ago

This never landed in Fedora, and we found an issue with this in c9s: This used awk "!seen[$1]++" instead of awk '!seen[$1]++', which is not the same thing at all...

I've fixed this now, and also extended the test to check both an .a and .o file. That serves the dual purpose of checking that both work and checking that more than one file is processed correctly (as that's what awk with double quotes effectively amounted to).

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/d5f315877ee54918b0491b31dc286884

Pull-Request has been merged by fweimer

9 months ago