rpms / redhat-rpm-config

Clone
Source Code
GIT

#167 Implement ELF package notes
Merged 2 years ago by ngompa. Opened 2 years ago by zbyszek.
rpms/ zbyszek/redhat-rpm-config add-package-notes  into  rawhide

file modified
+16 
@@ -246,6 +246,22 @@ 
 

  

  Properly fixing the failure is always preferred!
 

  

+ ### Package note on ELF objects
 

+ 

+ A note that describes the package name, version, and architecture is
 

+ inserted via a linker script (`%_package_note_file`). The script is
 

+ generated when `%set_build_flags` is called. The linker option that
 

+ injects the linker script is added to `%{build_ldflags}` via the
 

+ `%{_package_note_flags}` macro.
 

+ 

+ To opt out of the use of the linker script, include this in the spec file:
 

+ 

+     %undefine _package_note_flags
 

+ 

+ To opt out of generation of the linker script, include this in the spec file:
 

+ 

+     %undefine _generate_package_note_file
 

+ 

  ### Post-build ELF object processing
 

  

  By default, DWARF debugging information is separated from installed

file modified
+2 -1 
@@ -72,7 +72,7 @@ 
 

  # When clang is used as a linker driver, it does not auto-detect the LTO
 

  # bytecode and neither does bfd, so we need to explicitly pass the -flto
 

  # flag when linking.
 

- %build_ldflags -Wl,-z,relro %{_ld_as_needed_flags} %{_ld_symbols_flags} %{_hardened_ldflags} %{_annotation_ldflags} %[ "%{toolchain}" == "clang" ? "%{?_lto_cflags}" : "" ] %{_build_id_flags}
 

+ %build_ldflags -Wl,-z,relro %{_ld_as_needed_flags} %{_ld_symbols_flags} %{_hardened_ldflags} %{_annotation_ldflags} %[ "%{toolchain}" == "clang" ? "%{?_lto_cflags}" : "" ] %{_build_id_flags} %{?_package_note_flags}
 

  

  # Expands to shell code to set the compiler/linker environment
 

  # variables CFLAGS, CXXFLAGS, FFLAGS, FCFLAGS, LDFLAGS if they have
 
@@ -80,6 +80,7 @@ 
 

  # been set implicitly at the start of the %%build section.
 

  # LT_SYS_LIBRARY_PATH is used by libtool script.
 

  %set_build_flags \
 

+   %{?_generate_package_note_file} ; \
 

    CFLAGS="${CFLAGS:-%{build_cflags}}" ; export CFLAGS ; \
 

    CXXFLAGS="${CXXFLAGS:-%{build_cxxflags}}" ; export CXXFLAGS ; \
 

    FFLAGS="${FFLAGS:-%{build_fflags}}" ; export FFLAGS ; \

file modified
+5 -1 
@@ -6,7 +6,7 @@ 
 

  

  Summary: Red Hat specific rpm configuration files
 

  Name: redhat-rpm-config
 

- Version: 208
 

+ Version: 209
 

  Release: 1%{?dist}
 

  # No version specified.
 

  License: GPL+
 
@@ -102,6 +102,7 @@ 
 

  Requires: fpc-srpm-macros
 

  Requires: gnat-srpm-macros
 

  Requires: nim-srpm-macros
 

+ Requires: package-notes-srpm-macros
 

  %endif
 

  

  Requires: rpm >= 4.11.0
 
@@ -190,6 +191,9 @@ 
 

  %doc buildflags.md
 

  

  %changelog
 

+ * Thu Jan 13 2022 Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> - 209-1
 

+ - Add package ELF note to the default LDFLAGS
 

+ 

  * Tue Jan 04 2022 Tom Stellard <tstellar@redhat.com> - 208-1
 

  - Call %%set_build_flags before %%build, %%check, and %%install stages

DO NOT MERGE YET — THIS IS A DRAFT.

This implements https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects.
The macros take effect if package-notes package is is installed. If not installed, there should
be no effect. In this PR, redhat-rpm-config gets Recommends:package-notes. I also want to
add package-notes to the build root, in mock and koji . For the latter, I'll file a PR for comps.
I'll also open a releng ticket for the former.

I tested this with a few packages and it seems to work. If people are happy with the general
approach, I'll do more formalized testing.

@tstellar: this builds on your change to call %set_build_flags in all sections. It would be great
if you could review my changes here…

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

LGTM

Why aren't you just making this a hard dependency and giving a disabler macro flag? This weak dependency thing makes stuff too complex for build-side stuff.

This was it'll be available also for local builds, but it's still possible to opt out.

This way

The unrelated commits all look good. Maybe we can push them separately, as they are not blocked?

And the main commit here seems to need some adjustments on the CI test.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

This was it'll be available also for local builds, but it's still possible to opt out.

This way

Traditionally, opting out is done by setting a macro, either in ~/.rpmmacros or in the package spec file. I don't know why we would do it differently for this.

This was it'll be available also for local builds, but it's still possible to opt out.

This way

Traditionally, opting out is done by setting a macro, either in ~/.rpmmacros or in the package spec file. I don't know why we would do it differently for this.

There are two ways to opt out: by setting the macro, or by not having the helper package. I added the second way to make things more flexible. But if y'all think that's too much flexibility, I'll just change Recommends→Requires. This would make other things easier, because it's not necessary to modify comps and koji config then.

5 new commits added

  • Recommend package-notes
  • Inject linker script to generate package notes
  • test/gcc-fedora-flags: modernize quoting
  • Change %define to %global
  • buildflags: use consistent indentation and other minor fixes
2 years ago

To say that the test results are cryptic would be a significant understatement… Even figuring out which tests failed, without saying why is nearly impossible, because there is no clear statement anywhere.

But I think no-new-dependency is on the list… Let's see if it works now.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

rebased onto 6718de1cddc2e38c8f35e0fecccd2184c1db0b6b
2 years ago

I've ment the flags failures at first. The no-new-dependency is easy, just add package-notes to tests/no-new-dependency/requires.txt

BTW package-notes only has main branch, not rawhide.

Reviewing the new dependency: package-notes requires python3dist(simplejson) and hence python. That is a blocker.

Using Recommends and adding it via comps does not solve that. Don't add Python (and Python libraries) to the default buildroot.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

rebased onto 6d24fb31e9d38c08144e5f84256d58019f2bfac7
2 years ago

I think I misunderstood what packages go into the buildroot. If I do mock --shell, I get python3, so I assumed that depending on python3 is not an option, and only python3-simplejson would be a new dependency.

OK, we'll write a simple generator is bash or lua…

Build succeeded.

Default mock --shell does not have python3:

$ mock -r fedora-rawhide-x86_64 --scrub=cache
...
$ mock -r fedora-rawhide-x86_64 --init
...
$ mock -r fedora-rawhide-x86_64 --shell
<mock-chroot> sh-5.1# python3
sh: python3: command not found

Even if python3 already was in the buildroot, adding any library would require a way to bootstrap it when we upgrade Python.

rebased onto 18d35aab8227310ed9d49361352ffdc92bec648f
2 years ago

V2:
the generator is now a bash script, provided by the package package-notes-srpm-macros.
That package isn't available yet, but it's building now in koji.
Only the new package is pulled into the buildroot, and it only has two small files.

https://src.fedoraproject.org/rpms/package-notes/c/c0df21c2709f303cf36690d267364795406aad01?branch=rawhide
adds package-notes-srpm-macros.

Any ideas for particularly hairy packages to build? I tried gpm, systemd, and gcc so far. I'll try kernel and mpich next.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

@zbyszek The libguestfs packages are probably a good test case. They're super-weird and do half a dozen programming languages from one source package.

rebased onto 57edf0c
2 years ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

I rebased this onto Tom's set_build_flags patch and I'm building things in COPR now.

I started a COPR build before this rebase, and the results were:
- systemd fails to build on s390x and arm, something about bpf and clang, not related.
- glibc failed with a timeout, but it builds fine in mock. In my local build, the new flags weren't used for glibc binaries. This is a shortcoming, but I don't think it's terribly important: we don't have to cover all packages with the change at the beginning.
- libguestfs failed on amd64 and i386 in tests. A kernel OOPS was triggered ;( [https://download.copr.fedorainfracloud.org/results/zbyszek/package-notes/fedora-rawhide-x86_64/03188645-libguestfs/build.log.gz]
- python3.9, python3.10, python3.11 failed in tests: https://download.copr.fedorainfracloud.org/results/zbyszek/package-notes/fedora-rawhide-s390x/03188671-python3.11/build.log.gz.
- rust-zram-generator: ICE in rustc on s390x and arm
- git and zstd failed because the linker script wasn't found. So this is an actual error. I hope it'll be fine after the rebase.

I have a feeling that this will be a very long mass rebuild ;(

moreutils ignores LDFLAGS too, but it builds fine.

In the dist-git test, gcc-fedora-flags fails with the following message:
+ annocheck hello-cpp
annocheck: Version 10.47.
Hardened: hello-cpp: WARN: (component: main): Instrumentation enabled - this is probably a mistake for production binaries.
Hardened: hello-cpp: info: (component: main): Run with -v for more information.
Hardened: hello-cpp: PASS.

But it happens the same with 208 and this PR. So I think this is related to the transition to gcc-12.

I built 96 packages in copr (and some in mock): https://copr.fedorainfracloud.org/coprs/zbyszek/package-notes/builds/

The results are that no packages failed because of the ELF notes stuff (AFAICT).
I didn't inspect all successfully-built packages to check that the build artifacts have the build note. For the dozen or so that I looked into, glibc and moreutils had no notes. The rest had a correctly-looking note (readelf --notes … | grep 'description data' | sed -e "s/\s*description data: //g" -e "s/ //g" | xxd -p -r | jq).

specific results:

syslinux            copr FAIL, i386, x86_64: cc1: error: '-fcf-protection' is not compatible with this target
lmdb                copr FAIL, arm, i386: qemu: uncaught target signal 6 (Aborted) - core dumped in tests
findutils           copr FAIL, arm, i386, s390x: configure: error: This package requires a 64-bit 'time_t' type, which your system appears to support. You might try configuring with 'CPPFLAGS="-m64" LDFLAGS="-m64"'. To build with a 32-bit time_t anyway (not recommended), configure with 'TIME_T_32_BIT_OK=yes'.
m4              copr FAIL, arm, s390x: Output from m4:
                                           /builddir/build/BUILD/m4-1.4.19/checks/../src/m4: internal error detected: Segmentation fault
tar                 copr FAIL, arm, i386, s390x: | conftest.c:441:36: error: expected expression before ')' token
                                                 |   441 | if (sizeof ((volatile sig_atomic_t)))
grep                copr FAIL, arm, s390x: XFAIL triple-backref (exit status: 1)
gawk                copr FAIL, ppc64le, s390x: some error about strftime in tests
libomp              copr FAIL, s390x: error: Architecture is excluded: s390x
gnome-text-editor   copr FAIL, i386: ../src/cjhtextregionbtree.h:296:26: error: array subscript 255 is above array bounds of ‘struct CjhTextRegionRun[26]’ [-Werror=array-bounds]
sway                copr FAIL: Problem: nothing provides requested (pkgconfig(wlroots) >= 0.15.0 with pkgconfig(wlroots) < 0.16)
wayland             copr FAIL, arm, i386, s390x: tests fail: qemu: uncaught target signal 6 (Aborted) - core dumped
harfbuzz            copr FAIL, i386: Status code: 404 for https://kojipkgs.fedoraproject.org/repos/rawhide/latest/i386/repodata/83c08e5a12802186ec14eaa6aa0318cb2ec1b60aeeb52bdc4b48d1b423015444-filelists.xml.gz (IP: 38.145.60.21)
emacs               copr FAIL: patch fails to apply
boost               copr FAIL, arm, s390x: copr timeout
vxl                 copr FAIL, arm, i386, s390x: copr timeout
vte291              copr FAIL, i386: ../bindings/vala/app.vala:1006.42-1082.5: error: value is less accessible than constant `Test.Options.entries'
openssl             copr FAIL, arm, s390x: timeout
                    (But there's also a bunch of scary warnings like 'free' called on unallocated object 'recp' [-Wfree-nonheap-object], somebody should look into this)
xmlrpc-c            copr FAIL: lib/libutil/meson.build:35:5: ERROR: Got unknown keyword arguments "install"
zeromq              copr FAIL: s390x: src/ypipe.hpp:174:31: error: request for member 'front' in '((zmq::ypipe_t<T, N>*)this)->zmq::ypipe_t<T, N>::_queue', which is of non-class type 'int'
zsh                 copr FAIL: arm, s390x: ./A01grammar.ztst: test failed.
mpich               copr FAIL: error: Directory not found: …/BUILDROOT/mpich-3.4.1-4.fc36.x86_64/usr/lib64/mpich/lib/cmake
minetest            copr FAIL, ppc64le, s390x: No matching package to install: 'luajit-devel'
rust-zram-generator copr FAIL, ICE in rustc on s390x and arm
python3.9       copr FAIL, tests fail
python3.10      copr FAIL, tests fail
python3.11      copr FAIL, tests fail
glibc               copr FAIL, timeout
                    mock works, flags UNUSED
libguestfs          copr FAIL, kernel oops
systemd             copr FAIL, s390x arm, something about bpf and clang
                    mock works, arm, amd64
moreutils           copr works, flags UNUSED

It seems that at least on the copr builders, s390x and arm are rather poorly. But the failures don't seem related to this feature at all.

It'd be great if this was merged quickly, it is only two days before the mass rebuild starts.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

I wonder why Python fails in your copr but nowhere else. I've checked and it only fails on some architectures, that we normally don't build in copr. I guess it is related to emulation, but the i686 failure surprised me.

I was wondering that too… I rebuilt python3.10 in local mock with the notes, and it builds fine (amd64). The notes seem to be attached correctly.

The control build fails the same way.

This still fails the CI.

Hardened: hello-cpp: Overall: FAIL (due to MAYB results).

Seems like it confuses annocheck?

:thumbsup:

But it happens the same with 208 and this PR. So I think this is related to the transition to gcc-12.

I see. Sorry for the noise.

Since we had part of the discussion on IRC, clarification: my understanding is that the test failures are unrelated to this PR, see https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/167#comment-94208.

In other words:

$ mock --enablerepo=local -r fedora-rawhide-x86_64 -i annobin-annocheck
$ mock --enablerepo=local -r fedora-rawhide-x86_64 --shell
# echo 'int main() {return 0;}' >test.c
# gcc -Wall -o test $(rpmbuild --eval %build_cflags) $(rpmbuild --eval %build_ldflags) test.c
# annocheck ./test 
annocheck: Version 10.47.
Hardened: test: WARN: (component: main): Instrumentation enabled - this is probably a mistake for production binaries.
Hardened: test: info: (component: main):  Run with -v for more information.
Hardened: test: Overall: FAIL (due to MAYB results).

Pull-Request has been merged by ngompa
2 years ago
Metadata
None
No Tags
Changes Summary 3
+16 -0
file changed
buildflags.md
+2 -1
file changed
macros
+5 -1
file changed
redhat-rpm-config.spec
Powered by Pagure 5.13.3
Documentation • File an Issue • About this Instance • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.