#51 Templates for forge stuff
Opened 7 months ago by nim. Modified 11 days ago
rpms/ nim/redhat-rpm-config dev  into  master

file modified
+2

@@ -1,4 +1,6 @@ 

  -- Convenience Lua functions that can be used within rpm macros

+ -- Most of those emulate

+ -- https://github.com/rpm-software-management/rpm/issues/573

  

  -- Set a spec variable

  -- Echo the result if verbose

file modified
+3

@@ -3,6 +3,9 @@ 

  # A directory for rpm macros

  %rpmmacrodir /usr/lib/rpm/macros.d

  

+ # A directory for spec template examples

+ %_spectemplatedir %{_datadir}/rpmdevtools/fedora

+ 

  # A directory for appdata metainfo.  This has changed between releases so a

  # macro is useful.

  %_metainfodir %{_datadir}/metainfo

file modified
+43 -2

@@ -82,6 +82,21 @@ 

  # Documentation

  Source900: buildflags.md

  

+ # Spec templates

+ # Because redhat-rpm-config does not have a proper upstream repository like

+ # everyone else files end up directly in the src.rpm. Therefore, use a fake

+ # extension for example spec files as otherwise copr tries to build them

+ # instead of the main spec file. However, that also means pretty much every dev

+ # tool from text editors to SCM web frontends won't apply the correct syntax

+ # highlighting when inspecting those files. And the spec files get more

+ # complicated as a result. This sucks.

+ # https://bugzilla.redhat.com/show_bug.cgi?id=1655322

+ Source950: spectemplate-forge-0-release.fake

+ Source951: spectemplate-forge-1-tag.fake

+ Source952: spectemplate-forge-2-commit.fake

+ Source953: spectemplate-forge-3-multi.fake

+ Source954: spectemplate-forge-4-branch.fake

+ 

  BuildArch: noarch

  BuildRequires: perl-generators

  Requires: coreutils

@@ -115,24 +130,43 @@ 

  Conflicts: gcc < 8.0.1-0.22

  

  Provides: system-rpm-config = %{version}-%{release}

+ Provides: redhat-rpm-macros = %{version}-%{release}

  

  %global rrcdir /usr/lib/rpm/redhat

+ # Pull in _spectemplatedir definition, the spec file uses it

+ %{load:%{SOURCE100}}

  

  %description

  Red Hat specific rpm configuration files.

  

  %package -n kernel-rpm-macros

  Summary: Macros and scripts for building kernel module packages

- Requires: redhat-rpm-config >= 13

+ Requires:  %{name} = %{version}-%{release}

  

  %description -n kernel-rpm-macros

  Macros and scripts for building kernel module packages.

  

+ %package  -n redhat-rpm-templates

+ Summary:   Example rpm spec templates for the macros provided by %{name}

+ BuildArch: noarch

+ Requires:  %{name} = %{version}-%{release}

+ Requires:  rpmdevtools

+ 

+ %description -n redhat-rpm-templates

+ This package contains documented rpm spec templates showcasing how to use the

+ macros provided by %{name}.

+ 

  %prep

  # Not strictly necessary but allows working on file names instead

  # of source numbers in install section

  %setup -c -T

  cp -p %{sources} .

+ for template in spectemplate*\.fake ; do

+   rad="${template%.fake}"

+   mv "${rad}.fake" "${rad}.spec"

+   grep -v '^#' "${rad}.spec" > "${rad}-bare.spec"

+   touch -r "${rad}.spec" "${rad}-bare.spec"

+ done

  

  %install

  mkdir -p %{buildroot}%{rrcdir}

@@ -161,6 +195,10 @@ 

  install -p -m 644 -t %{buildroot}%{_rpmluadir}/fedora common.lua

  install -p -m 644 -t %{buildroot}%{_rpmluadir}/fedora/srpm forge.lua

  

+ install -m 0755 -vd   %{buildroot}%{_spectemplatedir}

+ install -m 0644 -vp   spectemplate*\.spec \

+                       %{buildroot}%{_spectemplatedir}

+ 

  %files

  %dir %{rrcdir}

  %{rrcdir}/macros

@@ -185,7 +223,7 @@ 

  %dir %{_rpmluadir}/fedora

  %dir %{_rpmluadir}/fedora/srpm

  %dir %{_rpmluadir}/fedora/rpm

- %{_rpmluadir}/fedora/*.lua

+ %{_rpmluadir}/fedora/*lua

  %{_rpmluadir}/fedora/srpm/*lua

  

  %doc buildflags.md

@@ -201,6 +239,9 @@ 

  %{rrcdir}/find-provides.d/modalias.prov

  %{_rpmconfigdir}/macros.d/macros.kmp

  

+ %files -n redhat-rpm-templates

+ %{_spectemplatedir}/*.spec

+ 

  %changelog

  * Mon Jul 01 2019 Florian Festi <ffesti@redhat.com> - 134-1

  - Switch binary payload compression to Zstandard level 19

@@ -0,0 +1,74 @@ 

+ # Any software publishing website, permitting the download of source archives

+ # via normalized URLs, that can be deduced from a project root URL and version,

+ # commit, tag, scm, extension… values is a “forge” that can be supported by

+ # the forgemeta macro. Common examples are GitLab and GitHub. forgemeta

+ # centralizes and abstracts our knowledge about those forges, so packagers do

+ # not have to handle download quirks manually.

+ #

+ # forgemeta makes it easy to switch from release to tag to commit source

+ # archives. Using forgemeta, forge download URLs or guideline changes are

+ # propagated to spec files without manual refactoring.

+ #

+ # When those changes result in a different naming or structure of the source

+ # archive, the source file needs to be uploaded to the build system before

+ # rebuilding existing spec files. That is the main drawback of using

+ # forgemeta.

+ #

+ # This template shows how to package a release state using the forgemeta macro.

+ # That should be the default target use case.

+ # Other less common use cases are documented in separate templates.

+ #

+ # The project url on the forge, for example

+ # https://gitlab.gnome.org/GNOME/gtk/

+ %global forgeurl 

+ #

+ # Packaging a release requires setting Version before calling forgemeta.

+ # Because most forges follow the git model, and git made no provision for

+ # release objects, forgemeta will try to guess the customary way to write

+ # release tags on the selected forge.

+ # If it guesses wrong use the forge-tag template instead of this one.

+ Version:         

+ #

+ # forgemeta converts the suppplied rpm variables to variables that can be used

+ # in the spec file. Most of those can be overriden before or after the

+ # forgemeta call.

+ #  – use the “-i” flag to display the variables forgemeta reads and sets

+ #  – use the “-v” flag if you want verbose processing

+ #  – remove  “-i” and “-v” before commit

+ %forgemeta

+ 

+ # The following lines use  variables computed by forgemeta as default values.

+ # You can replace them with manual definitions. For example, replace forgeurl

+ # with the project homepage if it exists separately from the repository URL.

+ # Only replace the variables when it adds value to the spec file and you

+ # understand the consequences. Release ordering is controlled by the packager

+ # with x%{?dist}/0.x%{?dist} number chains.

+ Name:    

+ Release: 1%{?dist}

+ Summary: 

+ URL:     %{forgeurl}

+ Source:  %{forgesource}

+ %description

+ 

+ 

+ %prep

+ # forgesetup calls setup with the correct flags for archives downloaded from

+ # the selected forge. A forgeautosetup autosetup wrapper is also provided. If

+ # you disagree with the computed setup/autosetup flags, just call

+ # setup/autosetup directly.

+ #  – use the “-v” flag if you want verbose processing

+ %forgesetup

+ #

+ # After this point the archive extraction is done. forgemeta is no longer used.

+ 

+ %build

+ 

+ %install

+ 

+ %check

+ 

+ %files

+ %license 

+ %doc 

+ 

+ %changelog

@@ -0,0 +1,46 @@ 

+ # This template shows how to package a tag state using the forgemeta macro.

+ # For more explanations on forgemeta, read the forgemeta-release template.

+ #

+ # The project url on the forge

+ %global forgeurl 

+ #

+ # The tag being packaged

+ %global tag      

+ #

+ #  – use the “-i” flag to display the variables forgemeta reads and sets

+ #  – use the “-v” flag if you want verbose processing

+ #  – remove  “-i” and “-v” before commit

+ %forgemeta

+ 

+ # The following lines use variables computed by forgemeta as default values.

+ # You can replace them with manual definitions.

+ # forgemeta will prepend tag information to dist. Release ordering is

+ # controlled by the packager with x%{?dist}/0.x%{?dist} numbers chains.

+ Name:    

+ Version: 

+ Release: 1%{?dist}

+ Summary: 

+ URL:     %{forgeurl}

+ Source:  %{forgesource}

+ %description

+ 

+ 

+ %prep

+ #  – an autosetup wrapper, forgeautosetup, is also provided

+ #  – use the “-v” flag if you want verbose processing; remove it before commit

+ #  – call forgesetup/autosetup directly if you do not like the result

+ %forgesetup

+ #

+ # After this point the archive extraction is done. forgemeta is no longer used.

+ 

+ %build

+ 

+ %install

+ 

+ %check

+ 

+ %files

+ %license 

+ %doc 

+ 

+ %changelog

@@ -0,0 +1,46 @@ 

+ # This template shows how to package a commit state using the forgemeta macro.

+ # For more explanations on forgemeta, read the forgemeta-release template.

+ #

+ # The project url on the forge

+ %global forgeurl 

+ #

+ # The commit being packaged (when using git, a full hash)

+ %global commit   

+ #

+ #  – use the “-i” flag to display the variables forgemeta reads and sets

+ #  – use the “-v” flag if you want verbose processing

+ #  – remove  “-i” and “-v” before commit

+ %forgemeta

+ 

+ # The following lines use variables computed by forgemeta as default values.

+ # You can replace them with manual definitions.

+ # forgemeta will prepend commit information to dist. Release ordering is

+ # controlled by the packager with x%{?dist}/0.x%{?dist} numbers chains.

+ Name:    

+ Version: 

+ Release: 1%{?dist}

+ Summary: 

+ URL:     %{forgeurl}

+ Source:  %{forgesource}

+ %description

+ 

+ 

+ %prep

+ #  – an autosetup wrapper, forgeautosetup, is also provided

+ #  – use the “-v” flag if you want verbose processing; remove it before commit

+ #  – call forgesetup/autosetup directly if you do not like the result

+ %forgesetup

+ #

+ # After this point the archive extraction is done. forgemeta is no longer used.

+ 

+ %build

+ 

+ %install

+ 

+ %check

+ 

+ %files

+ %license 

+ %doc 

+ 

+ %changelog

@@ -0,0 +1,81 @@ 

+ # This template shows how to package multiple source archives using the

+ # forgemeta macro.

+ #

+ # PACKAGING MULTIPLE PROJECT ARCHIVES IN A SINGLE SPEC IS DISCOURAGED. IT OFTEN

+ # REQUIRES ERROR-INDUCING VERSIONNING DECISIONS. In rpm, versioning is not an

+ # administrative decoration. Versioning controls the upgrade logic. If upstream

+ # deemed necessary to release a project through multiple archives, mirror its

+ # decision using multiple spec files. That will be safer, simpler, and

+ # ultimately less work.

+ #

+ # For more explanations on forgemeta, read the forgemeta-release template.

+ #

+ # Start by declaring the characteristics of each source using a	number-suffixed

+ # variable block:

+ #  – no suffix and zero suffix blocks are aliases

+ #  – they are “special” and identify the main source archive

+ #  — they use Version: to identify a release

+ #  – other blocks use a version<number> variable for the same need

+ #  – the syntax is otherwise identical to single archive mode; see the various

+ #    forge templates for examples.

+ #

+ # Main archive. In this example we package a full release

+ %global forgeurl0 

+ Version:          

+ 

+ # Second archive.

+ %global forgeurl1 

+ %global version1  

+ 

+ # Third archive. This time a tag. Continue as necessary.

+ %global forgeurl2 

+ %global tag2      

+ 

+ #  – use the “-a” flag to process all the source archives in a single pass

+ #  – use     “-z <number>” to process only the declaration block suffixed with

+ #    <number>

+ #  — without “-a” of “-z <number>” only the main archive is processed

+ #  – use the “-i” flag to display the variables forgemeta reads and sets

+ #  – use the “-v” flag if you want verbose processing

+ #  – remove  “-i” and “-v” before commit

+ %forgemeta -a

+ 

+ # The following lines use variables computed by forgemeta as default values.

+ # You can replace them with manual definitions.

+ # Release ordering is controlled by the packager with x%{?dist}/0.x%{?dist}

+ # numbers chains.

+ Name:    

+ Release: 1%{?dist}

+ Summary: 

+ URL:     %{forgeurl0}

+ # https://github.com/rpm-software-management/rpm/issues/539

+ Source0: %{forgesource0}

+ Source1: %{forgesource1}

+ Source2: %{forgesource2}

+ %description

+ 

+ 

+ %prep

+ #  – use the “-a” flag to process all the source archives in a single pass

+ #  – use     “-z <number>” to process only the declaration block suffixed with

+ #    <number>

+ #  — without “-a” of “-z <number>” only the main archive is processed

+ #  – an autosetup wrapper, forgeautosetup, is also provided

+ #  – forgeautosetup does not understand “-a”

+ #  – use the “-v” flag if you want verbose processing; remove it before commit

+ #  – call forgesetup/autosetup directly if you do not like the result

+ %forgesetup -a

+ #

+ # After this point the archive extraction is done. forgemeta is no longer used.

+ 

+ %build

+ 

+ %install

+ 

+ %check

+ 

+ %files

+ %license 

+ %doc 

+ 

+ %changelog

@@ -0,0 +1,49 @@ 

+ # This template shows how to package a branch state using the forgemeta macro.

+ #

+ # BRANCH STATES ARE NOT REPRODUCIBLE AND SHOULD NEVER BE SHARED WITH OTHERS.

+ #

+ # For more explanations on forgemeta, read the forgemeta-release template.

+ #

+ # The project url on the forge

+ %global forgeurl 

+ #

+ # The branch being packaged

+ %global branch   

+ #

+ #  – use the “-i” flag to display the variables forgemeta reads and sets

+ #  – use the “-v” flag if you want verbose processing

+ #  – remove  “-i” and “-v” before commit

+ %forgemeta

+ 

+ # The following lines use variables computed by forgemeta as default values.

+ # You can replace them with manual definitions.

+ # forgemeta will prepend branch information to dist. Release ordering is

+ # controlled by the packager with x%{?dist}/0.x%{?dist} number chains.

+ Name:    

+ Version: 

+ Release: 1%{?dist}

+ Summary: 

+ URL:     %{forgeurl}

+ Source:  %{forgesource}

+ %description

+ 

+ 

+ %prep

+ #  – an autosetup wrapper, forgeautosetup, is also provided

+ #  – use the “-v” flag if you want verbose processing; remove it before commit

+ #  – call forgesetup/autosetup directly if you do not like the result

+ %forgesetup

+ #

+ # After this point the archive extraction is done. forgemeta is no longer used.

+ 

+ %build

+ 

+ %install

+ 

+ %check

+ 

+ %files

+ %license 

+ %doc 

+ 

+ %changelog

So I've pushed following:

  • 5335ee5 common: add a writevars helper
  • 353f8b9 common: add a setcurrent helper
  • 7b6ec46 forge: fix handling of / in refs as showcased by github.com/dnstap/golang-dnstap
  • d6f7114 forge: keep distprefixes lowercase, even for not primary ones

I didn't push the rest because:

  • %cleandescription has nothing to do with description, it's about doing something with text.
  • Improve %setfiles argparsing is not there because it uses inconsistent naming inc / ex while the rest of macro uses exclude, exclude_from_file and such.
  • Templates are not there because I don't think it has anything to do with redhat-rpm-config.
  • Changing Provides and naming are not there due to obvious reasons.

As documented in https://github.com/rpm-software-management/rpm/issues/566 %cleandescription is used to clean all the empty lines added in %descriptions by rpm expands, because upstream rpm does not want to bother cleaning them up and asked for the filtering to be done in some other place

How do you want the macro to be named?

I can change the two variables in %setfiles , they’re low level enough they don’t need typing every day. Many such variables in Go and Fonts macro do need regular typing by packagers though, and they are already long enough without shortening the suffixes is a usability requirements, so they will diverge from those.

Lastly, putting the templates FPC side only means months of review to clarify any bit of phrasing, so I guess no one will bother improving them in the future.

1 new commit added

  • use longer variable names in listfiles as requested in https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/51
7 months ago

rebased onto e6faeaa8c238d54d6b08cb37b6ed37bf8e50fc6d

5 months ago

The 's' is here to mark the variables contain lists of elements, they're not single-valued. It's not like an exclude keyword in an rpm filelist where you can add as many lines as files to be excluded, it you redefine the variable, you overwrite its previous value;

That being said, if you confirm you want the s gone anyway, I will remove it, whatever it gets to get it merged at last.

I've asked initially rename it because it is regex in all other RPM macro which I was supposed would be here too.

The variables are not holding regexes they're holding space separated shell globs

An example of listfiles_includes for fonts would be something like

IBM-Plex-Sans/fonts/complete/otf/*otf IBM-Plex-Sans-Condensed/fonts/complete/otf/*otf IBM-Plex-Sans-Hebrew/fonts/complete/otf/*otf

(for IBM Plex Sans)

It's shell globs because that's what people are used to when manipulating files, and the shell is good at finding what's actually on the filesystem

As a random remark of the day...

  • A directory for spec template examples

  • %_spectemplatedir %{_sysconfdir}/rpmdevtools
    ...
  • %config(noreplace) %{_spectemplatedir}/*.spec

Example templates, as config files, in etc? Are these really supposed to be modified on per-host basis? Just thinking that templates sounds like something that would belong someplace in /usr/share, non-config.

As a random remark of the day...

A directory for spec template examples

%_spectemplatedir %{_sysconfdir}/rpmdevtools
...
%config(noreplace) %{_spectemplatedir}/*.spec

Example templates, as config files, in etc? Are these really supposed to be modified on per-host basis? Just thinking that templates sounds like something that would belong someplace in /usr/share, non-config.

Hi @ pmatilai

Thanks for taking a look at things
Right now the variable just points at where rpmdevtools puts its own spec templates since that's the closer thing we have to a spec template library today

If you prefer I can redefine it to /usr/share/rpm/templates ? I don't much care as long as the default template location is specified in redhat-rpm-config so people do not have to invent inconsistent conventions.

What directory location would please you most ?

Right, didn't realize rpmdevtools has uses it for such things, that seems equally non-optimal.
Maybe /usr/share/rpmdevtools, and then update rpmdevtools to use that too where appropriate (there seems to be actual config material in /etc/rpmdevtools too)

I don't much care about the exact directory either, but please don't steal rpm namespace, rpm itself will want /usr/share/rpm sooner or later. Just wanted to point out that the distro as a whole has been moving this kind of not-really-config stuff out of /etc for quite some time now.

I think rpmdevtools puts it in /etc because people are supposed to be able to customize templates for their own needs. But that could be handled the same way as fontconfig, systemd etc with the system canonical version in /usr/share and the user overlay in /etc.

I will update the pr with /usr/share/rpmdevtools

I've asked initially rename it because it is regex in all other RPM macro which I was supposed would be here too.

Anyway I will remove the s just to be done with it

rebased onto f2d796504ed1a6908605db3bf737f2ba7d762388

5 months ago

Anyway all the requested changes have been made, and you have a copr to check the result
https://copr.fedorainfracloud.org/coprs/nim/fonts-rpm-macros/

(the go part really needs rpm issue 104 finished)

Can it be merged now? All this rebasing and retesting is a lot of work, and some of the dependant packages should really have been updated in Fedora months/years ago

Can this finally be merged? The requested changes have been made long ago.

The macro enhancements are necessary to push
https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/
forward

I am not sure but maybe @tibbs can help here to review nim's changes?

1 new commit added

  • forge: add pagure driver
3 months ago

11 new commits added

  • forge: add pagure driver
  • misc: add a reformatdescriptiontext macro helper
  • misc: move spec templates to %{_datadir} as requested in https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/51
  • misc: listfiles: improve argument passing, simplify it and make it more robust
  • misc: writevars: fix code formatting
  • forge: workaround bz1655322
  • forge: rename templates so they are ordered sensibly
  • forge: fix whitespace in templates
  • forge: document %{load:%{SOURCE100}} in the spec file
  • forge: clean up naming to be more generic and better fit with other macro packages in the distribution
  • forge: add some templates and documentation
3 months ago

@nim The only reason rpmdevtools has it in /etc is because I haven't gotten around to fixing it. It's not going to stay there. I intend to move it to /usr/share soon. I'm currently thinking /usr/share/rpmdevtools/templates for these. Perhaps put them in %{_datadir}/rpmdevtools/templates/%{_vendor}?

@ngompa right now the PR uses %{_datadir}/rpmdevtools as suggested by @pmatilai

I honestly do not care what the value is as long as it is specified somewhere so templates are discoverable by packagers without a lot of filesystem search. Can you just tell me what to put in the PR? It's been holding back two packaging reworks and hundreds of badly needed Fedora package cleanups and updates for months now.

rebased onto 27969b0e20e2c607ca4ff1acb777ca2f7d14ca03

3 months ago
⋊> ~/P/f/r/redhat-rpm-config on kkt ⨯ rpm --eval '%{reformatdescriptiontext:foo  \'                                                                                         17:12:11

                                      bar 

                                      baz}'
(sed 's|^[[:blank:]]*$||g' | sed '/^$/N;/^\n$/D' | sed '/./,$!d' | tac | sed '/./,$!d' | tac | fold -s)

it doesn't really work.

I've merged other parts. I'm not going to merge templates until you sort it out with rpmdevtools maintainer.

Will some version of the forge macros with pagure support (i.e., the forge: add pagure driver commit) ever be available in F29?

rebased onto b2e0d3357733c8919fe4c1b74a1b8dfc2a759e57

2 months ago

3 new commits added

  • forge: add some templates and documentation
  • misc: add a filterdescr macro helper
  • misc: listfiles: improve argument passing, simplify it and make it more robust
2 months ago

rebased onto e37bc9363bcee99cfd16c3bea8f130ecff7b7bb8

2 months ago

@ignatenkobrain
The description filtering has been rewritten in lua as requested. Please check and merge

1 new commit added

  • misc: protect listfiles against empty input, to avoid harmless but annoying messages in build logs
2 months ago

3 new commits added

  • misc: protect listfiles against empty input, to avoid harmless but annoying messages in build logs
  • misc: add a filterdescr helper, lua style
  • forge: add some templates and documentation
2 months ago

Ok, I merged everything with some changes except of templates. As of now, do not put any more fixes which are not related to templates into this branch.

Thanks.

@ignatenkobrain Thanks for the merging. That unlocks the mass Golang rebuild in FC31 (and maybe quite a lot of font package updates but not sure they will make it at this late hour)

The remaining part depends on @ngompa then

rebased onto f2678d1

11 days ago