#25 Create shared, sysusers, tmpfiles and network sub-packages
Closed 9 months ago by lorbus. Opened 2 years ago by lorbus.
rpms/ lorbus/systemd sysusers  into  rawhide

file modified
+12
@@ -23,6 +23,10 @@ 

  o_container = open('.file-list-container', 'w')

  o_remote = open('.file-list-remote', 'w')

  o_tests = open('.file-list-tests', 'w')

+ o_shared = open('.file-list-shared', 'w')

+ o_networkd = open('.file-list-networkd', 'w')

+ o_sysusers = open('.file-list-sysusers', 'w')

+ o_tmpfiles = open('.file-list-tmpfiles', 'w')

  o_rest = open('.file-list-rest', 'w')

  for file in files(buildroot):

      n = file.path[1:]
@@ -51,6 +55,14 @@ 

          o = o_pam

      elif '/rpm/' in n:

          o = o_rpm_macros

+     elif re.search(r'/libsystemd-shared-...\.so$|/lib/systemd$', n, re.X):

+         o = o_shared

+     elif re.search(r'networkd(?!.*\.conf)', n, re.X):

+         o = o_networkd   

+     elif re.search(r'sysusers(?!.*\.conf)', n, re.X):

+         o = o_sysusers

+     elif re.search(r'tmpfiles(?!.*\.conf)', n, re.X):

+         o = o_tmpfiles

      elif re.search(r'/lib.*\.pc|/man3/|/usr/include|(?<!/libsystemd-shared-...).so$', n):

          o = o_devel

      elif '/usr/lib/systemd/tests' in n:

file modified
+79 -3
@@ -20,7 +20,7 @@ 

  Name:           systemd

  Url:            https://www.freedesktop.org/wiki/Software/systemd

  Version:        246.4

- Release:        1%{?dist}

+ Release:        2%{?dist}

  # For a breakdown of the licensing, see README

  License:        LGPLv2+ and MIT and GPLv2+

  Summary:        System and Service Manager
@@ -150,6 +150,10 @@ 

  Requires:       %{name}-pam = %{version}-%{release}

  Requires:       %{name}-rpm-macros = %{version}-%{release}

  Requires:       %{name}-libs = %{version}-%{release}

+ Requires:       %{name}-shared = %{version}-%{release}

+ Requires:       %{name}-sysusers = %{version}-%{release}

+ Requires:       %{name}-tmpfiles = %{version}-%{release}

+ Recommends:     %{name}-networkd = %{version}-%{release}

  Recommends:     diffutils

  Requires:       util-linux

  Recommends:     libxkbcommon%{?_isa}
@@ -307,6 +311,58 @@ 

  This package contains systemd-journal-gatewayd,

  systemd-journal-remote, and systemd-journal-upload.

  

+ %package networkd

+ Summary:        A system service that manages network configurations

+ Requires:       %{name}%{?_isa} = %{version}-%{release}

+ License:        LGPLv2+

+ Requires(post):   systemd

+ Requires(preun):  systemd

+ Requires(postun): systemd

+ 

+ %description networkd

+ %{summary}.

+ 

+ It detects and configures network devices as they appear,

+ as well as creating virtual network devices.

+ 

+ %package shared

+ Summary:       Shared library object for systemd

+ License:       LGPLv2+

+ 

+ %description shared

+ This package contains libsystemd-shared-%(c=%{version}; echo ${c:0:2}).so

+ and owns the %{_prefix}/lib/systemd directory.

+ 

+ %package sysusers

+ Summary:       Declarative allocation of system users and groups

+ Requires:      %{name}-shared%{?_isa} = %{version}-%{release}

+ License:       LGPLv2+

+ 

+ %description sysusers

+ Systemd-sysusers uses the files from sysusers.d directory to create system

+ users and groups and to add users to groups, at package installation or boot

+ time.

+ 

+ This tool may be used to allocate system users and groups only, it is not

+ useful for creating non-system (i.e. regular, "human") users and groups,

+ as it accesses /etc/passwd and /etc/group directly, bypassing any more 

+ complex user databases, for example any database involving NIS or LDAP.

+ 

+ %package tmpfiles

+ Summary:       Creates, deletes and cleans up volatile and temporary files and directories

+ Requires:      %{name}-shared%{?_isa} = %{version}-%{release}

+ License:       LGPLv2+

+ 

+ %description tmpfiles

+ Systemd-tmpfiles uses the files from tmpfiles.d to create volatile files

+ and directories during boot and to do periodic cleanup afterwards.

+ 

+ tmpfiles.d configuration files provide a generic mechanism to define the

+ creation of regular files, directories, pipes, and device nodes, adjustments

+ to their access mode, ownership, attributes, quota assignments, and contents,

+ and finally their time-based removal. It is mostly commonly used for volatile

+ and temporary files and directories.

+ 

  %package tests

  Summary:       Internal unit tests for systemd

  Requires:      %{name}%{?_isa} = %{version}-%{release}
@@ -627,8 +683,6 @@ 

                  serial-getty@.service \

                  console-getty.service \

                  debug-shell.service \

-                 systemd-networkd.service \

-                 systemd-networkd-wait-online.service \

                  systemd-resolved.service \

                  systemd-homed.service \

                  >/dev/null || :
@@ -763,6 +817,14 @@ 

  %systemd_postun_with_restart systemd-journal-upload.service

  %firewalld_reload

  

+ %preun networkd

+ if [ $1 -eq 0 ] ; then

+         systemctl disable --quiet \

+                 systemd-networkd.service \

+                 systemd-networkd-wait-online.service \

+                 >/dev/null || :

+ fi

+ 

  %global _docdir_fmt %{name}

  

  %files -f %{name}.lang -f .file-list-rest
@@ -801,9 +863,23 @@ 

  

  %files journal-remote -f .file-list-remote

  

+ %files networkd -f .file-list-networkd

+ 

+ %files shared -f .file-list-shared

+ 

+ %files sysusers -f .file-list-sysusers

+ 

+ %files tmpfiles -f .file-list-tmpfiles

+ 

  %files tests -f .file-list-tests

  

  %changelog

+ * Mon Sep 7 2020 Christian Glombek <lorbus@fedoraproject.org> - 246.4-2

+ - Create shared, sysusers and tmpfiles sub-packages:

+   Splits libsystemd-shared object, systemd-sysusers and systemd-tmpfiles

+   into their own subpackages

+ - Split out networkd sub-package and add to main package as recommended dependency

+ 

  * Wed Sep  2 2020 Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> - 246.4-1

  - Update to latest stable version: a rework of how the unit cache mtime works

    (hopefully #1872068, #1871327, #1867930), plus various fixes to

I've been meaning to do this for a while. I haven't tested the build, please let me know if I'm missing anything.

This stems from Lennart's comment here: https://github.com/systemd/systemd/issues/13653#issuecomment-549165547

Might be worth also doing it for tmpfiles, as suggested by Lennart in the comment

rebased onto 7267757ca6d993587b3f6db0501ad000ac19f729

2 years ago

rebased onto fa9c04af4e0a5d4d55375dd9f907e60f6431e8a5

2 years ago

rebased onto 17e0c118b626e0db48a1f0893343daa53bf102a5

2 years ago

rebased onto 8dbe7b2a4577ee237f5b7df2c85227d02ea7ef4e

2 years ago

I fumbled with the file list regex a bit, the latest build from koji-ci now shows the sysusers subpackage to contain this:

$ rpm -ql systemd-sysusers-245.5-3.fc33.x86_64.rpm 
/usr/bin/systemd-sysusers
/usr/lib/.build-id
/usr/lib/.build-id/5e/0057dc3ad69b11d32e822aa39b089f70e365bb
/usr/lib/systemd/system/sysinit.target.wants/systemd-sysusers.service
/usr/lib/systemd/system/systemd-sysusers.service
/usr/lib/sysusers.d
/usr/share/man/man5/sysusers.d.5.gz
/usr/share/man/man8/systemd-sysusers.8.gz
/usr/share/man/man8/systemd-sysusers.service.8.gz

Sorry, but this won't work at all like this. systemd-sysusers is linked to systemd-shared-nnn.so, which is part of systemd.rpm. So either systemd-sysusers would need to be compiled without that dependency (which would blow up the size of the binary a lot), or systemd-shared needs to be moved too, or systemd-shared needs to be moved a new package that both systemd-shared.rpm and systemd.rpm depend on...

Thanks for reviewing this! I'd prefer to split out systemd-shared into another subpackage then. The goal of this is to be able to install sysusers with as few dependencies as possible.

That would still pull in all the libraries. So the savings might be rather small.
... I'm not against, but I would like to see how the sizes turn out.

rebased onto 2decaa52370b19bfb8eb307b76510282427dad29

2 years ago

I've split out the libsystemd shared object into systemd-shared, and sysusers and tmpfiles into their own sub-packages, with a dependency on the shared sub-package.

Do sysusers and tmpfiles depend on anything in systemd-libs or is the libsystemd-shared-nnn.so all they need?

I wonder if it would be better to expose a static build of systemd-sysusers (from upstream Meson) and then build a separate RPM package (potentially from a separate spec file) with both the new package and the systemd package providing a working /usr/bin/systemd-sysusers.

(I can think of a similar approach in packages coreutils-single or glibc-minimal-langpack, which I know exist on RHEL 8, and provide alternative versions of their corresponding packages, but with some slightly different build options...)

rebased onto 1a817814c4de47f3ea56327e300112ea0badab18

2 years ago

cqoreutils-single is the only example I know about. glibc-minimal-langpack is just an empty package the satisfies the dependencies. I'm all for having a standalone version if this package. May want a Suggests in place so users normally get the shared version.

See #27 for an alternative approach.

(Note that it requires an upstream PR to be merged and released first.)

Makes sense to me. I'm closing this PR in favor of #27

Pull-Request has been closed by lorbus

2 years ago

Pull-Request has been reopened by lorbus

2 years ago

Reopening this. Can one of the packagers please take a look?

I now feel this is orthogonal to #27

If #27 gets pushed then you'll get a new systemd-standalone package that doesn't depend on any of the other systemd packages (i.e. doesn't pull in systemd-libs) and will contain only two binaries systemd-sysusers and systemd-tmpfiles.

Wasn't that what you really wanted with this PR?

I think I'd like #27 to ship systemd-sysusers-standalone and systemd-tmpfiles-standalone packages separately.

Those would then cleanly conflict with systemd-sysusers and systemd-tmpfiles from this PR.

Yes, we can do separate systemd-standalone-sysusers and systemd-standalone-tmpfiles if you think you might need only one or the other not both in some use cases...

But I don't understand why you think you might still want to break the current systemd RPMs (full installation) to have a separate sysusers or tmpfiles that still depend on systemd-libs. Can you clarify?

In which case would you use the systemd-sysusers or systemd-tmpfiles that depend on systemd-libs in which it's not OK to install the whole of the systemd package with the system manager and all other binaries?

That would allow systems where the standalone binary is installed to also install the rest of systemd later on, without having to replace the standalone binary with the shared one (this is especially useful for rpm-ostree based OSes, where you'd need to explicitly replace the package when installing systemd.

Would the shared, but split out binaries actually require systemd-libs? I was under the impression they'd only need systemd-shared containing only libsystemd-shared which is also introduced in this PR.

I think we can make that work without breaking the packages.

Perhaps by adding Obsoletes: systemd-standalone-sysusers (and tmpfiles) in the main systemd package, then dnf install systemd will automatically remove the other package? (Not sure this is how this works...)

In any case, I think I'd prefer to avoid breaking this down into too many multiple packages, I think there's a certain amount of pain that gets added with each package that you add and I think avoiding that would be good... But I don't know, maybe that's just my 2c...

In any case, looks like we'll get the "standalone" change upstream, and shortly after I'll pick up #27 to see how we can get that one sorted out. (Totally fine with me to split that one into separate packages for sysusers and tmpfiles. And eventually systemctl it seems.)

Cheers!
Filipe

rebased onto 43c2a09

2 years ago

What value does this bring? This seems to be particularly complex for no effective space savings, added complexity for providing systemd services, and makes it difficult to be certain that we actually have everything needed installed in systemd environments.

I'd prefer to change the build config to break systemd-shared.so into 2 parts: one without the extra deps, and a depending one that adds in the separate deps. Then you can make a subpackage of the "core" tools without bringing in all the deps. This would avoid the alternative packages. I find the coreutils-single approach to be sub-optimal, including not inheriting (AFAICT) the FIPS certification for the hash functions, with no link to a certified library (openssl when certified)

We actually have the upstream changes we need for #27, so I'd suggest we should go with that one instead.

The commit in PR #27 creates separate standalone packages for tools people might want to use on non-systemd systems (or containers not running systemd as PID 1) but without affecting the normal installation, which keeps working the same as it does today...

I'll follow up and update that one. Once we merge that, we can look into dropping this one.

@lorbus Please don't split sysusers and tmpfiles into their own subpackages. Instead, if you must split this out, a generic -systools subpackage would be better, since the singular -systools package can be forced into the minimal environment so that scriptlets still work.

@ngompa this shouldn't change how scriplets work -- the systemd main package still requires sysusers and tmpfiles subpackages and therefore pulls them in. This is only intended to add the ability to install them standalone, if needed.

@filbranden I still see this PR and #27 as orthogonal - with #27 one gets a small size benefit when installing stand-alone but one cannot install the main systemd package where those standalone packages are installed without removing them (which is especially important when layering rpms onto an rpm-ostree). This approach here should allow for a conflict-free main package installation on systems where the sysusers and/or tmpfiles sub-package is already installed.

I've split out the networkd sub-package commit #34 since it's not really related to the sysusers/tmpfiles split.

@lorbus We don't have the systemd main package in container base images, and we would want these tools in there, especially with @zbyszek's recent changes to make it so %systemd_requires isn't used as much in our packages.

@ngompa I'm still failing to see your point here - with this PR both sysusers and tmpfiles could be installed within a container with only a dependency on libsystemd shared object, which is also split out into a subpackage.

For your container use-case, @filbranden's PR #27 is probably best suited, as it would allow installing sysusers and tmpfiles as self-contained packages without any shared object dependencies.

With this here, all packages requiring sysusers could now just add a Requires: systemd-sysusers, without declaring a dependency on the rest of systemd.

@lorbus #27 is unnecessary, generally speaking. I don't particularly care for statically linking libsystemd into tools. My point is that we're probably going to change things very soon to require and expect sysusers and tmpfiles to exist in all environments for packages...

@ngompa Regarding the tools with statically linked libsystemd, there's a significant total size difference and also dependency on other shared libraries that get pulled or not for the standalone binaries.

See for example:
https://github.com/systemd/systemd/pull/16061#issuecomment-641566933

Sorry for the delay. #25 and #34 are now merged. I think we should let the dust settle on those changes before doing further enhancements. So this PR might be still be relevant, but I'll return to it later.

So... is there something left to merge here? Or should we close this and pursue further changes in separate PRs?

Oh, this totally fell under table on my side (thanks for the reminder, Dusty!). Looks like this can be closed indeed.

Pull-Request has been closed by lorbus

9 months ago