#1 Package tests
Merged 3 years ago by jplesnik. Opened 3 years ago by jplesnik.
Unknown source rawhide  into  rawhide

file added
+1
@@ -0,0 +1,1 @@

+ 1

file added
+7
@@ -0,0 +1,7 @@

+ --- !Policy

+ product_versions:

+   - fedora-*

+ decision_context: bodhi_update_push_stable

+ subject_type: koji_build

+ rules:

+   - !PassingTestCaseRule {test_case_name: fedora-ci.koji-build.tier0.functional}

file modified
+48 -7
@@ -1,6 +1,6 @@

  Name:           perl-Sys-Virt

- Version:        7.0.0

- Release:        2%{?dist}

+ Version:        7.1.0

+ Release:        1%{?dist}

  Summary:        Represent and manage a libvirt hypervisor connection

  License:        GPLv2+ or Artistic

  URL:            https://metacpan.org/release/Sys-Virt
@@ -9,6 +9,7 @@

  BuildRequires:  coreutils

  BuildRequires:  findutils

  BuildRequires:  gcc

+ BuildRequires:  git

  BuildRequires:  libvirt-devel >= %{version}

  BuildRequires:  perl-devel

  %if 0%{?fedora} || 0%{?rhel} > 7
@@ -30,11 +31,10 @@

  BuildRequires:  perl(Time::HiRes)

  BuildRequires:  perl(XML::XPath)

  BuildRequires:  perl(XML::XPath::XMLParser)

- # Optional tests only

- BuildRequires:  perl(Test::CPAN::Changes)

- BuildRequires:  perl(Test::Pod) >= 1.00

- BuildRequires:  perl(Test::Pod::Coverage) >= 1.00

- BuildRequires:  git

+ # Optional tests are not executed

+ #BuildRequires:  perl(Test::CPAN::Changes)

+ #BuildRequires:  perl(Test::Pod) >= 1.00

+ #BuildRequires:  perl(Test::Pod::Coverage) >= 1.00

  Requires:       perl(:MODULE_COMPAT_%(eval "$(perl -V:version)"; echo $version))

  

  %description
@@ -42,9 +42,33 @@

  machine management APIs. This allows machines running within arbitrary

  virtualization containers to be managed with a consistent API.

  

+ %package tests

+ Summary:        Tests for %{name}

+ Requires:       %{name} = %{?epoch:%{epoch}:}%{version}-%{release}

+ Requires:       perl-Test-Harness

+ 

+ %description tests

+ Tests from %{name}-%{version}. Execute them

+ with "%{_libexecdir}/%{name}/test".

+ 

  %prep

  %autosetup -S git -n Sys-Virt-v%{version}

  

+ # Help file to recognise the Perl scripts and normalize shebangs

+ for F in t/*.t; do

+     if head -1 "$F" | grep -q -e '^#!.*perl' ; then

+         perl -MConfig -pi -e 's|^#!.*perl\b|$Config{startperl}|' "$F"

+     else

+         perl -i -MConfig -ple 'print $Config{startperl} if $. == 1' "$F"

+     fi

+     chmod +x "$F"

+ done

+ 

+ # Remove release tests

+ for F in t/005-pod.t t/010-pod-coverage.t t/015-changes.t; do

+     rm "$F"

+     perl -i -ne 'print $_ unless m{^\Q'"$F"'\E}' MANIFEST

+ done

  

  %build

  perl Build.PL installdirs=vendor
@@ -53,9 +77,20 @@

  %install

  ./Build install destdir=$RPM_BUILD_ROOT create_packlist=0

  find $RPM_BUILD_ROOT -type f -name '*.bs' -size 0 -delete

+ 

+ # Install tests

+ mkdir -p %{buildroot}/%{_libexecdir}/%{name}

+ cp -a t %{buildroot}/%{_libexecdir}/%{name}

+ cat > %{buildroot}/%{_libexecdir}/%{name}/test << 'EOF'

+ #!/bin/sh

+ cd %{_libexecdir}/%{name} && exec prove -I . -j "$(getconf _NPROCESSORS_ONLN)"

+ EOF

+ chmod +x %{buildroot}/%{_libexecdir}/%{name}/test

+ 

  %{_fixperms} $RPM_BUILD_ROOT/*

  

  %check

+ export HARNESS_OPTIONS=j$(perl -e 'if ($ARGV[0] =~ /.*-j([0-9][0-9]*).*/) {print $1} else {print 1}' -- '%{?_smp_mflags}')

  ./Build test

  

  %files
@@ -65,8 +100,14 @@

  %{perl_vendorarch}/Sys*

  %{_mandir}/man3/*

  

+ %files tests

+ %{_libexecdir}/%{name}

  

  %changelog

+ * Tue Mar 02 2021 Jitka Plesnikova <jplesnik@redhat.com> - 7.1.0-1

+ - 7.1.0 bump

+ - Package tests

+ 

  * Wed Jan 27 2021 Fedora Release Engineering <releng@fedoraproject.org> - 7.0.0-2

  - Rebuilt for https://fedoraproject.org/wiki/Fedora_34_Mass_Rebuild

  

file added
+5
@@ -0,0 +1,5 @@

+ summary: Sanity tests

+ discover:

+     how: fmf

+ execute:

+     how: tmt

file modified
+1 -1
@@ -1,1 +1,1 @@

- SHA512 (Sys-Virt-v7.0.0.tar.gz) = c0ab179af11526a9dc983374765bd04400821f6f334d22f56b1f6452f54f880fe6882e9e0e121ae36010eb305d0632d574c969febefd8a32c9a731879d8fadb6

+ SHA512 (Sys-Virt-v7.1.0.tar.gz) = b75196248b56dffc03b671383bbd4fe7ef2edab8b77dc3cf612af961d8270c7bc09ec8d125a8f581bf9cd2a85bb6309bc7ba435ef5224f0913ae4980741ae732

@@ -0,0 +1,4 @@

+ summary: Upstream tests

+ component: perl-Sys-Virt

+ require: perl-Sys-Virt-tests

+ test: /usr/libexec/perl-Sys-Virt/test

no initial comment

Pull-Request has been merged by jplesnik

3 years ago

I don't find this change desirable because it is impacting the user facing RPM packaging to satisfy non-user facing CI. I don't see much value in this CI either because it is just repeating tests that have already been run during the build, and can be re-run by Koschei when a dependancy changes.

Why did you just merge this when I did not agree to it ?

I merged because I am owner of the package. I want to enable gating for my packages and run at least upstream tests in this case.
You added your comment after I merge it.

I also agree that packaging the tests is costly comparing to the fact that they are already performed at build time. On the other hand the build-time tests are performed against a code in the working directory which, e.g. by a packaging mistake, can differ from a code shipped and installed by an RPM package. Hence the advantage of the packaged tests is that it indeed validates the code Fedora delivers.

It also eases regression testing: You can install the tests subpackage without hassling with building from sources. That's not so difficult in case of Perl packages, but prominently soothing for complex C or Rust or Go packages.

The packaged tests also open a possibility to perform a regression testing across multiple packages. This is currently achieved by Koschei, but Koschei can be quite slow, skips some architectures and the gap between packages in a build root and a testing repository can be quite large. There are prospects to execute the tests of the reverse dependencies on submitting updates into Bodhi. E.g. as part of the gating. That would catch regressions in the testing phase which is indeed the purpose of Bodhi. In contrast, Koschei sees the error after the broken package becomes stable. That's too late.

And I also heard an opinion that %check phases could be completely skipped at build time and replaced with the packaged tests. That would significantly improve bootstrapping Fedora by simplifying a dependency graph for building. The tests would be performed asynchronously later.