#9 init FMF CI gating
Merged a month ago by kevinb. Opened 7 months ago by mkolar.
rpms/ mkolar/gdb main  into  main

file added
+1
@@ -0,0 +1,1 @@ 

+ 1

file added
+19
@@ -0,0 +1,19 @@ 

+ --- !Policy

+ product_versions:

+   - fedora-*

+ decision_context: bodhi_update_push_stable

+ subject_type: koji_build

+ rules:

+   - !PassingTestCaseRule {test_case_name: fedora-ci.koji-build.tier0.functional}

+ --- !Policy

+ product_versions:

+   - rhel-8

+ decision_context: osci_compose_gate

+ rules:

+   - !PassingTestCaseRule {test_case_name: osci.brew-build.tier0.functional}

+ --- !Policy

+ product_versions:

+   - rhel-9

+ decision_context: osci_compose_gate

+ rules:

+   - !PassingTestCaseRule {test_case_name: osci.brew-build.tier0.functional}

file added
+6
@@ -0,0 +1,6 @@ 

+ summary: CI Gating Plan

+ discover:

+     how: fmf

+     directory: tests

+ execute:

+     how: beakerlib
psss commented 6 months ago

The 'beakerlib' execute method has been deprecated.
Use 'how: tmt' in the execute step instead (L2).
Set 'framework: beakerlib' in test metadata (L1).

https://tmt.readthedocs.io/en/latest/spec/tests.html#framework

file removed
-31
@@ -1,31 +0,0 @@ 

- Justification

- 

- Adds tests according to the CI wiki specifically the standard test

- interface in the spec.

- 

- The playbook includes Tier1 level test cases that have been tested in

- the following contexts and is passing reliably: Classic and

- Container. Test logs are stored in the artifacts directory.

- 

- The following steps are used to execute the tests using the standard

- test interface: Test environment

- 

- Make sure you have installed packages from the spec

- 

-     # rpm -q ansible python2-dnf libselinux-python standard-test-roles

-     ansible-2.3.2.0-1.fc26.noarch

-     python2-dnf-2.6.3-11.fc26.noarch

-     libselinux-python-2.6-7.fc26.x86_64

-     standard-test-roles-2.4-1.fc26.noarch

- 

- Run tests for Classic (must be run as root)

- 

-     # export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory)

-         # ansible-playbook --tags=classic tests.yml

- 

- Run tests for Container (must be run as root)

- 

-     # export ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory)

-     # export TEST_SUBJECTS=docker:docker.io/library/fedora:26

-         # ansible-playbook --tags=container tests.yml

- 

@@ -1,4 +1,4 @@ 

- sta

+ start

  si 30

  n 10

  up

@@ -0,0 +1,17 @@ 

+ summary: Debug a system binary.

+ description: ''

+ contact:

+ - Marek Polacek <mpolacek@redhat.com>

+ component:

+ - gdb

+ test: ./runtest.sh

+ framework: beakerlib

+ recommend:

+ - gdb

+ - coreutils

+ - coreutils-debuginfo

+ - glibc

+ - glibc-debuginfo

+ duration: 5m

+ extra-summary: /tools/gdb/Sanity/debug-system-binary

+ extra-task: /tools/gdb/Sanity/debug-system-binary

@@ -27,8 +27,7 @@ 

  # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  

  # Include Beaker environment

- . /usr/bin/rhts-environment.sh || exit 1

- . /usr/lib/beakerlib/beakerlib.sh || exit 1

+ . /usr/share/beakerlib/beakerlib.sh || exit 1

  

  CMD="gdb"

  BIN="$(which $CMD)"

@@ -14,5 +14,4 @@ 

  b xexit

  c

  c

- c

  q

@@ -0,0 +1,22 @@ 

+ summary: We debug a binary that is delivered via the Toolset.

+ description: ''

+ contact:

+ - Marek Polacek <mpolacek@redhat.com>

+ component:

+ - gdb

+ test: ./runtest.sh

+ framework: beakerlib

+ recommend:

+ - gdb

+ - coreutils

+ - binutils

+ - binutils-debuginfo

+ - glibc

+ - glibc-debuginfo

+ - gcc

+ - gcc-debuginfo

+ - zlib

+ - zlib-debuginfo

+ duration: 5m

+ extra-summary: /tools/gdb/Sanity/debug-toolset-binary

+ extra-task: /tools/gdb/Sanity/debug-toolset-binary

@@ -32,8 +32,7 @@ 

  # but in fact, it doesn't have to.  It should not fail either way.

  

  # Include Beaker environment

- . /usr/bin/rhts-environment.sh || exit 1

- . /usr/lib/beakerlib/beakerlib.sh || exit 1

+ . /usr/share/beakerlib/beakerlib.sh || exit 1

  

  CMD="gdb"

  BIN="$(which $CMD)"

file removed
-3
@@ -1,3 +0,0 @@ 

- #!/bin/bash

- export TEST_DOCKER_EXTRA_ARGS="--privileged"

- exec merge-standard-inventory "$@"

file removed
-24
@@ -1,24 +0,0 @@ 

- ---

- # This first play always runs on the local staging system

- - hosts: localhost

-   roles:

-   - role: standard-test-beakerlib

-     tags:

-     - classic

-     - container

-     tests:

-     - debug-toolset-binary

-     - debug-system-binary

-     required_packages:

-     - gdb

-     - coreutils

-     - binutils

-     - binutils-debuginfo

-     - glibc

-     - glibc-debuginfo

-     - gcc

-     - gcc-debuginfo

-     - zlib

-     - zlib-debuginfo

-     - coreutils-debuginfo

-     - which

no initial comment

Should we be concerned about the "Fedora CI - dist-git test" error?

(If not, I have no problem with merging it.)

This looks like an infrastructure issue, but please leave the request open until the investigation is complete.

Any update on the "Fedora CI - dist-git test" error investigation?

Should I merge this patch?

I am still waiting for feedback from CI guys. Because it fails due to missing debuginfo dependencies.

We definitely do not want to install debuginfo by default, looking around how we could solve this

right, the dependencies are correct recommend ... looking if we could enable debuginfo repositories by default, so no changes to tmt are needed ....

anyway, filed it to tmt:

https://github.com/psss/tmt/issues/624

What is the correct solution in my POW.

Trying to add a workaround that would unblock you until it is resolved.

The 'beakerlib' execute method has been deprecated.
Use 'how: tmt' in the execute step instead (L2).
Set 'framework: beakerlib' in test metadata (L1).

https://tmt.readthedocs.io/en/latest/spec/tests.html#framework

Is this okay to merge now? If so I'll do it, but I don't know enough about the infrastructure to evaluate the changes being made.

I would recommend to fix the execute method as commented above. Regarding the debuginfo packages, we don't have the support in tmt implemented yet. Not sure if @mvadkert was able to workaround this somehow in the pipeline.

@mkolar @kevinb would it make sense to try using debuginfod instead of installing the debugging information locally? Here is how debuginfod is used with systemtap gating. It should be just a matter of exporting DEBUGINFOD_URLS. Thoughts?

@mcermak, I think it's worth a try...

@mcermak, I'd rather wait until tmt-1.6 (May 31), which is scheduled to implement a proper debuginfo installation.

1 new commit added

  • drop unwanted scratch build rule for gating
3 months ago

@mcermak, I'd rather wait until tmt-1.6 (May 31), which is scheduled to implement a proper debuginfo installation.

What's the status on this? Is it ready to merge?

Michal is on PTO now, so we'll need to wait for him. I think using debuginfod would be perfectly fine, but Michal wants to go another way. Let's give him a space to got the way he prefers. Thanks everyone.

The support for installing debuginfo packages in tmt has been released at the beginning of June and local run seems to be working fine for the config in this pull request:

> tmt run -av provision -h container -i fedora:rawhide
/var/tmp/tmt/run-009
Found 1 plan.

/plans/ci
summary: CI Gating Plan
    warn: The 'beakerlib' execute method has been deprecated.
    warn: Use 'how: tmt' in the execute step instead (L2).
    warn: Set 'framework: beakerlib' in test metadata (L1).
    warn: Support for old methods will be dropped in March 2021.
    discover
        how: fmf
        directory: /tmp/gdb
        hash: 93e3bd5
        summary: 2 tests selected
            /tests/debug-system-binary
            /tests/debug-toolset-binary
    provision
        how: container
        image: fedora:rawhide
        name: tmt-var-tmp-tmt-run-009-plans-ci
        distro: Fedora 35 (Container Image Prerelease)
        kernel: 5.12.12-300.fc34.x86_64
        summary: 1 guest provisioned
    prepare
        how: install
        summary: Install required packages
        name: requires
        package: 1 package requested
            beakerlib
        how: install
        summary: Install recommended packages
        name: recommends
        package: 6 packages requested
            binutils
            coreutils
            gcc
            gdb
            glibc
            zlib
        debuginfo: 5 packages requested
            binutils
            coreutils
            gcc
            glibc
            zlib
        summary: 2 preparations applied
    execute
        how: tmt
            00:00:01 /tests/debug-system-binary [1/2]
            00:00:01 /tests/debug-toolset-binary [2/2]
        summary: 2 tests executed
    report
        how: display
            pass /tests/debug-system-binary
            pass /tests/debug-toolset-binary
        summary: 2 tests passed
    finish
        container: stopped
        container: removed
        summary: 0 tasks completed

total: 2 tests passed

@thrix, is the fresh tmt used in the Testing Farm?

I would just recommend to update the plan config in this way:

summary: CI Gating Plan
discover:
    how: fmf
execute:
    how: tmt

The beakerlib execute method has been obsoleted and there was never directory key for the discover step.

@psss no still 1.5 :( to update to 1.6 we need to move to RHEL8 ubi image base :/ I do not think it will be done until I leave for PTO on Friday sorry

@keiths, It is only waiting for tmt-1.6.

@mkolar @mcermak tmt-1.6 is now enabled in Fedora CI

tmt 1.6 is out. retesting

Looking at the build logs (for the failed scratch build), the problem seems to be gnulib related. This really puzzled me, because I cannot see how this would happen using current sources. I noticed that the build is using gdb-10.1-5.fc35.src.rpm. This isn't right; gdb-10.2-6 is the current version for f35 / rawhide.

Using gdb-10.2-6, I've performed my own scratch build on rawhide. It works. See:

https://koji.fedoraproject.org/koji/taskinfo?taskID=73396705

rebased onto 96774cf

2 months ago

Pull-Request has been merged by kevinb

a month ago