rpms / selinux-policy

Clone
Source Code
GIT

#226 Draft: Providing tmt version of the reboot test
Opened 2 years ago by mzeleny. Modified 2 years ago
rpms/ mzeleny/selinux-policy tests-reboot-fmf  into  rawhide

Merge
file added
+1 
@@ -0,0 +1,1 @@ 
 

+ 1

file added
+5 
@@ -0,0 +1,5 @@ 
 

+ summary: tmt selinux-policy tests
 

+ discover:
 

+     how: fmf
 

+ execute:
 

+     how: tmt

file added
+6 
@@ -0,0 +1,6 @@ 
 

+ summary: Check SELinux denials since boot
 

+ contact: Martin Zelený <mzeleny@redhat.com>
 

+ test: ./test.sh
 

+ framework: beakerlib
 

+ duration: 15m
 

+ enabled: true

file added
+25 
@@ -0,0 +1,25 @@ 
 

+ #!/bin/bash
 

+ # vim: dict+=/usr/share/beakerlib/dictionary.vim cpt=.,w,b,u,t,i,k
 

+ . /usr/share/beakerlib/beakerlib.sh || exit 1
 

+ 

+ rlJournalStart
 

+     if [ -z "${REBOOT_COUNT}" ] || [ "${REBOOT_COUNT}" -eq 0 ]; then
 

+         rlPhaseStartSetup "Setup before reboot"
 

+             rlRun "defaultKernel=\$(grubby --default-kernel)"
 

+             rlRun "grubby --args=enforcing=0 --update-kernel ${defaultKernel}"
 

+ 

+             rlRun "tmt-reboot"
 

+         rlPhaseEnd
 

+     else
 

+         rlPhaseStartTest "Test after reboot"
 

+             rlRun -s "dmesg"
 

+             rlAssertNotGrep 'type=1300' $rlRun_LOG -i
 

+             rlAssertNotGrep 'type=1400' $rlRun_LOG -i
 

+             rlFileSubmit $rlRun_LOG 'dmesg.log'
 

+ 

+             rlRun -s "ausearch -m avc -m selinux_err -m user_avc -ts boot" 1
 

+             rlAssertGrep '<no matches>' $rlRun_LOG
 

+             rlFileSubmit $rlRun_LOG 'ausearch.log'
 

+         rlPhaseEnd
 

+     fi
 

+ rlJournalEnd

file removed
-50 
@@ -1,50 +0,0 @@ 
 

- ---
 

- - hosts: localhost
 

-   vars:
 

-   - artifacts: "{{ lookup('env', 'TEST_ARTIFACTS')|default('./artifacts', true) }}"
 

-   tags:
 

-   - classic
 

-   tasks:
 

-   # switch SELinux to permissive mode
 

-   - name: Get default kernel
 

-     command: "grubby --default-kernel"
 

-     register: default_kernel
 

-   - debug: msg="{{ default_kernel.stdout }}"
 

-   - name: Set permissive mode
 

-     command: "grubby --args=enforcing=0 --update-kernel {{ default_kernel.stdout }}"
 

- 

-   - name: reboot
 

-     block:
 

-       - name: restart host
 

-         shell: sleep 2 && shutdown -r now "Ansible updates triggered"
 

-         async: 1
 

-         poll: 0
 

-         ignore_errors: true
 

- 

-       - name: wait for host to come back
 

-         wait_for_connection:
 

-           delay: 10
 

-           timeout: 300
 

- 

-       - name: Re-create /tmp/artifacts
 

-         command: mkdir /tmp/artifacts
 

- 

-       - name: Gather SELinux denials since boot
 

-         shell: |
 

-             result=pass
 

-             dmesg | grep -i -e type=1300 -e type=1400 > /tmp/avc.log && result=fail
 

-             ausearch -m avc -m selinux_err -m user_avc -ts boot &>> /tmp/avc.log
 

-             grep -q '<no matches>' /tmp/avc.log || result=fail
 

-             echo -e "\nresults:\n- test: reboot and collect AVC\n  result: $result\n  logs:\n  - avc.log\n\n" > /tmp/results.yml
 

-             ( [ $result = "pass" ] && echo PASS test-reboot || echo FAIL test-reboot ) > /tmp/test.log
 

- 

-     always:
 

-       - name: Pull out the artifacts
 

-         fetch:
 

-           dest: "{{ artifacts }}/"
 

-           src: "{{ item }}"
 

-           flat: yes
 

-         with_items:
 

-           - /tmp/test.log
 

-           - /tmp/avc.log
 

-           - /tmp/results.yml

file removed
-10 
@@ -1,10 +0,0 @@ 
 

- ---
 

- - hosts: localhost
 

-   roles:
 

-   - role: standard-test-beakerlib
 

-     tags:
 

-     - classic
 

-     repositories:
 

-     - repo: "https://src.fedoraproject.org/tests/selinux.git"
 

-       dest: "selinux"
 

-       fmf_filter: "tier:1 | component:selinux-policy"

This PR just shows way how the STI test with reboot can be rewritten to tmt.

Do not merge! It can spoil testing.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

In order to enable tmt tests a simple plan is needed:

tmt plan create --template base /plans/base

Current test results are from the STI config.

1 new commit added

  • tmt: adding test plan
2 years ago

2 new commits added

  • tmt: adding test plan
  • Providing tmt version of the reboot test
2 years ago

2 new commits added

  • tmt: adding test plan
  • Providing tmt version of the reboot test
2 years ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

Hm, CI pipeline still picks STI tests. Seems the only way to test this is to remove STI playbooks in the pull request.

1 new commit added

  • tmt: Removing previous STI tests
2 years ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

1 new commit added

  • tmt: fixing paths
2 years ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

The reboot was executed as expected but the run finished because of:

err: Connection to 172.31.29.128 closed by remote host.
Command returned '255'.

We've tested provision -h connect with @fnecas and reboot was working nicely with the connect provision method. @mvadkert, do you see why the same would fail when run in testing farm?

@psss hmm, do I see it right that for 

15:07:39             Run command 'ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oIdentitiesOnly=yes -i /etc/citool.d/id_rsa_artemis root@172.31.29.128 reboot'.

are you expecting 0 return code?

15:07:39             err: Warning: Permanently added '172.31.29.128' (ECDSA) to the list of known hosts.
15:07:39             err: Connection to 172.31.29.128 closed by remote host.
15:07:39             Command returned '255'.

Looks to me the machine went down faster than the reboot command completed? I would say we should not care about return code here ...

Looks to me the machine went down faster than the reboot command completed?

Seems so...

I would say we should not care about return code here ...

Agreed, pull request ready for review.

@mvadkert, the fix has been merged. Do you think you could try the latest tmt from copr to verify the reboot would work in the testing farm?

@psss I will try it via my localhost ...

recheck

(need a new rpm here)

citest

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

@psss tested with tmt-1.8.0-1.20211122162916456807.pr950.22.g85f2a77.el8.noarch

[12:15:30] [+] [worker_0] [stdout] [/plans/base] /var/ARTIFACTS/work-baseIrr5CT
[12:15:30] [+] [worker_0] [stdout] [/plans/base] Found 1 plan.
[12:15:30] [+] [worker_0] [stdout] [/plans/base] 
[12:15:30] [+] [worker_0] [stdout] [/plans/base] /plans/base
[12:15:31] [+] [worker_0] [stdout] [/plans/base] summary: tmt selinux-policy tests
[12:15:31] [+] [worker_0] [stdout] [/plans/base]     discover
[12:15:31] [+] [worker_0] [stdout] [/plans/base]         how: fmf
[12:15:31] [+] [worker_0] [stdout] [/plans/base]         directory: /var/ARTIFACTS/workdir-repository-None-DQtw0i
[12:15:31] [+] [worker_0] [stdout] [/plans/base]         hash: 5856981d9
[12:15:32] [+] [worker_0] [stdout] [/plans/base]         summary: 1 test selected
[12:15:32] [+] [worker_0] [stdout] [/plans/base]             /tests/reboot
[12:15:32] [+] [worker_0] [stdout] [/plans/base]     provision
[12:15:32] [+] [worker_0] [stdout] [/plans/base]         how: connect
[12:15:32] [+] [worker_0] [stdout] [/plans/base]         guest: 172.31.25.19
[12:15:32] [+] [worker_0] [stdout] [/plans/base]         user: root
[12:15:33] [+] [worker_0] [stdout] [/plans/base]         key: /etc/citool.d/id_rsa_artemis
[12:15:33] [+] [worker_0] [stdout] [/plans/base]         distro: Fedora Linux 36 (Cloud Edition Prerelease)
[12:15:33] [+] [worker_0] [stdout] [/plans/base]         kernel: 5.16.0-0.rc0.20211112git5833291ab6de.12.fc36.x86_64
[12:15:34] [+] [worker_0] [stdout] [/plans/base]         summary: 1 guest provisioned
[12:15:35] [+] [worker_0] [stdout] [/plans/base]     prepare
[12:15:36] [+] [worker_0] [stdout] [/plans/base]         how: install
[12:15:36] [+] [worker_0] [stdout] [/plans/base]         summary: Install required packages
[12:15:37] [+] [worker_0] [stdout] [/plans/base]         name: requires
[12:15:37] [+] [worker_0] [stdout] [/plans/base]         package: 1 package requested
[12:15:57] [+] [worker_0] [stdout] [/plans/base]             beakerlib >= 1.28
[12:15:57] [+] [worker_0] [stdout] [/plans/base]         summary: 1 preparation applied
[12:15:58] [+] [worker_0] [stdout] [/plans/base]     execute
[12:15:58] [+] [worker_0] [stdout] [/plans/base]         how: tmt
[12:16:05] [+] [worker_0] [stdout] [/plans/base]         exit-first: False
[12:16:45] [+] [worker_0] [stdout] [/plans/base]             00:00:01 /tests/reboot [1/1]
[12:16:47] [+] [worker_0] [stdout] [/plans/base]             00:00:01 /tests/reboot [1/1]
[12:16:47] [+] [worker_0] [stdout] [/plans/base]         summary: 1 test executed
[12:16:47] [+] [worker_0] [stdout] [/plans/base]     report
[12:16:47] [+] [worker_0] [stdout] [/plans/base]         how: display
[12:16:47] [+] [worker_0] [stdout] [/plans/base]             fail /tests/reboot
[12:16:47] [+] [worker_0] [stdout] [/plans/base]         summary: 1 test failed
[12:16:47] [+] [worker_0] [stdout] [/plans/base]     finish
[12:16:47] [+] [worker_0] [stdout] [/plans/base]         summary: 0 tasks completed
[12:16:47] [+] [worker_0] [stdout] [/plans/base] 
[12:16:47] [+] [worker_0] [stdout] [/plans/base] total: 1 test failed

beakerlib log:

http://pastebin.test.redhat.com/1010320

can you check if it expected now?

can you check if it expected now?

Yes, thanks, now looks much better. @mmalik, @omos, could you please have a look at the test failure? Is that a real test fail or something we should still look into? Thanks.
Metadata
None
No Tags
Changes Summary 6
+1
file added
.fmf/version
+5
file added
plans/base.fmf
+6
file added
tests/reboot/main.fmf
+25
file added
tests/reboot/test.sh
-50
file removed
tests/tests-reboot.yml
-10
file removed
tests/tests.yml
Powered by Pagure 5.13.3
Documentation • File an Issue • About this Instance • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.