#1
Add CI tests using the standard test interface
Opened 5 years ago
by jkrysl.
Modified 5 years ago
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
 ignatenkobrain commented 5 years ago | ||
It is the repo for our test suite that contains the 'snapper_basic' test. | ||
|
||
|
||
why is it not packaged? | ||
It does not seem to be required, as there are tests that run tests from repo too, see for example https://src.fedoraproject.org/rpms/NetworkManager/blob/master/f/tests/tests.yml | ||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
since you execute python setup.py install, why are those not installed? | ||
All of them are on pip, but python3-augeas version there is too old for our test suite, so instead of installing augeas-libs and compiling it we install python3-augeas straight away. The other 3 are not really required, as python-stqe setup.py can install them correctly, but since we are already installing python3-augeas rpm and we are testing Fedora, it is better to test on the versions that are packaged in Fedora. | ||
|
||
|
||
|
||
|
||
Adds tests according to the CI wiki [0] specifically the standard test interface in the spec [1].
The playbook includes Tier1 level test cases that have been tested in the following contexts and is passing reliably: Classic.
Test logs are stored in the Artifacts directory.
The following steps are used to execute the tests using the standard test interface:
- Classic
sudo ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory) TEST_SUBJECTS="" TEST_ARTIFACTS=$PWD/artifacts ansible-playbook --tags classic tests.yml[0] https://fedoraproject.org/wiki/CI
[1] https://fedoraproject.org/wiki/Changes/InvokingTests
Test Logs: (If you you would like a pointer to the complete log, I can include that as well)
All tests were executed on Fedora Rawhide
- Classic
<snip>
TASK [standard-test-basic : Execute tests] *************
changed: [localhost] => (item={'install_stqe': {'dir': 'python-stqe', 'run': 'python3 setup.py install'}})
changed: [localhost] => (item={'snapper_basic': {'dir': './', 'run': 'stqe-test run -t lvm/snapper/snapper_basic.py'}})
TASK [standard-test-basic : Check the results] *************
changed: [localhost]
TASK [standard-test-basic : Set role result] *************
ok: [localhost]
TASK [include_role : str-common-final] *************
TASK [str-common-final : Pull out the logs from test environment to test runner] *************
changed: [localhost]
TASK [str-common-final : Report role result] *************
ok: [localhost] => {
"msg": [
"Tests failed: False",
"Tests msg: ",
""
]
}
PLAY RECAP *************
localhost : ok=20 changed=9 unreachable=0 failed=0 $ cat artifacts/test.log
PASS install_stqe
PASS snapper_basic
and this is?