This repository contains tests for clang.
Tests are executed using TMT (Test Management Tool). The repository includes a TMT plan that runs all the tests by default. All the tests support the following context dimensions:
distro
arch
collection
(only applicable to RHEL 7)To run the tests:
# Run in the default virtual environment
tmt -c distro=<distro> -c arch=<arch> run -avv
# Run on a scheduled system
tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname>
# Run specific tests
tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname> test -n <regex>
Please consider the following when adding new tests:
runtest.sh
. If a different test is needed, you need
to overwrite the test:
key in the test main.fmf
.tests/<test_name>
. Additionally they can be
placed into subdirectories if grouping make sense. E.g.
tests/regression/<test_name>
.tests/main.fmf
, inherited
by all tests. When adding new ones, be careful not to overwrite existing keys
from the inherited tests/main.fmf
.require+:
instead require:
. For advanced use and more examples check
FMF documentation.After creating the new test, consider running tmt <context> tests show
<test-name>
to review that the resulting metadata such as requires, enable or
test keys are as expected. For example:
$ tmt -c distro=fedora-38 tests show ucrt64-toolchain
/tests/ucrt64-toolchain
summary Test that ucrt64 toolchain is detected
contact Jesus Checa Hidalgo <jcheca@redhat.com>
component 'llvm-toolset' and 'clang'
test ./runtest.sh
path /tests/ucrt64-toolchain
framework shell
manual false
tty false
require 'ucrt64-gcc-c++' and 'clang'
duration 5m
enabled true
result respect
$ tmt -c distro=rhel-9.3 tests show ucrt64-toolchain
/tests/ucrt64-toolchain
summary Test that ucrt64 toolchain is detected
contact Jesus Checa Hidalgo <jcheca@redhat.com>
component 'llvm-toolset' and 'clang'
test ./runtest.sh
path /tests/ucrt64-toolchain
framework shell
manual false
tty false
require 'ucrt64-gcc-c++' and 'clang'
duration 5m
enabled false
result respect
Use the following templates when creating new tests.
FMF template. Uncomment needed fields, remove the rest.
summary: Descriptive summary for the test
# Maximum duration for the test
duration: 2m
# Short tests should be tier <= 1 so they're used for gating.
tier: 1
# Add an additional contact(s) if applicable
# contact+:
# - Somebody <somebody@redhat.com>
# Additional requires
# require+:
# - rpm-build
# Adjustments based on context dimensions
# adjust+:
# - when: collection is defined
# enabled: false
# because: Test not supported in collections
# link+:
# - verifies: https://issues.redhat.com/browse/...
# tag+:
# - sometag