This repository contains tests for llvm.
Tests are executed using TMT (Test Management Tool). The repository includes a TMT plan that runs all the tests by default. All the tests support the following context dimensions:
distro
arch
collection
(only applicable to RHEL 7)To run the tests:
# Run in the default virtual environment
tmt -c distro=<distro> -c arch=<arch> run -avv
# Run on a scheduled system
tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname>
# Run specific tests
tmt -c distro=<distro> -c arch=<arch> run -avv provision -h connect -g <ip_or_hostname> test -n <regex>
Please consider the following when adding new tests:
test.sh
. If a different test is needed, you need
to overwrite the test:
key in the test main.fmf
.tests/<test_name>
. Additionally they can be
placed into subdirectories if grouping make sense. E.g.
tests/regression/<test_name>
.tests/main.fmf
, inherited
by all tests. When adding new ones, be careful not to overwrite existing keys
from the inherited tests/main.fmf
.require+:
instead require:
. For advanced use and more examples check
FMF documentation.After creating the new test, consider running tmt <context> tests show
<test-name>
to review that the resulting metadata such as requires, enable or
test keys are as expected. For example:
$ tmt -c distro=fedora-38 -c arch=aarch64 tests show binutils-plugin-ar
/tests/binutils-plugin-ar
summary Make sure the the LLVM plugin for binutils' ar works
description Test that ar picks the correct plugin when archiving
LLVM bitcode.
contact Jesus Checa Hidalgo <jcheca@redhat.com>
component 'llvm-toolset' and 'llvm'
test ./test.sh
path /tests/binutils-plugin-ar
framework shell
manual false
tty false
require binutils
llvm
llvm-libs
llvm-devel
clang
duration 1m
enabled true
result respect
tier 1
Use the following templates when creating new tests.
FMF template. Uncomment needed fields, remove the rest.
summary: Descriptive summary for the test
# Maximum duration for the test
duration: 2m
# Short tests should be tier <= 1 so they're used for gating.
tier: 1
# Add an additional contact(s) if applicable
# contact+:
# - Somebody <somebody@redhat.com>
# Additional requires
# require+:
# - rpm-build
# Adjustments based on context dimensions
# adjust+:
# - when: collection is defined
# enabled: false
# because: Test not supported in collections
# link+:
# - verifies: https://issues.redhat.com/browse/...
# tag+:
# - sometag
This project uses pre-commit
Please install pre-commit
using pip install pre-commit
as described here. Then navigate to your clone of this project and install the git hook scripts using pre-commit install
. This will run pre-commit on every git commit
that you make in this repository from now on.