#103 Cleanup of tests for easier future sync to downstream repos
Merged 2 years ago by tstellar. Opened 2 years ago by happz.

tests/tests-llvm.yml tests/tests.yml
file renamed
+26 -7
@@ -4,10 +4,22 @@ 

      tags:

        - classic

      required_packages:

+       # Required for rust and possibly other dependant packages

+       #

+       # NOTE #1: drop the requirement once the transitional period between releases is gone

+       # and llvm-compat is no longer a) needed, and b) part of the build.

+       #

+       # NOTE #2: keep it the first requirement. Depending on the order of requirements, placing it somewhere

+       # else in the list may lead to harness uninstalling and re-installing LLVM packages. That a) would be

+       # a waste of time, b) could potentially hide issues, leading to testing unexpected NVRs.

+       #

+       # - llvm-compat

+ 

        - rust

        - cargo

-       - binutils

+ 

        # the requirements below are for the integration suite

+       - binutils

        - cmake

        - llvm-devel

        - clang
@@ -15,15 +27,21 @@ 

        - clang-tools-extra

        - compiler-rt

        - ninja-build

-       - libcxx-devel

        - libomp-devel

-       - python-lit

        - lld

        - lldb

        - git

        - make

        - libstdc++-static

        - clang-devel

+ 

+       # In Fedora, python-lit is valid. In RHEL, however, the actual package is python3-lit.

+       - python-lit

+       # - python3-lit

+ 

+       # Required in Fedora, but not shipped with RHEL. For future reference, comment the libcxx out as needed.

+       - libcxx-devel

+ 

      tests:

        - rust-sanity:

            dir: ./
@@ -36,7 +54,7 @@ 

        # https://bugzilla.redhat.com/show_bug.cgi?id=1793250

        - libllvm-size:

            dir: ./

-           run: test `stat -L -c %s /usr/lib64/libLLVM.so` -lt 100000000

+           run: test `stat -L -c %s /usr/lib64/libLLVM.so` -lt 104857600

        # This test ensures that the spec file still builds correctly with

        # %global compat_build 1

        # FIXME: This fails, because the CI system has a hard-coded timeout of 4
@@ -47,8 +65,9 @@ 

        - llvm-config:

            dir: ./

            run: llvm-config --version

-       # make sure versionned llvm-config symlink is properly setup

-       - versionned-llvm-config:

+       # make sure versioned llvm-config symlink is properly setup

+       - versioned-llvm-config:

            dir: ./

-           run: llvm-config-12 --version

+           run: llvm-config-13 --version

+       # NOTE: don't forget to disable libcxx tests when libcxx is not available, by adding -DENABLE_LIBCXX=OFF.

        - integration-test-suite

  • tests.yml => tests-llvm.yml, to allow composition by "dumping" tests
    from several LLVM components when testing a module or collection;
  • more comments, to highlight part where downstream maintaners and QEs
    need to modify tests;
  • fixed failing tests to get a clean table as a verification tests do
    work after my changes.

looks like we're not packaging analyze-cc and cie @tstellar does that ring any bell? These binaries should be installed alongside scan-build-py

Build succeeded.

looks like we're not packaging analyze-cc and cie @tstellar does that ring any bell? These binaries should be installed alongside scan-build-py

I think analyze-cc may have been renamed.

Shall I disable the test temporarily? It's not really that important to me whether it reveals an actual packaging issue at this moment, fixing such an issue would be a matter for another patch anyway.

@sergesanspaille I just checked and we do package analyze-cc, but it's installed into %{_libexecdir} maybe the integration test suite needs to be fixed?

@happz Note we were planning to switch to tmt:
https://src.fedoraproject.org/rpms/llvm/pull-request/100

I wasn't aware of that pull request, but it's perfectly fine with me. However, at this very moment, I'd rather have working and passing STI tests here and in the downstream repositories, with downstream following Fedora as much as possible - this seems to be a lower hanging fruit, and it would greatly improve the state of gating in downstream. Right after that, I'd gladly take care of switching to TMT here and in downstream repositories. Would that be an acceptable approach?

@tstellar if we install anayzer-cc in a path when scan-build can't find it, then it's our fault. i'll propose a small patch.

(for the record, debian also apply a patch, I'm preparing an upstream fix)

rebased onto b21766e

2 years ago

Pull-Request has been merged by tstellar

2 years ago

Build succeeded.

Zuul does not pick up the tests anymore after this PR: https://fedora.softwarefactory-project.io/zuul/buildset/37d824016e5e4cc3a1d77550e3019898

I assume that's because of the tests.yml rename.

Zuul does not pick up the tests anymore after this PR: https://fedora.softwarefactory-project.io/zuul/buildset/37d824016e5e4cc3a1d77550e3019898

I assume that's because of the tests.yml rename.

AFAICT, it's a perfectly valid name. See https://docs.fedoraproject.org/en-US/ci/standard-test-interface/#_invocation:

The testing system MUST run each playbook matching the glob tests/tests*.yml in the dist-git repo. Each of these files constitutes a test suite.

This only seems to look for a literal tests.yml: https://pagure.io/fedora-zuul-jobs/blob/master/f/roles/run-tests-yml/tasks/main.yaml#_30

Indeed. I filed an issue, https://pagure.io/fedora-ci/general/issue/280. Now, what to do before we get a fix for this bug, would ln -s tests-llvm.yml tests.yml help? I'd say it should, and I'd like to keep tests-llvm.yml in play for easier composition of tests from multiple components.

It looks like this is only an issue with Zuul, because the tests still get run by the Fedora CI -dist-git tests. I think it's fine to wait for a fix in Zuul, because the Zuul tests are just informational. The dist-git tests are what is actually used for gating.

Metadata