#1 Add tests to gc
Opened 2 years ago by pingou. Modified 2 years ago
rpms/ pingou/gc container  into  master

@@ -0,0 +1,10 @@ 

+ FROM registry.fedoraproject.org/fedora:FEDORA_VERSION

+ MAINTAINER Package maintainer <gc-owner@fedoraproject.org>

+ 

+ VOLUME ["/code"]

+ 

+ RUN dnf install -y gcc gc-devel

+ 

+ WORKDIR /code

+ 

+ ENTRYPOINT ["gcc", "-lgc", "tst-gc.c", "-o", "/code/tst-gc"]

file added
+23

@@ -0,0 +1,23 @@ 

+ /* Example program snagged from gc's homepage:

+ http://www.hboehm.info/gc/simple_example.html  */

+ 

+ #include "gc.h"

+ #include <assert.h>

+ #include <stdio.h>

+ 

+ int main()

+ {

+   int i;

+ 

+   GC_INIT();    /* Optional on Linux/X86; see below.  */

+   for (i = 0; i < 10000000; ++i)

+    {

+      int **p = (int **) GC_MALLOC(sizeof(int *));

+      int *q = (int *) GC_MALLOC_ATOMIC(sizeof(int));

+      assert(*p == 0);

+      *p = (int *) GC_REALLOC(q, 2 * sizeof(int));

+      if (i % 100000 == 0)

+        printf("Heap size = %d\n", GC_get_heap_size());

+    }

+   return 0;

+ }

file added
+132

@@ -0,0 +1,132 @@ 

+ # Tests for gc following the standard test interface defined at:

+ #    https://fedoraproject.org/wiki/CI

+ 

+ ---

+ - hosts: localhost

+   vars:

+     artifacts: "{{ lookup('env', 'TEST_ARTIFACTS')|default('./artifacts', true) }}"

+ 

+   tasks:

+   - name: Create /tmp/artifacts

+     file: path={{ item }} state=directory

+             owner=root mode=755 recurse=yes

+     with_items:

+       - /tmp/artifacts

+     tags:

+     - always

+ 

+   - name: Create the {{ artifacts }} directory locally

+     local_action:

+       module: file

+       args: path="{{ artifacts }}"

+             state=directory

+             owner=root

+             mode=755

+             recurse=yes

+     tags:

+     - always

+ 

+   # Classic and container block, install and compile the test program

+   - block:

+ 

+     - name: Install the required packages

+       package:

+           name: "{{ item }}"

+           state: present

+       with_items:

+         - gcc

+         - gc-devel

+         - gc

+ 

+     - name: Install the test files

+       copy: src=tst-gc.c dest="/tmp/artifacts" mode=0755

+ 

+     - name: Compile the test file

+       shell: gcc -lgc tst-gc.c -o tst-gc

+       args:

+         chdir: /tmp/artifacts

+ 

+     tags:

+     - classic

+     - container

+ 

+   # Atomic specific block, build the test program in a container (hosted locally)

+   - block:

+     - name: Get the OS version running

+       shell: grep VERSION_ID /etc/os-release | sed -e 's/VERSION_ID=//'

+       register: os_rel

+ 

+     - name: Create /tmp/gc_test/ locally

+       local_action:

+         module: file

+         args:

+           path: /tmp/gc_test/

+           state: directory

+ 

+     - name: Copy the Dockerfile and the test-gc.c file

+       local_action: copy src={{ item }} dest="/tmp/gc_test/" mode=0644

+       with_items:

+       - Dockerfile

+       - tst-gc.c

+ 

+     - name: Update the Dockerfile to pull from fedora:{{ os_rel.stdout }}

+       local_action: shell sed -i -e 's/FEDORA_VERSION/{{ os_rel.stdout}}/' Dockerfile

+       args:

+         chdir: /tmp/gc_test

+ 

+     - name: Make sure docker is started locally

+       local_action:

+         module: service

+         args:

+           name: docker

+           state: started

+ 

+     - name: Build the container

+       local_action: shell docker build -f /tmp/gc_test/Dockerfile . -t dockahbuild

+       args:

+         chdir: /tmp/gc_test

+ 

+     - name: Run the container so we can get the binary

+       local_action: shell docker run -v /tmp/gc_test:/code:Z dockahbuild

+       args:

+         chdir: /tmp/gc_test

+ 

+     - name: Move the test files from the local host into the atomic host

+       synchronize:

+         src: /tmp/gc_test/

+         dest: /tmp/artifacts/

+         mode: push

+         ssh_args: "-o UserKnownHostsFile=/dev/null"

+ 

+     - name: Move the files into the artifacts directory for safe keeping

+       local_action:

+         module: copy

+         args:

+           src="/tmp/gc_test/"

+           dest="{{ artifacts }}"

+ 

+     tags:

+       - atomic

+ 

+   # Generic block, run the test program, check its output and sync the artifacts

+   - block:

+     - name: Run the test file

+       shell: ./tst-gc > out

+       args:

+         chdir: /tmp/artifacts

+ 

+     - name: Check/grep the output

+       shell: grep "Heap size = [1-9][0-9]*" out

+       args:

+         chdir: /tmp/artifacts

+ 

+     tags:

+       - always

+ 

+     always:

+     - name: Pull out the logs

+       synchronize:

+         src: "/tmp/artifacts/./"

+         dest: "{{ artifacts }}"

+         mode: pull

+ 

This pull-request adds tests to the gc package to be consumed and run by the
Fedora Atomic CI pipeline and according to the test specifications.

This is the first of two pull-requests, using two different approaches for the
same outcome.
The test itself consists of compiling a small C file, executing it and checking
its output.

For classic and container testing, this is straight forward, install the
dependencies, compile the file, execute it and grep its output.

For testing an Atomic Host image, this pull-request relies on using a docker
container to compile this C program (because the Atomic Host images used to not
have the gcc command available).
The ansible playbook makes sure to retrieve a docker container corresponding to
the Fedora version of the Atomic Host image being tested.

This has pros and cons:

pros:

  • tests are decoupled from the packaging, so they can easily be updated without
    having to touch the package itself

cons:

  • while we make sure to use a docker container of the same Fedora release as
    the Atomic Host image, there is a potential risk of compiling the C file with
    a different gcc version as the one that is in the image or the one that was
    used to compile gc itself.

The second pull-request that I will open takes a different approach, it embeds
the C file into the rpm, compiles it in the %build section and ships it in a -tests sub-package.
This makes the ansible playbook a little easier as testing gc just becomes:
install gc-tests, run the program, grep its output.
It also has the advantages that tests can then be linked to a specific version of
gc, but the test that is currently being added is quite generic.

However, it also has some cons:

  • Changing/Updating the tests means updating the package (adding more tests
    means adding more SourceX in the spec file)
  • More not-really-package-related-lines in the spec file (ie: less clean spec file)

Since you are the package maintainers and are meant to maintain the tests in the
long term, I am opening these two pull-requests so you can see which approach
you like better.

Note: you can see how to run the tests against an Atomic Host image in the doc

The test itself consists of compiling a small C file, executing it and checking
its output.

Than it is overly trivial testcase; please take the testsuite from the package itself (that one which is run during build-time). Also, that's why I like the second approach a bit more.

But, in the end -- both the approaches are ugly and I dislike them; that's because of badly designed pipeline (yeah, I know we discussed this early enough, and now is too late)
- the integration testing should live in separate repository, to have only one integration testsuite for all the branches (not to copy-paste it across all the branches)
- to have separate namespace for more QA oriented people
- to allow me to re-use the testing scripts in different packages
- to allow me to run 'gc' tests also for CI purposes in copr (say. praiskup/gc project in Fedora Cop)

So, +1 to the second proposal, but if and only if @pingou (or anybody else) will take care of it.

One more point:
- the integration testing should be re-usable by end-users; so if there's new gc update, everybody can go and re-run the tests themselves locally

The second proposal is better I agree. It also allows people to run the test binaries by just installing gc-tests, and run them against any compatible version of gc.

Secondly, Most of this code in this pull request is boilerplate. It needs to be moved into standard-test-roles. This pull request does that: https://pagure.io/standard-test-roles/pull-request/61

Once that's done this just becomes a few lines of code, and the Dockerfile itself can be dropped.

Well the second approach is in #2, not here :)

So, +1 to the second proposal, but if and only if @pingou (or anybody else) will take care of it.

As said above, I will not be maintaining the tests, the idea is that the tests are maintained by the packagers much like the spec file is. You're the most capable persons to know what changed and why (together with upstream) and there you're in the position to make the call about adding, adjusting or even removing a test.

As said above, I will not be maintaining the tests, the idea is that the tests are maintained by the packagers much like the spec file is.

In that case, I think we should move nearly all the boilerplate out of here. It should be as simple as possible.