PHP tests reports with GitLab CI

Published on 10/22/20 at 6:09 PM. Updated on 10/22/20 at 6:13 PM.

Testing is good, but it's even better with reports !

PHP tests reports with GitLab CI

Photo by Gabriel Sollmann on Unsplash

In a continuous integration pipeline we are at least executing test suites. But we can add other tools: static analysis, syntax checking, ...

If one of the job fails, the code can be considered unstable and we won't deploy or deliver if it was planned in the next stages.

Anyway, it could be useful to see what failed from the CI platform UI, without digging in each job's logs.
And that's what we're going to see with PHP tools on GitLab CI.

This post won't show how to setup those tools, only their integration with GitLab.

Artifacts and the JUnit format

Artifacts in GitLab CI

Artifacts are files or directories saved when a job finishes. They are then available for download in the GitLab UI.
We declare them in the artifacts section of each job.

We can use this feature to save any generated document or programs built during a job.

Please note that an artifact is automatically available for all jobs of the stages following the one it has been created in.
(see Using cache in GitLab CI with Docker-in-Docker in « The save / load alternative » section)

Test reports in JUnit format

The gitlab-ci.yml file has multiple sub-sections in artifacts.
A reports key allows to list test/quality reports, under which we can add a junit key for the reports in the JUnit format:

test_job:
    artifacts:
        reports:
            junit:
                - report.xml

Test reports are saved even if the job fails.

The JUnit reports are XML files with a list of the executed tests. Tests are then displayed in GitLab UI in the « Tests » tab of the pipelines pages.
JUnit comes from Java but it is commonly used, that's why we will export reports in this format.

PHPUnit

Executing our unit tests

When executing a test suite, we can generate a JUnit report using the --log-junit option followed by a filename:

phpunit:
    artifacts:
        reports:
            junit: phpunit-report.xml
    script:
        - ./vendor/bin/phpunit --log-junit phpunit-report.xml

If the tests are executed inside a Docker container, the report will be created inside it, we will need to export it:

phpunit:
    artifacts:
        reports:
            junit: phpunit-report.xml
    before_script:
        - *docker-login
        - docker run -d $DOCKER_CI_IMAGE
    script:
        - docker exec app ./vendor/bin/phpunit --log-junit phpunit-report.xml
    after_script:
        - docker cp app:/path/to/phpunit-report.xml .

To understand the docker-login anchor and the DOCKER_CI_IMAGE variable, see this post.

Saving a code coverage report

PHPUnit allow to generate a code coverage report (if you have Xdebug, PCOV or PHPDBG). GitLab is able to get this report if the project is configured for that.
Going into the CI/CD settings, in the « General Pipelines » section, we find a « Test coverage parsing » section with examples.
For PHPUnit, paste the following RegEx:

^\s*Lines:\s*\d+.\d+\%

In the jobs, we add options to generate a code coverage report in text format:

./vendor/bin/phpunit --coverage-text --colors=never

Combine them with the JUnit report options !

And if you like decorating your README with badges, the « Coverage report » section brings you some examples:

[![coverage report](https://gitlab.com/my-group/my-project/badges/master/coverage.svg)](https://gitlab.com/my-group/my-project/-/commits/master)

PHPStan

As for PHPUnit, we can generate a JUnit report with PHPStan by using the --error-format option:

phpstan:
    artifacts:
        reports:
            junit:
                - phpstan.xml
    script:
        - ./vendor/bin/phpstan analyse --error-format=junit --no-progress -c ./phpstan.neon > phpstan.xml

With PHPStan the command remains the same if we execute it from a Docker container.

PHP CodeSniffer

Here again, PHP_CodeSniffer allows generating a JUnit report in the same way we did it with PHPStan, with the --report option:

php_codesniffer:
    artifacts:
        reports:
            junit:
                - phpcs-report.xml
    script:
        - ./vendor/bin/phpcs --report=junit > phpcs-report.xml

Conclusion

I only talked about a few tools, and you might use many others (Psalm, Behat, PHPSpec, ...), check if they allow you to generate JUnit reports !

The testing stage in important in CI, but it's always better if your testing tools are well integrated in your CI platform.

Comments: 0

Robot invasion coming from robohash.org