CI, Docker Images, Docs & Tests

This section contains an overview of the steps run on CI, as well as the tools used to simplify the testing (such as running Vulkan on CPU).

We use Github Actions to run the tests, which simplifies the workflows significantly for contributors.

The tests run on CPU, and can be triggered using the ACT command line interface (https://github.com/nektos/act) - once you install the command line (And start the Docker daemon) you just have to type:

$ act

[Python Tests/python-tests] 🚀  Start image=axsauze/kompute-builder:0.2
[C++ Tests/cpp-tests      ] 🚀  Start image=axsauze/kompute-builder:0.2
[C++ Tests/cpp-tests      ]   🐳  docker run image=axsauze/kompute-builder:0.2 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Python Tests/python-tests]   🐳  docker run image=axsauze/kompute-builder:0.2 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
...

CI Commands Triggered

The simplest way to see how this works is by looking at the github actions commands that are run.

These can be found through the following files:

When submitting a PR or merging a PR into master, both of these will run - you can see the logs through the github interface.

Running Vulkan on the CPU

We use Swiftshader to enable us to run the Vulkan Kompute framework directly on the CPU for the CI tests.

Even though Swiftshader is optimized to function as a high-performance CPU backend for Vulkan, there are several limitations, the most notable in context of Kompute are:

  • Loading files (spirv or text) leads to segfault

  • Loading raw text string shaders leads to segfault

This is one of the main reason why only a subset of the tests are run in the CI.

Dockerfiles

The dockerfiles created provide functionality to simplify the interaction with the system.

Image

Description

axsauze/kompute-builder:0.2

Main CI builder image with all required dependencies to build and run C++ & Python tests.

axsauze/swiftshader:0.1

Image building Swiftshader libraries only to reduce time via multi-staged builds

axsauze/vulkan-sdk:0.1

Image contained a linux build of the full Vulkan SDK to reduce time via multi-staged builds

Running / Building Documentation

In order to build the documentation you will need the following dependencies:

  • Install CI dependencies under scripts/requirements.txt

Once this installed:

  • You can build the documentation using the gendocsall cmake target

  • You can serve the documentation locally using the mk_run_docs command in the Makefile

Performing Release

In order to perform the release the following steps need to be carried out:

  • Build changelog
    • Generate latest changelog make build_changelog

    • Update latest tag in new CHANGELOG.md to be the vesion to release

  • Python Release
    • Build dependency:
      • Intsall dependency: pip install .

      • Ensure all tests pass in GPU and CPU: python -m pytest

      • Build distribution python setup.py sdist bdist_wheel

    • Test repo:
      • Push to test repo python -m twine upload –repository testpypi dist/*

      • Install python dependency: python -m pip install –index-url https://test.pypi.org/simple/ –no-deps kp

      • Ensure all tests pass in GPU and CPU: python -m pytest

    • Prod repo:
      • Push to test repo python -m twine upload dist/*

      • Install package from prod pypi pip install kp

      • Ensure all tests pass in GPU and CPU: python -m pytest

```