Developing with VaRA-TS

First, take a look at our install guide to learn how to set up the tool suite.

The VaRA-TS API Reference contains information about how to work with the tool-suite, as well as how to add your own research tools, Experiments, Reports, Plots, and more.

For further information about benchbuild related concepts, like Experiments or Projects, take a look at the benchbuild documentation.

Testing

Tests for the VaRA tool suite are located in the tests directory. The tests are run using pytest but most of the tests are written using unittest syntax.

Mocking the vara configuration

To ensure a flawless test execution, it is important that the order of test execution does not matter, i.e. each single test is idempotent and does not modify the environment. One common problem with this are the vara and benchbuild configurations, as they are stored in a single global object. We therefore provide a helper function replace_config() to set a test-specific vara (and if needed benchbuild) configuration that can be safely modified without affecting other tests. This function can be either used as a decorator similar to unittest.mock or as a context manager.

Test resources

Test resources, like report files or case studies, can be stored in the tests/TEST_INPUTS directory. This directory follows the same structure one would also use for the tool-suites installation environment. The path to this directory can be accessed via the TEST_INPUTS_DIR attribute. If using the replace_config() wrapper or context manager without providing an own configuration, the relevant paths in the replaced configuration object are already pointed to this environment. Keep the data put in this directory as small as possible to avoid bloating the repository.

Warning

Make sure that your tests do not write to this directory to maintain an idempotent test environment. If you really need to write files, use temporary directories instead.

---

Module: test_utils

Module for test utility functions.

tests.test_utils.get_test_config(tmp_path)

Get a vara config suitable for testing.

Configs returned by this function are meant to be passed to replace_config().

Parameters

tmp_path (Path) – path to put files that may be written during test execution

Return type

Configuration

Returns

a (deep)copy of the current vara config suitable for testing

tests.test_utils.get_bb_test_config()

Return type

Configuration

tests.test_utils.replace_config(replace_bb_config=False, tmp_path=None, vara_config=None, bb_config=None)

Replace the vara and benchbuild config while executing a (test) function.

This function can be used as a decorator or a context manager. It replaces the current vara or benchbuild configuration with a test config or a given config.

If used as a decorator, the replaced config is passed as an additional argument to the function. If used as a context manager, it binds the replaced config to the name given after the as.

Parameters

• replace_bb_config (bool) – whether to also replace the benchbuild config

• tmp_path (Optional[Path]) – path to put files that may be written during test execution. If absent, the wrapper will create a temporary directory that is deleted after restoring the config.

• vara_config (Optional[Configuration]) – if given, use this as the new vara config instead of a copy of the current one

• bb_config (Optional[Configuration]) – if given, use this as the new benchbuild config instead of a copy of the current one

Return type

Any

Returns

the wrapped function

class tests.test_utils.DummyGit(remote, local, clone=True, limit=10, refspec='HEAD', shallow=True, version_filter=<function Git.<lambda>>)

Bases: benchbuild.source.git.Git

A dummy git source that does nothing.

fetch()

Clone the repository, if needed.

This will create a git clone inside the global cache directory.

Parameters

version (Optional[str], optional) – [description]. Defaults to None.

Returns

[description]

Return type

str

version(target_dir, version='HEAD')

Create a new git worktree pointing to the requested version.

Parameters

• target_dir (str) – The filesystem path where the new worktree should live.

• version (str) – The desired version the new worktree needs to point to. Defaults to ‘HEAD’.

Returns

[description]

Return type

str

versions()

List all available versions of this source.

Returns

The list of all available versions.

Return type

List[str]