Projects¶

A project is an abstraction of a software project containing information on how to build and execute that software project. The tool suite already ships with some projects, but also allows you to add new ones.

How to add a new project to VaRA-TS
List of supported projects

How to add a new project to VaRA-TS¶

To add a new project, create a new class with VProject as its superclass. Place the class in one of the varats/projects/<group> folders, where <group> refers to the language of the project.

Additionally, add the project to the bb_config.py file in varats/tools.

Below, you can see a simplified version of the project Gravity with comments explaining the different parts of the project definition. You can use this as a template for your own project:

import typing as tp

import benchbuild as bb
from benchbuild.utils.cmd import make
from benchbuild.utils.settings import get_number_of_jobs
from plumbum import local

from varats.paper.paper_config import PaperConfigSpecificGit
from varats.project.project_domain import ProjectDomains
from varats.project.project_util import (
    ProjectBinaryWrapper,
    get_local_project_git_path,
    BinaryType,
    verify_binaries,
)
from varats.project.varats_project import VProject
from varats.utils.git_util import ShortCommitHash, RevisionBinaryMap
from varats.utils.settings import bb_cfg


class Gravity(VProject):
    """Programming language Gravity."""

    NAME = 'gravity'                   # The name of the project
    GROUP = 'c_projects'               # The group this project belongs to
    DOMAIN = ProjectDomains.PROG_LANG  # The application domain of this project

    SOURCE = [
        PaperConfigSpecificGit(
            project_name="gravity",
            remote="https://github.com/marcobambini/gravity.git",
            local="gravity",
            refspec="origin/HEAD",
            limit=None,
            shallow=False
        )
    ]

    @staticmethod
    def binaries_for_revision(
        revision: ShortCommitHash
    ) -> tp.List[ProjectBinaryWrapper]:
        binary_map = RevisionBinaryMap(get_local_project_git_path(Gravity.NAME))

        binary_map.specify_binary("gravity", BinaryType.EXECUTABLE)

        return binary_map[revision]

    def run_tests(self) -> None:
        """ This function defines tests and benchmarks for the project. """
        pass

    def compile(self) -> None:
        """ Contains instructions on how to build the project. """
        gravity_version_source = local.path(self.source_of_primary)
        clang = bb.compiler.cc(self)
        with local.cwd(gravity_version_source):
            with local.env(CC=str(clang)):
                bb.watch(make)("-j", get_number_of_jobs(bb_cfg()))

            verify_binaries(self)

Note

Some projects change their build system at some point in time. In such cases, you could check which version you are currently compiling and dispatch the build process to a more specific build function. For an example, have a look at the real Gravity project.

Specifying Project Binaries¶

The function binaries_for_revision() of a Project exports a list of binaries built by the project for a given revision, specifying the relative path from the project root to the created binary. When queried by an analysis, a list of ProjectBinaryWrappers is returned, where every wrapper specifies the name, location of the binary, and the BinaryType. To guarantee that all specified binaries were produced during compilation, one should add a verify_binaries() call after the compilation is done. In our example, the verify_binaries() call is directly after the call to make and still located within the gravity_version_source context, meaning the execution is still in the projects source folder.

Binary wrappers can be specified using the RevisionBinaryMap and its function specify_binary(). Since for some projects binary locations change during project life time, the RevisionBinaryMap allows to specify binary locations for specific ranges of project revisions using benchbuild.utils.revision_ranges.

Blocking revisions¶

It will happen that some revisions of a project fail to build because of missing dependencies or because of a bug in the project sources. The tool suite provides a facility to mark such revisions and block them from further experiments.

This can be done with benchbuild’s block_revisions source declaration decorator. This decorator allows you to block single revisions or larger ranges of revisions.

To block revisions, just add the block_revisions decorator around a git source declaration:

from benchbuild.utils.revision_ranges import (
    block_revisions,
    GoodBadSubgraph,
    RevisionRange,
    SingleRevision,
)

...

SOURCE = [
    block_revisions([
        RevisionRange(
            "0b8e0e047fc3d5e18ead3221ad54920f1ad0eedc",
            "8f417752dd14deea64249b5d32b6138ebc877fa9", "nothing to build"
        ),
        GoodBadSubgraph(["e8999a84efbd9c3e739bff7af39500d14e61bfbc"],
                        ["0e918ce0798407dd6c981e1cd26b4ba138d22fab"],
                        "missing -lm"),
        GoodBadSubgraph(["244c5aa91358a5b2472d351e6c7f38ba7da94ef6"],
                        ["371152de2f38534d4da332349d1def83fc66d5bc"],
                        "Visual studio project breaks makefile"),
        GoodBadSubgraph(["112be515b5ef3b67011c7272e5a50ac3a1fcadc4"],
                        ["b9a62dfad41ae06d029493cf4d5757de2a0281b2"],
                        "bug in gravity"),
        SingleRevision(
            "e207f0cc87bf57e9ccb6f0d18ff4fe4d6ef0c096", "bug in gravity"
        ),
        GoodBadSubgraph(["d2a04f92347fb5f2b6fd23bea9b0e12817cd6d8e"],
                        ["e8fbd6a4a2a9618456f1460dc9138b617dc7af4b"],
                        "bug in gravity"),
        GoodBadSubgraph(["968534c5d4f28501b7f34da48cab2c153ae7449b"],
                        ["0caf15328bda90ffb1911077e03b28ea9970208b"],
                        "bug in gravity"),
        GoodBadSubgraph(["e4f95e669a4c5cf2d142d5b0b72a11c117f7092f"],
                        ["09e59da4deff9b35224f4784fae9d0f132be9cea"],
                        "missing -lbsd"),
    ])(
        PaperConfigSpecificGit(
            "gravity",
            remote="https://github.com/marcobambini/gravity.git",
            local="gravity",
            refspec="origin/HEAD",
            limit=None,
            shallow=False
        )
    )
]

The decorator takes a list of revision ranges as argument. There are three different kinds of revision ranges:

SingleRevision: a single revision
RevisionRange: all revisions that lie between the two given revisions start and end, i.e., all revisions in the output of git log --pretty=%H --ancestry-path start..end
GoodBadSubgraph: blocks all revisions that contain a bad commit but no good commit

Each revision range can be annotated with a comment to keep track or the reason why the revisions were blocked. This helps not only when reasoning about the validity of experiments, but it also makes it easier to see whether a block can be fixed in the future (e.g., missing dependencies could be resolved by a containerized execution environment).

Using Containers¶

To use a project with our container support it has to declare which container image to use. This can be done by assigning a container image to the CONTAINER class variable:

CONTAINER = get_base_image(ImageBase.DEBIAN_10)

To ensure that the container already contains everything required by BenchBuild, you should use one of our base images using the function get_base_image(). You can then add additional layers to the image using BenchBuild’s buildah API. For example, you can create a container based on our Debian 10 base image and install wget like this:

get_base_image(ImageBase.DEBIAN_10).run("apt", "install", "-y", "wget")

For some projects, you may want to use different container images based on the project revision. This can be achieved by assigning CONTAINER a list of (RevisionRange, ContainerImage) tuples.

List of supported projects¶

Project	Group	Domain	FeatureModel	Main Source
`Open62541`	c_projects	Architecture		https://github.com/open62541/open62541.git
`Capstone`	c_projects	Binary Analysis Framework		https://github.com/capstone-engine/capstone.git
`Glibc`	c_projects	C Library		git://sourceware.org/git/glibc.git
`OpenCV`	c_projects	C++ Library		https://github.com/opencv/opencv.git
`Bitlbee`	c_projects	Chat client		https://github.com/bitlbee/bitlbee.git
`Irssi`	c_projects	Chat client		https://github.com/irssi/irssi.git
`Libvpx`	c_projects	Codec		https://github.com/webmproject/libvpx.git
`Opus`	c_projects	Codec	Model	https://github.com/xiph/opus.git
`X264`	c_projects	Codec	Model	https://code.videolan.org/videolan/x264.git
`Brotli`	c_projects	Compression	Model	https://github.com/google/brotli.git
`Bzip2`	c_projects	Compression	Model	https://github.com/libarchive/bzip2.git
`Gzip`	c_projects	Compression	Model	https://github.com/vulder/gzip.git
`Lrzip`	c_projects	Compression	Model	https://github.com/ckolivas/lrzip.git
`Lz4`	c_projects	Compression		https://github.com/lz4/lz4.git
`Xz`	c_projects	Compression	Model	https://github.com/xz-mirror/xz.git
`Glib`	c_projects	Data structures		https://github.com/GNOME/glib.git
`Redis`	c_projects	Database		https://github.com/antirez/redis.git
`Vim`	c_projects	Editor		https://github.com/vim/vim.git
`LibjpegTurbo`	c_projects	File format		https://github.com/libjpeg-turbo/libjpeg-turbo
`Libpng`	c_projects	File format		https://github.com/glennrp/libpng.git
`Libtiff`	c_projects	File format		https://gitlab.com/libtiff/libtiff.git
`Libxml2`	c_projects	File format		https://github.com/GNOME/libxml2.git
`Qemu`	c_projects	Hardware emulator		https://github.com/qemu/qemu.git
`Yara`	c_projects	Malware Analysis		https://github.com/VirusTotal/yara.git
`Bison`	c_projects	Parser		https://github.com/bincrafters/bison.git
`Gravity`	c_projects	Programming language		https://github.com/marcobambini/gravity.git
`Libssh`	c_projects	Protocol		https://github.com/libssh/libssh-mirror.git
`OpenSSL`	c_projects	Security		https://github.com/openssl/openssl.git
`OpenVPN`	c_projects	Security	Model	https://github.com/openvpn/openvpn.git
`Asterisk`	c_projects	Signal processing		https://github.com/asterisk/asterisk.git
`Libsigrok`	c_projects	Signal processing		https://github.com/sigrokproject/libsigrok.git
`Hypre`	c_projects	Solver		https://github.com/hypre-space/hypre.git
`PicoSATLoadTime`	c_projects	Solver		https://github.com/se-sic/picoSAT-vara
`PicoSAT`	c_projects	Solver	Model	https://github.com/se-sic/picoSAT-mirror
`Busybox`	c_projects	UNIX utils		https://github.com/mirror/busybox.git
`Coreutils`	c_projects	UNIX utils		https://github.com/coreutils/coreutils.git
`File`	c_projects	UNIX utils		https://github.com/file/file
`Gawk`	c_projects	UNIX utils		https://github.com/vulder/gawk.git
`Grep`	c_projects	UNIX utils		https://github.com/vulder/grep.git
`Htop`	c_projects	UNIX utils		https://github.com/htop-dev/htop.git
`Tmux`	c_projects	UNIX utils		https://github.com/tmux/tmux.git
`Git`	c_projects	Version control		https://github.com/git/git.git
`Tig`	c_projects	Version control		https://github.com/jonas/tig.git
`Curl`	c_projects	Web tools	Model	https://github.com/curl/curl.git
`DunePerfRegression`	cpp_projects	C++ Library	Model	https://github.com/se-sic/dune-VaRA.git
`Fmt`	cpp_projects	C++ Library		https://github.com/fmtlib/fmt.git
`Libzmq`	cpp_projects	C++ Library		https://github.com/zeromq/libzmq.git
`Ect`	cpp_projects	Compression	Model	https://github.com/fhanau/Efficient-Compression-Tool.git
`Lepton`	cpp_projects	Compression	Model	https://github.com/dropbox/lepton
`MongoDB`	cpp_projects	Database	Model	https://github.com/mongodb/mongo.git
`Doxygen`	cpp_projects	Documentation		https://github.com/doxygen/doxygen.git
`HyTeg`	cpp_projects	High Performance Applications	Model	https://github.com/se-sic/hyteg-VaRA.git
`FastDownward`	cpp_projects	Planning	Model	https://github.com/aibasel/downward.git
`Poppler`	cpp_projects	Rendering		https://gitlab.freedesktop.org/poppler/poppler.git
`Clasp`	cpp_projects	Solver	Model	https://github.com/potassco/clasp.git
`Z3`	cpp_projects	Solver	Model	https://github.com/Z3Prover/z3.git
`TwoLibsOneProjectInteractionDiscreteLibsSingleProject`	cpp_projects	Test project		LibraryAnalysisRepos/TwoLibsOneProjectInteractionDiscreteLibsSingleProject/Elementalist
`LinkerCheck`	default_checks	Test project
`FeaturePerfCSCollection`	perf_tests	Test project	Model	https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthCTCRTP`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthCTPolicies`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthCTTemplateSpecialization`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthCTTraitBased`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthDADynamicDispatch`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthDARecursion`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthFeatureHigherOrderInteraction`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthFeatureInteraction`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthFeatureLargeConfigSpace`	perf_tests	Test project	Model	https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthFeatureRestrictedConfigSpace`	perf_tests	Test project	Model	https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthIPCombined`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthIPRuntime`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthIPTemplate2`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthIPTemplate`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthOVInsideLoop`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthSAContextSensitivity`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthSAFieldSensitivity`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthSAFlowSensitivity`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`SynthSAWholeProgram`	perf_tests	Test project		https://github.com/se-sic/FeaturePerfCSCollection.git
`FeaturePerfRegression`	perf_tests	Test project	Model	https://github.com/se-sic/FeaturePerfRegression.git
`BasicTests`	test_projects	Test project		https://github.com/se-passau/vara-perf-tests.git
`BasicBugDetectionTestRepo`	test_projects	Test project		BugDetectionRepos/BasicTestRepo
`ExampleTestRepo`	test_projects	Test project		BasicTestRepos/ExampleRepo
`CommitFeatureInteractionExample`	test_projects	Test project		FeatureAnalysisRepos/CommitFeatureInteractionExample
`FeatureTestRepo`	test_projects	Test project		FeatureAnalysisRepos/FeatureInteractionExample
`MultiAuthorCoordination`	test_projects	Test project		BlameAnalysisRepos/MutliMethodAuthorCoordination
`TaintTests`	test_projects	Test project		https://github.com/se-passau/vara-perf-tests.git
`SVFPointsToAnalysisBenchmark`	test_projects	Test project		https://github.com/SVF-tools/Test-Suite.git

C - Project Details¶

class varats.projects.c_projects.bzip2.Bzip2(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Compression and decompression tool bzip2 (fetched by Git)

class varats.projects.c_projects.gzip.Gzip(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Compression and decompression tool Gzip (fetched by Git)

class varats.projects.c_projects.x264.X264(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Video encoder x264 (fetched by Git)

class varats.projects.c_projects.xz.Xz(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Compression and decompression tool xz (fetched by Git)

class varats.projects.c_projects.asterisk.Asterisk(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Open Source PBX and telephony toolkit.

class varats.projects.c_projects.bison.Bison(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

GNU Bison parser generator.

(fetched by Git)

class varats.projects.c_projects.bitlbee.Bitlbee(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

BitlBee brings IM (instant messaging) to IRC clients.

(fetched by Git)

class varats.projects.c_projects.brotli.Brotli(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Brotli compression format.

class varats.projects.c_projects.busybox.Busybox(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: UNIX utility wrapper BusyBox.

class varats.projects.c_projects.capstone.Capstone(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Capstone is a disassembly framework with the target of becoming the ultimate disasm engine for binary analysis and reversing in the security community.

class varats.projects.c_projects.coreutils.Coreutils(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: GNU coretuils / UNIX command-line tools (fetched by Git)

class varats.projects.c_projects.curl.Curl(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Curl is a command-line tool for transferring data specified with URL syntax.

(fetched by Git)

class varats.projects.c_projects.file.File(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: File command for recognizing the type of data contained in a file.

class varats.projects.c_projects.gawk.Gawk(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

GNU awk.

(fetched by Git)

class varats.projects.c_projects.git.Git(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Git.

class varats.projects.c_projects.glib.Glib(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

GLib is the low-level core library that forms the basis for projects such as GTK and GNOME. It provides data structure handling for C, portability wrappers, and interfaces for such runtime functionality as an event loop, threads, dynamic loading, and an object system.

(fetched by Git)

class varats.projects.c_projects.glibc.Glibc(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Standard GNU C-library.

class varats.projects.c_projects.gravity.Gravity(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Programming language Gravity.

class varats.projects.c_projects.grep.Grep(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: GNU Grep / UNIX command-line tools (fetched by Git)

class varats.projects.c_projects.htop.Htop(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Process visualization tool (fetched by Git)

class varats.projects.c_projects.hypre.Hypre(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: HYPRE is a library of high performance preconditioners and solvers featuring multigrid methods for the solution of large, sparse linear systems of equations on massively parallel computers.

class varats.projects.c_projects.irssi.Irssi(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Irssi is a modular chat client that is most commonly known for its text mode user interface.

(fetched by Git)

class varats.projects.c_projects.libjpeg_turbo.LibjpegTurbo(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Libjpeg-turbo is a JPEG image codec.

class varats.projects.c_projects.libpng.Libpng(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Picture library.

(fetched by Git)

class varats.projects.c_projects.libsigrok.Libsigrok(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

The sigrok project aims at creating a portable, cross-platform, Free/Libre/Open-Source signal analysis software suite.

(fetched by Git)

class varats.projects.c_projects.libssh.Libssh(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

SSH library.

(fetched by Git)

class varats.projects.c_projects.libtiff.Libtiff(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Libtiff is a library for reading and writing Tagged Image File Format files.

class varats.projects.c_projects.libvpx.Libvpx(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Codec SDK libvpx (fetched by Git)

class varats.projects.c_projects.libxml2.Libxml2(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Libxml2 is a software library for parsing XML documents.

class varats.projects.c_projects.lrzip.Lrzip(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Compression and decompression tool lrzip (fetched by Git)

class varats.projects.c_projects.lz4.Lz4(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

LZ4 is lossless compression algorithm.

(fetched by Git)

class varats.projects.c_projects.open62541.Open62541(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: open62541 (http://open62541.org) is an open source implementation of OPC UA (OPC Unified Architecture / IEC 62541) written in the C language.

class varats.projects.c_projects.openssl.OpenSSL(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: TLS-framework OpenSSL (fetched by Git)

class varats.projects.c_projects.openvpn.OpenVPN(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

OpenVPN is open-source commercial software that implements virtual private network techniques to create secure point-to-point or site-to-site connections in routed or bridged configurations and remote access facilities.

(fetched by Git)

class varats.projects.c_projects.opus.Opus(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Opus is a codec for interactive speech and audio transmission over the Internet.

class varats.projects.c_projects.picosat.PicoSAT(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: PicoSAT is a SAT solver.

class varats.projects.c_projects.picosat.PicoSATLoadTime(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Adapted version of picoSAT that has been refactored, such that it does not require a field-sensitive analysis.

class varats.projects.c_projects.qemu.Qemu(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

QEMU, the FAST!

processor emulator.

class varats.projects.c_projects.redis.Redis(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Redis is an in-memory database that persists on disk.

(fetched by Git)

class varats.projects.c_projects.tig.Tig(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Tig: text-mode interface for Git

class varats.projects.c_projects.tmux.Tmux(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Terminal multiplexer Tmux.

class varats.projects.c_projects.vim.Vim(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Text processing tool vim.

class varats.projects.c_projects.yara.Yara(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

YARA is a tool aimed at (but not limited to) helping malware researchers to identify and classify malware samples.

With YARA you can create descriptions of malware families (or whatever you want to describe) based on textual or binary patterns.

class varats.projects.cpp_projects.opencv.OpenCV(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Open Source Computer Vision Library.

C++ - Project Details¶

class varats.projects.cpp_projects.clasp.Clasp(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Clasp is an answer set solver for (extended) normal and disjunctive logic programs.

class varats.projects.cpp_projects.doxygen.Doxygen(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Doxygen.

class varats.projects.cpp_projects.dune.DunePerfRegression(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Simulation framework for various applications in mathematics and physics.

Note

Currently Dune CANNOT be compiled with the Phasar passes activated in vara. Trying to do so will crash the compiler

If you use Dune with an experiment that uses the vara compiler, add -mllvm –vara-disable-phasar to the projects cflags to disable phasar passes.

This will still allow to analyse compile-time variability.

Might need deps:

klu
spqr
umfpack
eigen3

class varats.projects.cpp_projects.ect.Ect(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Efficient Compression Tool (or ECT) is a C++ file optimizer.

It supports PNG, JPEG, GZIP and ZIP files.

class varats.projects.cpp_projects.fast_downward.FastDownward(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Planning tool FastDownward (fetched by Git)

class varats.projects.cpp_projects.fmt.Fmt(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: A modern formatting library.

class varats.projects.cpp_projects.hyteg.HyTeg(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

C++ framework for large scale high performance finite element simulations based on (but not limited to) matrix-free geometric multigrid.

Notes

1. Currently, HyTeg CANNOT be compiled with the Phasar passes activated in vara. Trying to do so will crash the compiler

If you use Dune with an experiment that uses the vara compiler, add -mllvm –vara-disable-phasar to the projects cflags to disable phasar passes. This will still allow to analyse compile-time variability.

2. Due to the way that benchbuild generates the build folder names when running experiments in different configurations, HyTeg currently DOES NOT work out of the box when creating a case study with multiple configurations. This is due to benchbuild creating a temporary folder name with a comma in it to separate the revision and configuration id. This comma will be misinterpreted when the path for the eigen library is passed onto the linker.

There is a limited workaround for this: 1. Copy the eigen library revision that you want HyTeg to use to some other accessible location (That has no comma in its absolute path) 2. Set the environment variable EIGEN_PATH to point to the absolute path of that directory

This can be achieved by either EXPORT-ing it manually, adding it

to your .benchbuild.yml configuration or (when running with slurm) adding the export to your slurm scripts

class varats.projects.cpp_projects.lepton.Lepton(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Lepton is a tool and file format for losslessly compressing JPEGs by an average of 22%.

class varats.projects.cpp_projects.libzmq.Libzmq(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: The ZeroMQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products.

class varats.projects.cpp_projects.mongodb.MongoDB(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

MongoDB is a cross-platform document-oriented database program.

Classified as a NoSQL database program, MongoDB uses JSON-like documents with optional schemas.

class varats.projects.cpp_projects.poppler.Poppler(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Poppler is a free software utility library for rendering Portable Document Format documents.

class varats.projects.cpp_projects.two_libs_one_project_interaction_discrete_libs_single_project.TwoLibsOneProjectInteractionDiscreteLibsSingleProject(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Class to analyse interactions between two discrete libraries and one project.

class varats.projects.cpp_projects.z3.Z3(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Z3 is a theorem prover from Microsoft Research.

Test - Project Details¶

class varats.projects.test_projects.basic_tests.BasicTests(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Basic tests:

Different small test files

class varats.projects.test_projects.bug_provider_test_repos.BasicBugDetectionTestRepo(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Example project demonstrating how to use a repo from the vara-test- repos.

class varats.projects.test_projects.example_test_repo.ExampleTestRepo(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Example project demonstrating how to use a repo from the vara-test- repos.

class varats.projects.test_projects.feature_test_repo.FeatureTestRepo(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Example project for feature analyses.

class varats.projects.test_projects.feature_test_repo.CommitFeatureInteractionExample(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Example project for commit feature interactions.

class varats.projects.test_projects.multi_author_coordination.MultiAuthorCoordination(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶: Example project demonstrating a scenario where multiple authors interact with each other through file, function and data-flows.

class varats.projects.test_projects.taint_tests.TaintTests(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

Taint tests:

Different small test files for taint propagation

class varats.projects.test_projects.test_suite.SVFPointsToAnalysisBenchmark(revision=NOTHING, name=NOTHING, domain=NOTHING, group=NOTHING, container=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_uuid=NOTHING, builddir=NOTHING, source=NOTHING, workloads=NOTHING, primary_source=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶

SVFPointsToAnalysisBenchmark provides an easy way to execute the examples from the Test-Suite micro-benchmark suite for testing points to analyses.

Which can be found at https://github.com/SVF-tools/Test-Suite.