Evaluation Workspace

A workspace is the top-level container for all of your data, experiments, and artifacts. The CLI can only manipulate a limited subset of the workspace’s properties, more can be achieved by looking at its structure and files.

Structure

The workspace is a directory with the following structure:

  • sequences/ - a directory containing all the sequences in the workspace

  • results/ - a directory containing all the results in the workspace

  • cache/ - a directory containing all the cache in the workspace

  • analysis/ - a directory containing all the analysis in the workspace

Configuration

The workspace has a configuration file called config.yaml that contains all the configuration for the workspace. This file is usually automatically generated by the CLI, but can also be edited manually or even created from scratch.

Configuration reference (config.yaml)

The toolkit reads the following top-level keys from config.yaml:

  • stack (required): stack identifier (integrated stack name) or inline stack definition (see stack description).

  • registry (required): list of tracker registry locations. Paths can point to directories or directly to trackers.ini / trackers.yaml files.

  • sequences (optional): path to dataset directory, relative to workspace root by default. Default is sequences.

  • report (optional): report configuration (style, tracker sort order, and report index entries).

Minimal example

stack: tests/basic
registry:
        - ./trackers.ini

Typical custom-dataset example

stack: tests/basic
registry:
        - ./trackers.ini
        - ../shared/trackers
sequences: ./sequences

Notes

  • Registry and sequence paths are resolved relative to the workspace root.

  • The CLI –registry argument appends additional search paths on top of those defined in config.yaml.

  • If sequences is omitted, the toolkit uses <workspace>/sequences.

  • Report content can be left empty; in that case default report sections are generated.