Analysis module

The analysis module contains classes that implement various performance analysis methodologies. It also contains a parallel runtime with caching capabilities that enables efficient execution of large-scale evaluations.

This module contains classes and functions for analysis of tracker performance.

The analysis is performed on the results of an experiment.

class vot.analysis.Analysis(**kwargs)

Base class for all analysis classes.

Analysis is a class that descibes computation of one or more performance metrics for a given experiment.

property axes: Axes

Returns axes semantic description for the result grid.

property cached: bool

Returns whether the analysis should be cached.

commit(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence])

Commits the analysis for execution on default processor.

compatible(experiment: Experiment)

Checks if the analysis is compatible with the experiment type.

compute(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

Computes the analysis for the given experiment, trackers and sequences. The dependencies are the results of the dependnent analyses. The result is a grid with the results of the analysis. The grid is indexed by trackers and sequences. The axes are described by the axes() method.

Parameters

  • experiment (Experiment) – Experiment to compute the analysis for.

  • trackers (List[Tracker]) – List of trackers to compute the analysis for.

  • sequences (List[Sequence]) – List of sequences to compute the analysis for.

  • dependencies (List[Grid]) – List of dependencies of the analysis.

Returns: Grid with the results of the analysis.

dependencies() → List[Analysis]

Returns a list of dependencies of the analysis.

This is used to determine the order of execution of the analysis.

describe() → Tuple[Result]

Returns a tuple of descriptions of results of the analysis.

property identifier: str

Returns a unique identifier of the analysis.

This is used to determine if the analysis has been already computed.

run(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence])

Runs the analysis on default processor.

property title: str

Returns the title of the analysis.

If name is not set, then the default title is returned.

class vot.analysis.Axes(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Semantic information for axis in analysis grid.

class vot.analysis.Curve(name: str, dimensions: int, abbreviation: Optional[str] = None, minimal: Optional[Tuple[float]] = None, maximal: Optional[Tuple[float]] = None, labels: Optional[Tuple[str]] = None, trait: Optional[str] = None)

Curve is a list of 2+ dimensional results.

The number of elements in a list can vary between samples.

class vot.analysis.Drawable(name: str, abbreviation: Optional[str] = None, trait: Optional[str] = None)

Base class for results that can be visualized in plots.

property trait

Trait of the data, used for specification.

class vot.analysis.Label(*args, **kwargs)

Label describes a single categorical output of an analysis.

Can have a set of possible values.

class vot.analysis.Measure(name: str, abbreviation: Optional[str] = None, minimal: Optional[float] = None, maximal: Optional[float] = None, direction: Optional[Sorting] = Sorting.UNSORTABLE)

Measure describes a single value numerical output of an analysis.

Can have minimum and maximum value as well as direction of sorting.

property direction: Sorting

Direction of sorting.

If Sorting.UNSORTABLE, then the measure is not sortable.

property maximal: float

Maximal value of the measure.

If None, then the measure is not bounded from above.

property minimal: float

Minimal value of the measure.

If None, then the measure is not bounded from below.

exception vot.analysis.MissingResultsException(*args: object)

Exception class that denotes missing results during analysis.

class vot.analysis.Multidimensional(name: str, dimensions: int, abbreviation: Optional[str] = None, minimal: Optional[Tuple[float]] = None, maximal: Optional[Tuple[float]] = None, labels: Optional[Tuple[str]] = None, trait: Optional[str] = None)

Base class for multidimensional results.

This class is used to describe results that can be visualized in a scatter plot.

property dimensions

Number of dimensions of the result.

label(i)

Label for the i-th dimension.

maximal(i)

Maximal value of the i-th dimension.

If None, then the measure is not bounded from above.

minimal(i)

Minimal value of the i-th dimension.

If None, then the measure is not bounded from below.

class vot.analysis.Plot(name: str, abbreviation: Optional[str] = None, wrt: str = 'frames', minimal: Optional[float] = None, maximal: Optional[float] = None, trait: Optional[str] = None)

Plot describes a result in form of a list of values with optional minimum and maximum with respect to some unit.

The results of the same analysis for different trackers should have the same number of measurements (independent variable).

property maximal

Maximal value of the measure.

If None, then the measure is not bounded from above.

property minimal

Minimal value of the measure.

If None, then the measure is not bounded from below.

property wrt

Unit of the independent variable.

class vot.analysis.Point(name: str, dimensions: int, abbreviation: Optional[str] = None, minimal: Optional[Tuple[float]] = None, maximal: Optional[Tuple[float]] = None, labels: Optional[Tuple[str]] = None, trait: Optional[str] = None)

Point is a two or more dimensional numerical output that can be visualized in a scatter plot.

class vot.analysis.Result(name: str, abbreviation: Optional[str] = None, description: Optional[str] = '')

Abstract result object base.

This is the base class for all result descriptions.

property abbreviation: str

Abbreviation, if empty, then name is used.

Can be used to define a shorter text representation.

property description: str

Description of the result, used in reports.

property name: str

Name of the result, used in reports.

class vot.analysis.SeparableAnalysis(**kwargs)

Analysis that is separable with respect to trackers and/or sequences, each part can be processed in parallel as a separate job.

The separation is determined by the result of the axes() method: Axes.BOTH means separation in tracker-sequence pairs, Axes.TRACKER means separation according to trackers and Axes.SEQUENCE means separation according to sequences.

class SeparablePart(trackers, sequences, tid, sid)
sequences

Alias for field number 1

sid

Alias for field number 3

tid

Alias for field number 2

trackers

Alias for field number 0

property axes: Axes

Returns the axes of the analysis.

This is used to determine how the analysis is split into parts.

compute(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

The blocking non-parallel version of computation that can be called directly. Splits the job in parts and runs them sequentially. For parallel execution use the analysis processor.

Parameters

  • experiment (Experiment) – Experiment from which to take results

  • trackers (List[Tracker]) – Trackers to run analysis on

  • sequences (List[Sequence]) – Sequences to run analysis on

  • dependencies (List[Grid]) – Results from depndencies, if you override the class and add dependencies, you also have to override this function and handle them.

Returns

Results in a data grid object

Return type

Grid

join(trackers: List[Tracker], sequences: List[Sequence], results: List[Tuple[Any]])

Joins the results of the analysis into a single grid. The results are indexed by trackers and sequences.

Parameters

  • trackers (List[Tracker]) – List of trackers to compute the analysis for.

  • sequences (List[Sequence]) – List of sequences to compute the analysis for.

  • results (List[Tuple[Any]]) – List of results of the analysis.

Returns

Grid with the results of the analysis.

Return type

Grid

static select(meta: Analysis, data: Grid, tracker: int, sequence: int) → Grid

Select appropriate subpart of dependency results for the part, used internally by sequential and parallel processor. This method handles propagation across “singleton” dimension.

The idea is that a certain part of the analysis will only require the part of the result corresponding to the tracker and/or sequence that it is processing.

Parameters

  • meta (Analysis) – Description of the dependency analysis

  • data (Grid) – Returned data of the dependency

  • tracker (int) – Index of the tracker required by the part or None

  • sequence (int) – Index of the sequence required by the part or None

Returns

Subsection of the result, still in Grid format.

Return type

Grid

separate(trackers: List[Tracker], sequences: List[Sequence]) → List[SeparablePart]

Separates the analysis into parts that can be processed separately.

Parameters

  • trackers (List[Tracker]) – List of trackers to compute the analysis for.

  • sequences (List[Sequence]) – List of sequences to compute the analysis for.

Returns: List of parts of the analysis.

abstract subcompute(experiment: Experiment, tracker, sequence, dependencies: List[Grid]) → Tuple[Any]

This method is called for every part of the analysis.

Parameters

  • experiment (Experiment) – [description]

  • tracker ([type]) – [description]

  • sequence ([type]) – [description]

  • dependencies (List[Grid]) – Dependencies of the analysis, note that each dependency is processed using select function to only contain information relevant for the current part of the analysis

Returns

Tuple of results of the analysis

Return type

Tuple[Any]

class vot.analysis.SequenceAggregator(**kwargs)

Base class for sequence aggregators.

Sequence aggregators take the results of a tracker and aggregate them over sequences.

abstract aggregate(tracker: Tracker, sequences: List[Sequence], results: Grid) → Tuple[Any]

Aggregate the results of the analysis over sequences for a single tracker.

Parameters

  • tracker (Tracker) – Tracker to aggregate the results for.

  • sequences (List[Sequence]) – List of sequences to aggregate the results for.

  • results (Grid) – Results of the analysis for the tracker and sequences.

property axes: Axes

The analysis is separable in trackers.

compute(_: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

Compute the analysis for a list of trackers and sequences.

Parameters

  • trackers (List[Tracker]) – List of trackers to compute the analysis for.

  • sequences (List[Sequence]) – List of sequences to compute the analysis for.

  • dependencies (List[Grid]) – List of dependencies, should be one grid with results of the dependency analysis.

Returns

Grid with the results of the analysis.

Return type

Grid

class vot.analysis.SequenceSeparableAnalysis(**kwargs)

Separate analysis into multiple per-tracker tasks, each of them is non- separable.

property axes: Axes

The analysis is separable in sequences.

abstract subcompute(experiment: Experiment, trackers: List[Tracker], sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single sequence.

class vot.analysis.Sorting(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Sorting direction enumeration class.

class vot.analysis.TrackerSeparableAnalysis(**kwargs)

Separate analysis into multiple per-tracker tasks, each of them is non- separable.

property axes: Axes

The analysis is separable in trackers.

abstract subcompute(experiment: Experiment, tracker: Tracker, sequences: List[Sequence], dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single tracker.

vot.analysis.is_special(region: Region, code=None) → bool

Check if the region is special (not a shape) and optionally if it has a specific code.

This module contains the implementation of the analysis processor.

The processor is responsible for executing the analysis tasks in parallel and caching the results.

exception vot.analysis.processor.AnalysisError(cause, task=None)

An exception that is raised when an analysis fails.

print(logoutput)

Print the error to the log output.

property root_cause

The root cause of the error.

property task

The task that caused the error.

class vot.analysis.processor.AnalysisFuture(key)

A future that represents the result of an analysis.

class vot.analysis.processor.AnalysisJoinTask(analysis: SeparableAnalysis, experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence])

A task that joins the results of a separable analysis.

class vot.analysis.processor.AnalysisPartTask(analysis: SeparableAnalysis, experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence])

A task that computes a part of a separable analysis.

class vot.analysis.processor.AnalysisProcessor(executor: Executor = None, cache: Cache = None)

A processor that computes analyses.

cancel()

Cancels all pending analyses.

commit(analysis: Analysis, experiment: Experiment, trackers: Union[Tracker, List[Tracker]], sequences: Union[Sequence, List[Sequence]]) → Future

Commits an analysis for computation. If the analysis is already being computed, the existing future is returned.

Parameters

  • analysis (Analysis) – The analysis to commit.

  • experiment (Experiment) – The experiment to commit the analysis for.

  • trackers (Union[Tracker, List[Tracker]]) – The trackers to commit the analysis for.

  • sequences (Union[Sequence, List[Sequence]]) – The sequences to commit the analysis for.

Returns

A future that represents the result of the analysis.

Return type

Future

static commit_default(analysis: Analysis, experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence])

Commits an analysis to the default analysis processor.

This method is thread-safe. If the analysis is already being computed, this method returns immediately.

static default()

Returns the default analysis processor for the current thread.

Returns

The default analysis processor for the current thread.

Return type

AnalysisProcessor

property pending

The number of pending analyses.

run(analysis: Analysis, experiment: Experiment, trackers: Union[Tracker, List[Tracker]], sequences: Union[Sequence, List[Sequence]]) → Grid

Runs an analysis on a set of trackers and sequences. This method is thread- safe. If the analysis is already being computed, this method returns immediately.

Parameters

  • analysis (Analysis) – The analysis to run.

  • experiment (Experiment) – The experiment to run the analysis on.

  • trackers (Union[Tracker, List[Tracker]]) – The trackers to run the analysis on.

  • sequences (Union[Sequence, List[Sequence]]) – The sequences to run the analysis on.

Returns

The results of the analysis.

Return type

Grid

static run_default(analysis: Analysis, experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence])

Runs an analysis on a set of trackers and sequences. This method is thread- safe. If the analysis is already being computed, this method returns immediately.

Parameters

  • analysis (Analysis) – The analysis to run.

  • experiment (Experiment) – The experiment to run the analysis on.

  • trackers (List[Tracker]) – The trackers to run the analysis on.

  • sequences (List[Sequence]) – The sequences to run the analysis on.

Returns

The results of the analysis.

Return type

Grid

property total

The total number of analyses.

wait()

Waits for all pending analyses to complete.

If no analyses are pending, this method returns immediately.

class vot.analysis.processor.AnalysisTask(analysis: Analysis, experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence])

A task that computes an analysis.

class vot.analysis.processor.DebugExecutor(strict=True)

A synchronous executor used for debugging.

Do not use it in practice.

class Task(fn, args, kwargs, promise)
args

Alias for field number 1

fn

Alias for field number 0

kwargs

Alias for field number 2

promise

Alias for field number 3

shutdown(wait=True)

Shuts down the executor. If wait is True, the method blocks until all tasks are completed.

Parameters

wait (bool, optional) – Wait for all tasks to complete. Defaults to True.

submit(fn, *args, **kwargs)

Submits a task to the executor.

class vot.analysis.processor.ExecutorWrapper(executor: Executor)

A wrapper for an executor that allows to submit tasks with dependencies.

submit(fn, *futures: Tuple[Future], mapping=None) → Future

Submits a task to the executor. The task will be executed when all futures are completed.

Parameters

  • fn (Callable) – The task to execute.

  • futures (Tuple[Future]) – The futures that must be completed before the task is executed.

  • mapping (Dict[Future, Any], optional) – A mapping of futures to values. Defaults to None.

Returns

A future that will be completed when the task is completed.

Return type

Future

property total

The total number of tasks submitted to the executor.

class vot.analysis.processor.FuturesAggregator(*futures: Tuple[Future])

A future that aggregates results from other futures.

cancel()

Cancels the future and all dependencies.

vot.analysis.processor.hashkey(analysis: Analysis, *args)

Compute a hash key for the analysis and its arguments.

The key is used for caching the results.

vot.analysis.processor.process_stack_analyses(workspace: Workspace, trackers: List[Tracker], sequences: Optional[List[str]] = None, experiments: Optional[List[str]] = None)

Process all analyses in the workspace stack. This function is used by the command line interface to run all the analyses provided in a stack.

Parameters

  • workspace (Workspace) – The workspace to process.

  • trackers (List[Tracker]) – The trackers to run the analyses on.

  • sequences (Optional[List[str]], optional) – The sequences to run the analyses on. If None, all sequences are used. Defaults to None.

  • experiments (Optional[List[str]], optional) – The experiments to run the analyses on. If None, all experiments are used. Defaults to None.

vot.analysis.processor.unwrap(arg)

Unwrap a single element list.

Accuracy analysis

Accuracy analysis.

Computes average overlap between predicted and groundtruth regions.

class vot.analysis.accuracy.AverageAccuracy(**kwargs)

Average accuracy analysis.

Computes average overlap between predicted and groundtruth regions.

aggregate(_: Tracker, sequences: List[Sequence], results: Grid)

Aggregate the results of the analysis.

Parameters

  • tracker (Tracker) – Tracker.

  • sequences (List[Sequence]) – List of sequences.

  • results (Grid) – Grid of results.

Returns

Tuple of results.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

This analysis requires a multirun experiment.

dependencies()

List of dependencies.

describe()

Describe the analysis.

class vot.analysis.accuracy.AverageSuccessPlot(**kwargs)

Average success plot analysis.

Computes the average success plot of the tracker.

aggregate(_: Tracker, sequences: List[Sequence], results: Grid)

Aggregate the results of the analysis.

Parameters

  • tracker (Tracker) – Tracker.

  • sequences (List[Sequence]) – List of sequences.

  • results (Grid) – Grid of results.

Returns

Tuple of results.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

This analysis is only compatible with multi-run experiments.

dependencies()

List of dependencies.

describe()

Describe the analysis.

class vot.analysis.accuracy.Overlaps(**kwargs)

Overlaps analysis.

Computes overlaps between predicted and groundtruth regions.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

describe()

Describe the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single sequence.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – List of dependencies.

Returns

Tuple of results.

Return type

Tuple[Any]

class vot.analysis.accuracy.SequenceAccuracy(**kwargs)

Sequence accuracy analysis.

Computes average overlap between predicted and groundtruth regions.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

describe()

Describe the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single sequence.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – List of dependencies.

Returns

Tuple of results.

Return type

Tuple[Any]

class vot.analysis.accuracy.SuccessPlot(**kwargs)

Success plot analysis.

Computes the success plot of the tracker.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

This analysis is only compatible with multi-run experiments.

describe()

Describe the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single sequence.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – List of dependencies.

Returns

Tuple of results.

Return type

Tuple[Any]

vot.analysis.accuracy.gather_overlaps(trajectory: List[Region], groundtruth: List[Region], burnin: int = 10, ignore_unknown: bool = True, ignore_invisible: bool = False, bounds=None, threshold: float = None, ignore_masks: List[Region] = None) → ndarray

Gather overlaps between trajectory and groundtruth regions.

Parameters

  • trajectory (List[Region]) – List of regions predicted by the tracker.

  • groundtruth (List[Region]) – List of groundtruth regions.

  • burnin (int, optional) – Number of frames to skip at the beginning of the sequence. Defaults to 10.

  • ignore_unknown (bool, optional) – Ignore unknown regions in the groundtruth. Defaults to True.

  • ignore_invisible (bool, optional) – Ignore invisible regions in the groundtruth. Defaults to False.

  • bounds ([type], optional) – Bounds of the sequence. Defaults to None.

  • threshold (float, optional) – Minimum overlap to consider. Defaults to None.

  • ignore_masks (List[Region], optional) – List of regions to ignore. Defaults to None.

Returns

List of overlaps.

Return type

np.ndarray

Failure analysis

This module contains the implementation of the FailureCount analysis.

The analysis counts the number of failures in one or more sequences.

class vot.analysis.failures.CumulativeFailureCount(**kwargs)

Count the number of failures over all sequences.

A failure is defined as a region is annotated as Special.FAILURE by the experiment.

aggregate(_: Tracker, sequences: List[Sequence], results: Grid)

Aggregate the analysis for a list of sequences. The aggregation is done by summing the number of failures for each sequence.

Parameters

  • sequences (List[Sequence]) – The list of sequences to aggregate.

  • results (Grid) – The results of the analysis for each sequence.

Returns

The aggregated analysis.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

dependencies()

Return the dependencies of the analysis.

describe()

Describe the analysis.

class vot.analysis.failures.FailureCount(**kwargs)

Count the number of failures in a sequence.

A failure is defined as a region is annotated as Special.FAILURE by the experiment.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

describe()

Describe the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single sequence.

vot.analysis.failures.count_failures(trajectory: List[Region]) → Tuple[int, int]

Count the number of failures in a trajectory.

A failure is defined as a region is annotated as Special.FAILURE by the experiment.

Long-term measures

This module contains the implementation of the long term tracking performance measures.

class vot.analysis.longterm.AccuracyRobustness(**kwargs)

Longterm multi-object accuracy-robustness measure.

property axes: Axes

Returns the axes of the analysis.

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

This analysis is compatible with multi-run experiments.

compute(_: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

Aggregate results from multiple sequences into a single value.

Parameters

  • experiment (Experiment) – Experiment.

  • trackers (List[Tracker]) – Trackers.

  • sequences (List[Sequence]) – Sequences.

  • dependencies (List[Grid]) – Dependencies.

Returns

Aggregated results.

Return type

Grid

dependencies() → List[Analysis]

Returns the dependencies of the analysis.

describe()

Describes the analysis.

class vot.analysis.longterm.AverageQualityAuxiliary(**kwargs)

Computes the average non-reported error, drift-rate error and absence-detection quality.

aggregate(tracker: Tracker, sequences: List[Sequence], results: Grid)

Aggregates the non-reported error, drift-rate error and absence-detection quality.

Parameters

  • tracker (Tracker) – Tracker.

  • sequences (List[Sequence]) – Sequences.

  • results (Grid) – Results.

Returns

Non-reported error, drift-rate error and absence-detection quality.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

This analysis is compatible with multi-run experiments.

dependencies()

Returns the dependencies of the analysis.

describe()

Describes the analysis.

class vot.analysis.longterm.CountFrames(**kwargs)

Counts the number of frames where the tracker is correct, fails, misses, hallucinates or notices an object.

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

This analysis is compatible with multi-run experiments.

describe()

Describes the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Computes the number of frames where the tracker is correct, fails, misses, hallucinates or notices an object.

class vot.analysis.longterm.FScoreCurve(**kwargs)

Computes the F-score curve for a tracker.

property axes

Axes of the analysis.

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

This analysis is compatible with unsupervised experiments.

compute(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

Computes the F-score curve for a tracker.

Parameters

  • experiment (Experiment) – Experiment to compute the F-score curve for.

  • trackers (List[Tracker]) – Trackers to compute the F-score curve for.

  • sequences (List[Sequence]) – Sequences to compute the F-score curve for.

  • dependencies (List[Grid]) – Dependencies of the analysis.

Returns

F-score curve for the given sequences.

Return type

Grid

dependencies()

Dependencies of the analysis.

describe()

Describes the analysis.

class vot.analysis.longterm.PrecisionRecall(**kwargs)

Computes the average precision/recall for a tracker.

property axes

Axes of the analysis.

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

This analysis is compatible with unsupervised experiments.

compute(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

Computes the average precision/recall for a tracker.

Parameters

  • experiment (Experiment) – Experiment to compute the average precision/recall for.

  • trackers (List[Tracker]) – Trackers to compute the average precision/recall for.

  • sequences (List[Sequence]) – Sequences to compute the average precision/recall for.

  • dependencies (List[Grid]) – Dependencies of the analysis.

Returns

Average precision/recall for the given sequences.

Return type

Grid

dependencies()

Dependencies of the analysis.

describe()

Describes the analysis.

class vot.analysis.longterm.PrecisionRecallCurve(**kwargs)

Computes the average precision/recall curve for a tracker.

aggregate(tracker: Tracker, sequences: List[Sequence], results: Grid) → Tuple[Any]

Computes the average precision/recall curve for a tracker.

Parameters

  • tracker (Tracker) – Tracker to compute the average precision/recall curve for.

  • sequences (List[Sequence]) – Sequences to compute the average precision/recall curve for.

  • results (Grid) – Results of the dependencies.

Returns

Average precision/recall curve for the given sequences.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

This analysis is compatible with unsupervised experiments.

dependencies()

Dependencies of the analysis.

describe()

Describes the analysis.

class vot.analysis.longterm.PrecisionRecallCurves(**kwargs)

Computes the precision/recall curves for a tracker for given sequences.

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

dependencies()

Dependencies of the analysis.

describe()

Describes the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Computes the precision/recall curves for a tracker for given sequences.

Parameters

  • experiment (Experiment) – Experiment to compute the precision/recall curves for.

  • tracker (Tracker) – Tracker to compute the precision/recall curves for.

  • sequence (Sequence) – Sequence to compute the precision/recall curves for.

  • dependencies (List[Grid]) – Dependencies of the analysis.

Returns

Precision/recall curves for the given sequence.

Return type

Tuple[Any]

class vot.analysis.longterm.QualityAuxiliary(**kwargs)

Computes the non-reported error, drift-rate error and absence-detection quality.

compatible(experiment: Experiment)

Checks if the experiment is compatible with the analysis.

This analysis is compatible with multi-run experiments.

describe()

Describes the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Computes the non-reported error, drift-rate error and absence-detection quality.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – Dependencies.

Returns

Non-reported error, drift-rate error and absence-detection quality.

Return type

Tuple[Any]

vot.analysis.longterm.compute_tpr_curves(trajectory: List[Region], confidence: List[float], sequence: Sequence, thresholds: List[float], ignore_unknown: bool = True, bounded: bool = True, ignore_masks: List[Region] = None) → Tuple[List[float], List[float]]

Compute the TPR curves for a given trajectory and confidence scores.

Parameters

  • trajectory (List[Region]) – Trajectory to compute the TPR curves for.

  • confidence (List[float]) – Confidence scores for the trajectory.

  • sequence (Sequence) – Sequence to compute the TPR curves for.

  • thresholds (List[float]) – Thresholds to compute the TPR curves for.

  • ignore_unknown (bool, optional) – Ignore unknown regions. Defaults to True.

  • bounded (bool, optional) – Bounded evaluation. Defaults to True.

  • ignore_masks (List[Region], optional) – Ignore masks. Defaults to None.

Returns

TPR curves for the given thresholds.

Return type

List[float], List[float]

vot.analysis.longterm.count_frames(trajectory: List[Region], groundtruth: List[Region], bounds=None, threshold: float = 0, ignore_masks: List[Region] = None) → float

Counts the number of frames where the tracker is correct, fails, misses, hallucinates or notices an object.

Parameters

  • trajectory (List[Region]) – Trajectory of the tracker.

  • groundtruth (List[Region]) – Groundtruth trajectory.

  • bounds (Optional[Region]) – Bounds of the sequence.

  • threshold (float) – Threshold for the overlap.

  • ignore_masks (List[Region]) – Ignore masks.

Returns

Number of frames where the tracker is correct, fails, misses, hallucinates or notices an object.

Return type

float

vot.analysis.longterm.determine_thresholds(scores: Iterable[float], resolution: int) → List[float]

Determine thresholds for a given set of scores and a resolution. The thresholds are determined by sorting the scores and selecting the thresholds that divide the sorted scores into equal sized bins.

Parameters

  • scores (Iterable[float]) – Scores to determine thresholds for.

  • resolution (int) – Number of thresholds to determine.

Returns

List of thresholds.

Return type

List[float]

Multi-start measures

This module contains the implementation of the accuracy-robustness analysis and EAO analysis for the multistart experiment.

class vot.analysis.multistart.AccuracyRobustness(**kwargs)

This analysis computes the accuracy-robustness curve for the multistart experiment.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

The experiment must be a multistart experiment.

describe()

Return the description of the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the accuracy-robustness for each sequence.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – List of dependencies.

Returns

Accuracy, robustness, AR curve, robustness, length of the sequence.

Return type

Tuple[Any]

class vot.analysis.multistart.AverageAccuracyRobustness(**kwargs)

This analysis computes the average accuracy-robustness curve for the multistart experiment.

aggregate(tracker: Tracker, sequences: List[Sequence], results: Grid)

Aggregate the results of the analysis.

Parameters

  • tracker (Tracker) – Tracker.

  • sequences (List[Sequence]) – List of sequences.

  • results (Grid) – Grid of results.

Returns

Aggregated results.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

The experiment must be a multistart experiment.

dependencies()

Return the dependencies of the analysis.

describe()

Return the description of the analysis.

class vot.analysis.multistart.EAOCurve(**kwargs)

This analysis computes the expected average overlap curve for the multistart experiment.

It is an aggregator of the curves for individual sequences.

aggregate(tracker: Tracker, sequences: List[Sequence], results: Grid) → Tuple[Any]

Aggregate the results of the analysis for multiple sequences. The sequences must contain at least one anchor.

Parameters

  • tracker (Tracker) – Tracker.

  • sequences (List[Sequence]) – List of sequences.

  • results (Grid) – Grid of results.

Returns

Results of the analysis.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

The experiment must be a multistart experiment.

dependencies()

Return the dependencies of the analysis.

describe()

Return the description of the analysis.

class vot.analysis.multistart.EAOCurves(**kwargs)

This analysis computes the expected average overlap curve for the multistart experiment.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

The experiment must be a multistart experiment.

describe()

Return the description of the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single sequence. The sequence must contain at least one anchor.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – List of dependencies.

Returns

Results of the analysis.

Return type

Tuple[Any]

class vot.analysis.multistart.EAOScore(**kwargs)

This analysis computes the expected average overlap score for the multistart experiment.

It does this by computing the EAO curve and then integrating it.

property axes

Return the axes of the analysis.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

The experiment must be a multistart experiment.

compute(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

Compute the analysis for multiple sequences. The sequences must contain at least one anchor.

Parameters

  • experiment (Experiment) – Experiment.

  • trackers (List[Tracker]) – List of trackers.

  • sequences (List[Sequence]) – List of sequences.

  • dependencies (List[Grid]) – List of dependencies.

Returns

Grid of results.

Return type

Grid

dependencies()

Return the dependencies of the analysis.

describe()

Return the description of the analysis.

class vot.analysis.multistart.MultiStartFragments(**kwargs)

This analysis computes the accuracy-robustness curve for the multistart experiment.

compatible(experiment: Experiment)

Check if the experiment is compatible with the analysis.

The experiment must be a multistart experiment.

describe()

Return the description of the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute the analysis for a single sequence. The sequence must contain at least one anchor.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – List of dependencies.

Returns

Results of the analysis.

Return type

Tuple[Any]

vot.analysis.multistart.compute_eao_partial(overlaps: List, success: List[bool], curve_length: int)

Compute the EAO curve for a single sequence. The curve is computed as the average overlap at each frame.

Parameters

  • overlaps (List) – List of overlaps for each frame.

  • success (List[bool]) – List of success flags for each frame.

  • curve_length (int) – Length of the curve.

Returns

EAO curve.

Return type

List[float]

Supervision analysis

This module constans common analysis routines for supervised experiment, e.g. Accuracy-Robustness and EAO as defined in VOT papers.

class vot.analysis.supervised.AccuracyRobustness(**kwargs)

Accuracy-Robustness analysis.

Computes accuracy and robustness of a tracker on a given sequence. Accuracy is defined as mean overlap of the tracker region with the groundtruth region. The overlap is computed only for frames where the tracker is not in initialization or failure state. The overlap is computed only for frames after the burnin period. Robustness is defined as a number of failures divided by the total number of frames.

compatible(experiment: Experiment)

Returns True if the analysis is compatible with the experiment.

Only SupervisedExperiment is compatible.

describe()

Returns description of the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Computes accuracy and robustness of a tracker on a given sequence.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequence (Sequence) – Sequence.

  • dependencies (List[Grid]) – Dependencies.

Returns

Accuracy, robustness, AR, number of frames.

Return type

Tuple[Any]

class vot.analysis.supervised.AverageAccuracyRobustness(**kwargs)

Average accuracy-robustness analysis. Computes average accuracy and robustness of a tracker on a given sequence.

Accuracy is defined as mean overlap of the tracker region with the groundtruth region. The overlap is computed only for frames where the tracker is not in initialization or failure state. The overlap is computed only for frames after the burnin period. Robustness is defined as a number of failures divided by the total number of frames. The analysis is computed as an average of accuracy and robustness over all sequences.

aggregate(tracker: Tracker, sequences: List[Sequence], results: Grid)

Aggregates results of the analysis.

Parameters

  • tracker (Tracker) – Tracker.

  • sequences (List[Sequence]) – List of sequences.

  • results (Grid) – Results of the analysis.

Returns

Accuracy, robustness, AR, number of frames.

Return type

Tuple[Any]

compatible(experiment: Experiment)

Returns True if the analysis is compatible with the experiment.

Only SupervisedExperiment is compatible.

dependencies()

Returns dependencies of the analysis.

describe()

Returns description of the analysis.

class vot.analysis.supervised.EAOCurve(**kwargs)

Expected Average Overlap curve analysis.

Computes expected average overlap of a tracker on a given sequence. The overlap is computed only for frames where the tracker is not in initialization or failure state. The overlap is computed only for frames after the burnin period. The analysis is computed as an average of accuracy and robustness over all sequences.

compatible(experiment: Experiment)

Returns True if the analysis is compatible with the experiment.

Only SupervisedExperiment is compatible.

describe()

Returns description of the analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequences: List[Sequence], dependencies: List[Grid]) → Tuple[Any]

Computes expected average overlap of a tracker on a given sequence.

Parameters

  • experiment (Experiment) – Experiment.

  • tracker (Tracker) – Tracker.

  • sequences (List[Sequence]) – List of sequences.

  • dependencies (List[Grid]) – Dependencies.

Returns

Expected average overlap.

Return type

Tuple[Any]

class vot.analysis.supervised.EAOScore(**kwargs)

Expected Average Overlap score analysis.

The analysis is computed as an average of EAO scores over multiple sequences.

property axes

Returns axes of the analysis.

compatible(experiment: Experiment)

Returns True if the analysis is compatible with the experiment.

Only SupervisedExperiment is compatible.

compute(experiment: Experiment, trackers: List[Tracker], sequences: List[Sequence], dependencies: List[Grid]) → Grid

Computes expected average overlap of a tracker on a given sequence.

Parameters

  • experiment (Experiment) – Experiment.

  • trackers (List[Tracker]) – List of trackers.

  • sequences (List[Sequence]) – List of sequences.

  • dependencies (List[Grid]) – Dependencies.

Returns

Expected average overlap.

Return type

Grid

dependencies()

Returns dependencies of the analysis.

describe()

Returns description of the analysis.

vot.analysis.supervised.compute_accuracy(trajectory: List[Region], sequence: Sequence, burnin: int = 10, ignore_unknown: bool = True, bounded: bool = True) → float

Computes accuracy of a tracker on a given sequence. Accuracy is defined as mean overlap of the tracker region with the groundtruth region. The overlap is computed only for frames where the tracker is not in initialization or failure state. The overlap is computed only for frames after the burnin period.

Parameters

  • trajectory (List[Region]) – Tracker trajectory.

  • sequence (Sequence) – Sequence to compute accuracy on.

  • burnin (int, optional) – Burnin period. Defaults to 10.

  • ignore_unknown (bool, optional) – Ignore unknown regions. Defaults to True.

  • bounded (bool, optional) – Consider only first N frames. Defaults to True.

Returns

Accuracy.

Return type

float

vot.analysis.supervised.compute_eao_curve(overlaps: List, weights: List[float], success: List[bool])

Computes EAO curve from a list of overlaps, weights and success flags.

vot.analysis.supervised.count_failures(trajectory: List[Region]) → Tuple[int, int]

Counts number of failures in a trajectory.

Failure is defined as a frame where the tracker is in failure state.

vot.analysis.supervised.locate_failures_inits(trajectory: List[Region]) → Tuple[int, int]

Locates failures and initializations in a trajectory.

Failure is defined as a frame where the tracker is in failure state.

Point tracking measures

This module contains the implementation of point tracking performance measures.

class vot.analysis.point.AveragePointAccuracy(**kwargs)

Dataset-level point tracking accuracy. Aggregates per-sequence results from PointAccuracy into a single weighted average over all sequences.

aggregate(tracker: Tracker, sequences: List[Sequence], results: Grid) → Tuple[Any]

Aggregate per-sequence results into dataset-level scores.

Parameters

  • tracker (Tracker) – The tracker.

  • sequences (List[Sequence]) – All sequences.

  • results (Grid) – Per-sequence results from PointAccuracy.

Returns

Tuple of (d_avg, d_1, d_2, d_4, d_8, d_16, total_frames).

Return type

Tuple[Any]

compatible(experiment: Experiment)

Only compatible with unsupervised experiments.

dependencies()

Returns the dependency on PointAccuracy.

describe()

Returns descriptions of the results produced by this analysis.

class vot.analysis.point.PointAccuracy(**kwargs)

Per-sequence point tracking accuracy. Computes d_avg and per-threshold accuracy scores for each tracker on each sequence.

compatible(experiment: Experiment)

Only compatible with unsupervised experiments.

describe()

Returns descriptions of the results produced by this analysis.

subcompute(experiment: Experiment, tracker: Tracker, sequence: Sequence, dependencies: List[Grid]) → Tuple[Any]

Compute point accuracy for a single tracker on a single sequence.

Parameters

  • experiment (Experiment) – The experiment.

  • tracker (Tracker) – The tracker.

  • sequence (Sequence) – The sequence.

  • dependencies (List[Grid]) – Unused.

Returns

Tuple of (d_avg, d_1, d_2, d_4, d_8, d_16, n_frames).

Return type

Tuple[Any]

vot.analysis.point.compute_point_accuracy(predicted: list, groundtruth: list, width: int, height: int, skip_first: bool = True) → Tuple[float, ...]

Compute per-threshold accuracy and d_avg for a single point trajectory.

For each frame where both prediction and groundtruth are valid points, the L2 distance is computed in a normalized coordinate space equivalent to 256x256 pixels, making the thresholds resolution-independent.

Parameters

  • predicted (list) – Predicted trajectory as a list of regions.

  • groundtruth (list) – Groundtruth trajectory as a list of regions.

  • width (int) – Frame width in pixels.

  • height (int) – Frame height in pixels.

Returns

Tuple of (d_avg, d_1, d_2, d_4, d_8, d_16, n_evaluated) where each d_t is the fraction of frames where normalized L2 error < t, and n_evaluated is the number of frames included in the computation.

Return type

Tuple[float, …]