Vital Doxygen

Types

Other

There are various other vital types that are also used to help direct algorithms or hold specific data associated with an image.

camera	camera_intrinsics
rgb_color	covariance	descriptor
descriptor_request	descriptor_set

Image

class kwiver::vital::image

The representation of an in-memory image.

Images share memory using the image_memory class. This is effectively a view on an image.

Subclassed by kwiver::vital::image_of< uint16_t >, kwiver::vital::image_of< T >

Public Functions

image(const image_pixel_traits &pt = image_pixel_traits ())

Default Constructor.

Parameters

• pt: Change the pixel traits of the image

image(size_t width, size_t height, size_t depth = 1, bool interleave = false, const image_pixel_traits &pt = image_pixel_traits ())

Constructor that allocates image memory.

Create a new blank (empty) image of specified size.

Parameters

• width: Number of pixels in width
• height: Number of pixel rows
• depth: Number of image channels
• pt: data type traits of the image pixels
• interleave: Set if the pixels are interleaved

image(const void *first_pixel, size_t width, size_t height, size_t depth, ptrdiff_t w_step, ptrdiff_t h_step, ptrdiff_t d_step, const image_pixel_traits &pt = image_pixel_traits ())

Constructor that points at existing memory.

Create a new image from supplied memory.

Parameters

• first_pixel: Address of the first pixel in the image. This does not have to be the lowest memory address of the image memory.
• width: Number of pixels wide
• height: Number of pixels high
• depth: Number of image channels
• w_step: pointer increment to get to next pixel column
• h_step: pointer increment to get to next pixel row
• d_step: pointer increment to get to next image channel
• pt: data type traits of the image pixels

image(const image_memory_sptr &mem, const void *first_pixel, size_t width, size_t height, size_t depth, ptrdiff_t w_step, ptrdiff_t h_step, ptrdiff_t d_step, const image_pixel_traits &pt = image_pixel_traits ())

Constructor that shares memory with another image.

Create a new image from existing image.

Parameters

• mem: Shared memory block to be used
• first_pixel: Address of the first pixel in the image. This does not have to be the lowest memory address of the image memory.
• width: Number of pixels wide
• height: Number of pixels high
• depth: Number of image channels
• w_step: pointer increment to get to next pixel column
• h_step: pointer increment to get to next pixel row
• d_step: pointer increment to get to next image channel
• pt: data type traits of the image pixels

image(const image &other)

Copy Constructor.

The new image will share the same memory as the old image

Parameters

• other: The other image.

const image &operator=(const image &other)

Assignment operator.

const image_memory_sptr &memory() const

Const access to the image memory.

image_memory_sptr memory()

Access to the image memory.

size_t size() const

The size of the image managed data in bytes.

The size of the image data in bytes.

This size includes all allocated image memory, which could be larger than width*height*depth*bytes_per_pixel.

Note

This size only accounts for memory which is owned by the image. If this image was constructed as a view into third party memory then the size is reported as 0.

const void *first_pixel() const

Const access to the pointer to first image pixel.

This may differ from data() if the image is a window into a large image memory chunk.

void *first_pixel()

Access to the pointer to first image pixel.

This may differ from data() if the image is a window into a larger image memory chunk.

size_t width() const

The width of the image in pixels.

size_t height() const

The height of the image in pixels.

size_t depth() const

The depth (or number of channels) of the image.

const image_pixel_traits &pixel_traits() const

The trait of the pixel data type.

ptrdiff_t w_step() const

The the step in memory to next pixel in the width direction.

ptrdiff_t h_step() const

The the step in memory to next pixel in the height direction.

ptrdiff_t d_step() const

The the step in memory to next pixel in the depth direction.

bool is_contiguous() const

Return true if the pixels accessible in this image form a contiguous memory block.

template <typename T>

T &at(unsigned i, unsigned j)

Access pixels in the first channel of the image.

Parameters

• i: width position (x)
• j: height position (y)

template <typename T>

const T &at(unsigned i, unsigned j) const

Const access pixels in the first channel of the image.

template <typename T>

T &at(unsigned i, unsigned j, unsigned k)

Access pixels in the image (width, height, channel)

template <typename T>

const T &at(unsigned i, unsigned j, unsigned k) const

Const access pixels in the image (width, height, channel)

void copy_from(const image &other)

Deep copy the image data from another image into this one.

void set_size(size_t width, size_t height, size_t depth)

Set the size of the image.

If the size has not changed, do nothing. Otherwise, allocate new memory matching the new size.

Parameters

• width: a new image width
• height: a new image height
• depth: a new image depth

class kwiver::vital::image_container

An abstract representation of an image container.

This class provides an interface for passing image data between algorithms. It is intended to be a wrapper for image classes in third-party libraries and facilitate conversion between various representations. It provides limited access to the underlying data and is not intended for direct use in image processing algorithms.

Subclassed by kwiver::arrows::ocv::image_container, kwiver::arrows::vcl::image_container, kwiver::arrows::vxl::image_container, kwiver::vital::simple_image_container

Public Functions

virtual ~image_container()

Destructor.

virtual size_t size() const = 0

The size of the image data in bytes.

This size includes all allocated image memory, which could be larger than width*height*depth.

virtual size_t width() const = 0

The width of the image in pixels.

virtual size_t height() const = 0

The height of the image in pixels.

virtual size_t depth() const = 0

The depth (or number of channels) of the image.

virtual image get_image() const = 0

Get and in-memory image class to access the data.

virtual video_metadata_sptr get_metadata() const

Get metadata associated with this image.

virtual void set_metadata(video_metadata_sptr md)

Set metadata associated with this image.

Detections

template <typename T>

class kwiver::vital::bounding_box

Coordinate aligned bounding box.

This class represents a coordinate aligned box. The coordinate system places the origin in the upper left.

A bounding box must be constructed with the correct geometry. Once created, the geometry can not be altered.

Public Functions

bounding_box(vector_type const &upper_left, vector_type const &lower_right)

Create box from two corner points.

Parameters

• upper_left: Upper left corner of box.
• lower_right: Lower right corner of box.

bounding_box(vector_type const &upper_left, T const &width, T const &height)

Create box from point and dimensions.

Parameters

• upper_left: Upper left corner point
• width: Width of box.
• height: Height of box.

bounding_box(T xmin, T ymin, T xmax, T ymax)

Create a box from four coordinates.

Parameters

• xmin: Minimum x coordinate
• ymin: minimum y coordinate
• xmax: Maximum x coordinate
• ymax: Maximum y coordinate

vector_type center() const

Get center coordinate of box.

Return

Center coordinate of box.

vector_type upper_left() const

Get upper left coordinate of box.

Return

Upper left coordinate of box.

vector_type lower_right() const

Get lower right coordinate of box.

Return

Lower right coordinate of box.

T width() const

Get width of box.

Return

Width of box.

T height() const

Get height of box.

Return

Height of box.

double area() const

Get area of box.

Return

Area of box.

class kwiver::vital::detected_object

Detected object class.

This class represents a detected object in image space.

There is one object of this type for each detected object. These objects are defined by a bounding box in the image space. Each object has an optional classification object attached.

Public Functions

detected_object(const bounding_box_d &bbox, double confidence = 1.0, detected_object_type_sptr classifications = detected_object_type_sptr())

Create detected object with bounding box and other attributes.

Parameters

• bbox: Bounding box surrounding detected object, in image coordinates.
• confidence: Detectors confidence in this detection.
• classifications: Optional object classification.

detected_object_sptr clone() const

Create a deep copy of this object.

Return

Managed copy of this object.

bounding_box_d bounding_box() const

Get bounding box from this detection.

The bounding box for this detection is returned. This box is in image coordinates. A default constructed (invalid) bounding box is returned if no box has been supplied for this detection.

Return

A copy of the bounding box.

void set_bounding_box(const bounding_box_d &bbox)

Set new bounding box for this detection.

The supplied bounding box replaces the box for this detection.

Parameters

• bbox: Bounding box for this detection.

double confidence() const

Get confidence for this detection.

This method returns the current confidence value for this detection. Confidence values are in the range of 0.0 - 1.0.

Return

Confidence value for this detection.

void set_confidence(double d)

Set new confidence value for detection.

This method sets a new confidence value for this detection. Confidence values are in the range of [0.0 - 1.0].

Parameters

• d: New confidence value for this detection.

uint64_t index() const

Get detection index.

This method returns the index for this detection.

The detection index is a general purpose field that the application can use to individually identify a detection. In some cases, this field can be used to correlate the detection of an object over multiple frames.

Return

Detection index fof this detections.

void set_index(uint64_t idx)

Set detection index.

This method sets tne index value for this detection.

The detection index is a general purpose field that the application can use to individually identify a detection. In some cases, this field can be used to correlate the detection of an object over multiple frames.

Parameters

• idx: Detection index.

const std::string &detector_name() const

Get detector name.

This method returns the name of the detector that created this element. An empty string is returned if the detector name is not set.

Return

Name of the detector.

void set_detector_name(const std::string &name)

Set detector name.

This method sets the name of the detector for this detection.

Parameters

• name: Detector name.

detected_object_type_sptr type()

Get pointer to optional classifications object.

This method returns the pointer to the classification object if there is one. If there is no classification object the pointer is NULL.

Return

Pointer to classification object or NULL.

void set_type(detected_object_type_sptr c)

Set new classifications for this detection.

This method supplies a new set of class_names and scores for this detection.

Parameters

• c: New classification for this detection

image_container_sptr mask()

Get detection mask image.

This method returns the mask image associated with this detection.

Return

Pointer to the mask image.

void set_mask(image_container_sptr m)

Set mask image for this detection.

This method supplies a new mask image for this detection.

Parameters

• m: Mask image

detected_object::descriptor_sptr descriptor() const

Get descriptor vector.

This method returns an optional descriptor vector that was used to create this detection. This is only set for certain object detectors.

Return

Pointer to the descriptor vector.

void set_descriptor(descriptor_sptr d)

Set descriptor for this detection.

This method sets a descriptor vector that was used to create this detection. This is only set for certain object detectors.

Parameters

• d: Descriptor vector

class kwiver::vital::detected_object_set

Set of detected objects.

This class represents a ordered set of detected objects. The detections are ordered on their basic confidence value.

Reentrancy considerations: Typical usage for a set is for a single detector thread to create a set. It is possible to have an application where two threads are accessing the same set concurrently.

Inherits from kwiver::vital::noncopyable

Unnamed Group

detected_object_set::iterator begin()

Detected object set iterators;.

This method returns an iterator for the set of detected objects. The iterator points to a shared pointer to a detected object.

Return

An iterator over the objects in this set;

Public Functions

detected_object_set()

Create an empty detection set.

This CTOR creates an empty detection set. Detections can be added with the add() method.

detected_object_set(std::vector<detected_object_sptr> const &objs)

Create new set of detected objects.

This CTOR creates a detection set using the supplied vector of detection objects. This can be used to create a new detection set from the output of a select() method.

Parameters

• objs: Vector of detected objects.

detected_object_set_sptr clone() const

Create deep copy.

This method creates a deep copy of this object.

Return

Managed copy of this object.

void add(detected_object_sptr object)

Add detection to set.

This method adds a new detection to this set.

Parameters

• object: Detection to be added to set.

void add(detected_object_set_sptr detections)

Add detection set to set.

This method adds a new detection set to this set.

Parameters

• detections: Detection set to be added to set.

size_t size() const

Get number of detections in this set.

This method returns the number of detections in the set.

Return

Number of detections.

bool empty() const

Returns whether or not this set is empty.

This method returns true if the set is empty, false otherwise.

Return

Whether or not the set is empty.

detected_object_set_sptr select(double threshold = detected_object_type::INVALID_SCORE) const

Select detections based on confidence value.

This method returns a vector of detections ordered by confidence value, high to low. If the optional threshold is specified, then all detections from the set that are less than the threshold are not in the selected set. Note that the selected set may be empty.

The returned vector refers to the actual detections in the set, so if you make changes to the selected set, you are also changing the object in the set. If you want a clean set of detections, call clone() first.

Return

List of detections.

Parameters

• threshold: Select all detections with confidence not less than this value. If this parameter is omitted, then all detections are selected.

detected_object_set_sptr select(const std::string &class_name, double threshold = detected_object_type::INVALID_SCORE) const

Select detections based on class_name.

This method returns a vector of detections that have the specified class_name. These detections are ordered by descending score for the name. Note that the selected set may be empty.

The returned vector refers to the actual detections in the set, so if you make changes to the selected set, you are also changing the object in the set. If you want a clean set of detections, call clone() first.

Return

List of detections.

Parameters

• class_name: class name
• threshold: Select all detections with confidence not less than this value. If this parameter is omitted, then all detections with the label are selected.

void scale(double scale_factor)

Scale all detection locations by some scale factor.

This method changes the bounding boxes within all stored detections by scaling them by some scale factor.

Parameters

• scale: Scale factor

void shift(double col_shift, double row_shift)

Shift all detection locations by some translation offset.

This method shifts the bounding boxes within all stored detections by a supplied column and row shift.

Note: Detections in this set can be shared by multiple sets, so shifting the detections in this set will also shift the detection in other sets that share this detection. If this is going to be a problem, clone() this set before shifting.

Parameters

• col_shift: Column (a.k.a. x, i, width) translation factor
• row_shift: Row (a.k.a. y, j, height) translation factor

kwiver::vital::attribute_set_sptr attributes() const

Get attributes set.

This method returns a pointer to the attribute set that is attached to this object. It is possible that the pointer is NULL, so check before using it.

Return

Pointer to attribute set or NULL

void set_attributes(attribute_set_sptr attrs)

Attach attributes set to this object.

This method attaches the specified attribute set to this object.

Parameters

• attrs: Pointer to attribute set to attach.

Other

class kwiver::vital::camera

An abstract representation of camera.

The base class of cameras is abstract and provides a double precision interface. The templated derived class can store values in either single or double precision.

Subclassed by kwiver::vital::simple_camera

Public Functions

virtual ~camera()

Destructor.

virtual camera_sptr clone() const = 0

Create a clone of this camera object.

virtual vector_3d center() const = 0

Accessor for the camera center of projection (position)

virtual vector_3d translation() const = 0

Accessor for the translation vector.

virtual covariance_3d center_covar() const = 0

Accessor for the covariance of camera center.

virtual rotation_d rotation() const = 0

Accessor for the rotation.

virtual camera_intrinsics_sptr intrinsics() const = 0

Accessor for the intrinsics.

virtual camera_sptr clone_look_at(const vector_3d &stare_point, const vector_3d &up_direction = vector_3d::UnitZ()) const = 0

Create a clone of this camera that is rotated to look at the given point.

Return

New clone, but set to look at the given point.

Parameters

• stare_point: the location at which the camera is oriented to point
• up_direction: the vector which is “up” in the world (defaults to Z-axis)

matrix_3x4d as_matrix() const

Convert to a 3x4 homogeneous projection matrix.

Note

This matrix representation does not account for lens distortion models that may be used in the camera_intrinsics

vector_2d project(const vector_3d &pt) const

Project a 3D point into a 2D image point.

double depth(const vector_3d &pt) const

Compute the distance of the 3D point to the image plane.

Points with negative depth are behind the camera

class kwiver::vital::camera_intrinsics

An abstract representation of camera intrinsics.

Subclassed by kwiver::vital::simple_camera_intrinsics

Public Functions

virtual ~camera_intrinsics()

Destructor.

virtual camera_intrinsics_sptr clone() const = 0

Create a clone of this object.

virtual double focal_length() const = 0

Access the focal length.

virtual vector_2d principal_point() const = 0

Access the principal point.

virtual double aspect_ratio() const = 0

Access the aspect ratio.

virtual double skew() const = 0

Access the skew.

virtual std::vector<double> dist_coeffs() const

Access the distortion coefficients.

matrix_3x3d as_matrix() const

Access the intrinsics as an upper triangular matrix.

Convert to a 3x3 calibration matrix.

Note

This matrix includes the focal length, principal point, aspect ratio, and skew, but does not model distortion

vector_2d map(const vector_2d &norm_pt) const

Map normalized image coordinates into actual image coordinates.

This function applies both distortion and application of the calibration matrix to map into actual image coordinates

vector_2d map(const vector_3d &norm_hpt) const

Map a 3D point in camera coordinates into actual image coordinates.

vector_2d unmap(const vector_2d &norm_pt) const

Unmap actual image coordinates back into normalized image coordinates.

This function applies both application of the inverse calibration matrix and undistortion of the normalized coordinates.

virtual vector_2d distort(const vector_2d &norm_pt) const

Map normalized image coordinates into distorted coordinates.

The default implementation is the identity transformation (no distortion)

virtual vector_2d undistort(const vector_2d &dist_pt) const

Unmap distorted normalized coordinates into normalized coordinates.

The default implementation is the identity transformation (no distortion)

struct kwiver::vital::rgb_color

Struct to represent an RGB tuple.

Public Functions

rgb_color()

Default constructor - set the color to white.

rgb_color(uint8_t const &cr, uint8_t const &cg, uint8_t const &cb)

Constructor.

rgb_color(rgb_color const &c)

Copy Constructor.

template <class Archive>

void serialize(Archive &archive)

Serialization of the class data.

template <unsigned N, typename T>

class kwiver::vital::covariance_

A representation of covariance of a measurement.

Public Functions

covariance_()

Default Constructor - Initialize to identity.

covariance_(const covariance_<N, T> &other)

Copy constructor.

template <typename U>

covariance_(const covariance_<N, U> &other)

Copy Constructor from another type.

covariance_(const T &value)

Constructor - initialize to identity matrix times a scalar.

covariance_(const Eigen::Matrix<T, N, N> &mat)

Constructor - from a matrix.

Averages off diagonal elements to enforce symmetry

Parameters

• mat: matrix to construct from.

covariance_<N, T> &operator=(const covariance_<N, T> &other)

Assignment operator.

Eigen::Matrix<T, N, N> matrix() const

Extract a full matrix.

T &operator()(unsigned int i, unsigned int j)

Return the i-th row, j-th column.

const T &operator()(unsigned int i, unsigned int j) const

Return the i-th row, j-th column (const)

const T *data() const

Access the underlying data.

bool operator==(covariance_<N, T> const &other) const

Equality operator.

bool operator!=(covariance_<N, T> const &other) const

Inequality operator.

template <class Archive>

void serialize(Archive &archive)

Serialization of the class data.

Public Static Attributes

const unsigned int data_size = N * ( N + 1 ) / 2

Number of unique values in a NxN symmetric matrix.

class kwiver::vital::descriptor

A representation of a feature descriptor used in matching.

Subclassed by kwiver::vital::descriptor_array_of< T >

Public Functions

virtual ~descriptor()

Destructor.

virtual std::type_info const &data_type() const = 0

Access the type info of the underlying data (double or float)

virtual std::size_t size() const = 0

The number of elements of the underlying type.

virtual std::size_t num_bytes() const = 0

The number of bytes used to represent the data.

virtual std::vector<byte> as_bytes() const = 0

Return the descriptor as a vector of bytes.

This should always work, even if the underlying type is not bytes

virtual std::vector<double> as_double() const = 0

Return the descriptor as a vector of doubles.

Return an empty vector if this makes no sense for the underlying type.

bool operator==(descriptor const &other) const

Equality operator.

bool operator!=(descriptor const &other) const

Inequality operator.

class kwiver::vital::descriptor_request

A representation of a descriptor request.

This is used by some arbitrary GUI to request and return computed descriptors on some region of the input imagery.

class kwiver::vital::descriptor_set

An abstract ordered collection of feature descriptors.

The base class descriptor_set is abstract and provides an interface for returning a vector of descriptors. There is a simple derived class that stores the data as a vector of descriptors and returns it. Other derived classes can store the data in other formats and convert on demand.

Subclassed by kwiver::arrows::ocv::descriptor_set, kwiver::arrows::vcl::descriptor_set, kwiver::vital::simple_descriptor_set

Public Functions

virtual ~descriptor_set()

Destructor.

virtual size_t size() const = 0

Return the number of descriptors in the set.

virtual std::vector<descriptor_sptr> descriptors() const = 0

Return a vector of descriptor shared pointers.

Algorithms

Base Types

class kwiver::vital::algorithm

An abstract base class for all algorithms.

This class is an abstract base class for all algorithm implementations.

Subclassed by kwiver::vital::algorithm_def< analyze_tracks >, kwiver::vital::algorithm_def< bundle_adjust >, kwiver::vital::algorithm_def< close_loops >, kwiver::vital::algorithm_def< compute_ref_homography >, kwiver::vital::algorithm_def< compute_stereo_depth_map >, kwiver::vital::algorithm_def< compute_track_descriptors >, kwiver::vital::algorithm_def< convert_image >, kwiver::vital::algorithm_def< detect_features >, kwiver::vital::algorithm_def< detected_object_filter >, kwiver::vital::algorithm_def< detected_object_set_input >, kwiver::vital::algorithm_def< detected_object_set_output >, kwiver::vital::algorithm_def< draw_detected_object_set >, kwiver::vital::algorithm_def< draw_tracks >, kwiver::vital::algorithm_def< dynamic_configuration >, kwiver::vital::algorithm_def< estimate_canonical_transform >, kwiver::vital::algorithm_def< estimate_essential_matrix >, kwiver::vital::algorithm_def< estimate_fundamental_matrix >, kwiver::vital::algorithm_def< estimate_homography >, kwiver::vital::algorithm_def< estimate_similarity_transform >, kwiver::vital::algorithm_def< extract_descriptors >, kwiver::vital::algorithm_def< feature_descriptor_io >, kwiver::vital::algorithm_def< filter_features >, kwiver::vital::algorithm_def< filter_tracks >, kwiver::vital::algorithm_def< formulate_query >, kwiver::vital::algorithm_def< image_filter >, kwiver::vital::algorithm_def< image_io >, kwiver::vital::algorithm_def< image_object_detector >, kwiver::vital::algorithm_def< initialize_cameras_landmarks >, kwiver::vital::algorithm_def< match_features >, kwiver::vital::algorithm_def< optimize_cameras >, kwiver::vital::algorithm_def< refine_detections >, kwiver::vital::algorithm_def< split_image >, kwiver::vital::algorithm_def< track_descriptor_set_input >, kwiver::vital::algorithm_def< track_descriptor_set_output >, kwiver::vital::algorithm_def< track_features >, kwiver::vital::algorithm_def< train_detector >, kwiver::vital::algorithm_def< triangulate_landmarks >, kwiver::vital::algorithm_def< uuid_factory >, kwiver::vital::algorithm_def< video_input >, kwiver::vital::algorithm_def< Self >

Public Functions

virtual std::string type_name() const = 0

Return the name of the base algorithm.

std::string impl_name() const

Return the name of this implementation.

config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

Get this alg’s configuration block .

This method returns the required configuration for the algorithm. The implementation of this method should be light-weight and only create and fill in the config block.

This base virtual function implementation returns an empty configuration.

Return

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(config_block_sptr config) = 0

Set this algorithm’s properties via a config block.

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Exceptions

• no_such_configuration_value_exception: Thrown if an expected configuration value is not present.
• algorithm_configuration_exception: Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

• config: The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(config_block_sptr config) const = 0

Check that the algorithm’s configuration config_block is valid.

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Return

true if the configuration check passed and false if it didn’t.

Parameters

• config: The config block to check configuration of.

Public Static Functions

void get_nested_algo_configuration(std::string const &type_name, std::string const &name, config_block_sptr config, algorithm_sptr nested_algo)

Helper function for properly getting a nested algorithm’s configuration.

Adds a configurable algorithm implementation switch for this algorithm. If the variable pointed to by nested_algo is a defined sptr to an implementation, its configuration parameters are merged with the given config_block .

Parameters

• type_name: The type name of the nested algorithm.
• name: An identifying name for the nested algorithm
• config: The config_block instance in which to put the nested algorithm’s configuration.
• nested_algo: The nested algorithm’s sptr variable.

void set_nested_algo_configuration(std::string const &type_name, std::string const &name, config_block_sptr config, algorithm_sptr &nested_algo)

Helper function for properly setting a nested algorithm’s configuration.

Helper method for properly setting a nested algorithm’s configuration.

If the value for the config parameter “type” is supported by the concrete algorithm class, then a new algorithm object is created, configured and returned via the nested_algo pointer.

The nested algorithm will not be set if the implementation switch (as defined in the get_nested_algo_configuration) is not present or set to an invalid value relative to the registered names for this type_name

Parameters

• type_name: The type name of the nested algorithm.
• name: Config block name for the nested algorithm.
• config: The config_block instance from which we will draw configuration needed for the nested algorithm instance.
• nested_algo: The nested algorithm’s sptr variable.

bool check_nested_algo_configuration(std::string const &type_name, std::string const &name, config_block_sptr config)

Helper function for checking that basic nested algorithm configuration is valid.

Helper method for checking that basic nested algorithm configuration is valid.

Check that the expected implementation switch exists and that its value is registered implementation name.

If the name is valid, we also recursively call check_configuration() on the set implementation. This is done with a fresh create so we don’t have to rely on the implementation being defined in the instance this is called from.

Parameters

• type_name: The type name of the nested algorithm.
• name: An identifying name for the nested algorithm.
• config: The config_block to check.

template <typename Self>

class kwiver::vital::algorithm_def

An intermediate templated base class for algorithm definition.

Uses the curiously recurring template pattern (CRTP) to declare the clone function and automatically provide functions to register algorithm, and create new instance by name. Each algorithm definition should be declared as shown below

class my_algo_def
: public algorithm_def<my_algo_def>
{
  ...
};

See

algorithm_impl

Inherits from kwiver::vital::algorithm

Public Types

typedef std::shared_ptr<Self> base_sptr

Shared pointer type of the templated vital::algorithm_def class.

Public Functions

virtual std::string type_name() const

Return the name of this algorithm.

Public Static Functions

static base_sptr create(std::string const &impl_name)

Factory method to make an instance of this algorithm by impl_name.

static std::vector<std::string> registered_names()

Return a vector of the impl_name of each registered implementation.

static void get_nested_algo_configuration(std::string const &name, config_block_sptr config, base_sptr nested_algo)

Helper function for properly getting a nested algorithm’s configuration.

Adds a configurable algorithm implementation switch for this algorithm_def. If the variable pointed to by nested_algo is a defined sptr to an implementation, its configuration parameters are merged with the given config_block .

Parameters

• name: An identifying name for the nested algorithm
• config: The config_block instance in which to put the nested algorithm’s configuration.
• nested_algo: The nested algorithm’s sptr variable.

static void set_nested_algo_configuration(std::string const &name, config_block_sptr config, base_sptr &nested_algo)

Instantiate nested algorithm.

A new concrete algorithm object is created if the value for the config parameter “type” is supported. The new object is returned through the nested_algo parameter.

The nested algorithm will not be set if the implementation switch (as defined in the get_nested_algo_configuration) is not present or set to an invalid value relative to the registered names for this algorithm_def.

Parameters

• name: Config block name for the nested algorithm.
• config: The config_block instance from which we will draw configuration needed for the nested algorithm instance.
• nested_algo: Pointer to the algorithm object is returned here.

static bool check_nested_algo_configuration(std::string const &name, config_block_sptr config)

Helper function for checking that basic nested algorithm configuration is valid.

Check that the expected implementation switch exists and that its value is registered implementation name.

If the name is valid, we also recursively call check_configuration() on the set implementation. This is done with a fresh create so we don’t have to rely on the implementation being defined in the instance this is called from.

Parameters

• name: An identifying name for the nested algorithm.
• config: The config_block to check.

Functionality

class kwiver::vital::algo::analyze_tracks

Abstract base class for writing out human readable track statistics.

Inherits from kwiver::vital::algorithm_def< analyze_tracks >

Subclassed by kwiver::vital::algorithm_impl< analyze_tracks, vital::algo::analyze_tracks >

Public Functions

virtual void print_info(kwiver::vital::track_set_sptr track_set, stream_t &stream = std::cout) const = 0

Output various information about the tracks stored in the input set.

Parameters

• track_set: the tracks to analyze
• stream: an output stream to write data onto

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::bundle_adjust

An abstract base class for bundle adjustment using feature tracks.

Inherits from kwiver::vital::algorithm_def< bundle_adjust >

Subclassed by kwiver::vital::algorithm_impl< bundle_adjust, vital::algo::bundle_adjust >, kwiver::vital::algorithm_impl< hierarchical_bundle_adjust, vital::algo::bundle_adjust >

Public Types

typedef std::function<bool(kwiver::vital::camera_map_sptr, kwiver::vital::landmark_map_sptr)> callback_t

Typedef for the callback function signature.

Public Functions

virtual void optimize(kwiver::vital::camera_map_sptr &cameras, kwiver::vital::landmark_map_sptr &landmarks, kwiver::vital::feature_track_set_sptr tracks, kwiver::vital::video_metadata_map_sptr metadata = nullptr) const = 0

Optimize the camera and landmark parameters given a set of feature tracks.

Implementations of this function should not modify the underlying objects contained in the input structures. Output references should either be new instances or the same as input.

Parameters

• cameras: the cameras to optimize
• landmarks: the landmarks to optimize
• tracks: the feature tracks to use as constraints
• metadata: the frame metadata to use as constraints

void set_callback(callback_t cb)

Set a callback function to report intermediate progress.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::close_loops

Abstract base class for loop closure algorithms.

Different algorithms can perform loop closure in a variety of ways, either in attempt to make either short or long term closures. Similarly to track_features, this class is designed to be called in an online fashion.

Inherits from kwiver::vital::algorithm_def< close_loops >

Subclassed by kwiver::vital::algorithm_impl< close_loops_bad_frames_only, vital::algo::close_loops >, kwiver::vital::algorithm_impl< close_loops_exhaustive, vital::algo::close_loops >, kwiver::vital::algorithm_impl< close_loops_keyframe, vital::algo::close_loops >, kwiver::vital::algorithm_impl< close_loops_multi_method, vital::algo::close_loops >, kwiver::vital::algorithm_impl< vxl::close_loops_homography_guided, vital::algo::close_loops >

Public Functions

virtual kwiver::vital::feature_track_set_sptr stitch(kwiver::vital::frame_id_t frame_number, kwiver::vital::feature_track_set_sptr input, kwiver::vital::image_container_sptr image, kwiver::vital::image_container_sptr mask = kwiver::vital::image_container_sptr ()) const = 0

Attempt to perform closure operation and stitch tracks together.

Return

an updated set of feature tracks after the stitching operation

Parameters

• frame_number: the frame number of the current frame
• input: the input feature track set to stitch
• image: image data for the current frame
• mask: Optional mask image where positive values indicate regions to consider in the input image.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::compute_ref_homography

Abstract base class for mapping each image to some reference image.

This class differs from estimate_homographies in that estimate_homographies simply performs a homography regression from matching feature points. This class is designed to generate different types of homographies from input feature tracks, which can transform each image back to the same coordinate space derived from some initial refrerence image.

Inherits from kwiver::vital::algorithm_def< compute_ref_homography >

Subclassed by kwiver::vital::algorithm_impl< compute_ref_homography_core, vital::algo::compute_ref_homography >

Public Functions

virtual kwiver::vital::f2f_homography_sptr estimate(kwiver::vital::frame_id_t frame_number, kwiver::vital::feature_track_set_sptr tracks) const = 0

Estimate the transformation which maps some frame to a reference frame.

Similarly to track_features, this class was designed to be called in an online fashion for each sequential frame. The output homography will contain a transformation mapping points from the current frame (with frame_id frame_number) to the earliest possible reference frame via post multiplying points on the current frame with the computed homography.

The returned homography is internally allocated and passed back through a smart pointer transferring ownership of the memory to the caller.

Return

estimated homography

Parameters

• frame_number: frame identifier for the current frame
• tracks: the set of all tracked features from the image

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::compute_stereo_depth_map

An abstract base class for detecting feature points.

Inherits from kwiver::vital::algorithm_def< compute_stereo_depth_map >

Public Functions

virtual kwiver::vital::image_container_sptr compute(kwiver::vital::image_container_sptr left_image, kwiver::vital::image_container_sptr right_image) const = 0

Compute a stereo depth map given two images.

Return

a depth map image

Exceptions

• image_size_mismatch_exception: When the given input image sizes do not match.

Parameters

• left_image: contains the first image to process
• right_image: contains the second image to process

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::compute_track_descriptors

An abstract base class for computing track descriptors.

Inherits from kwiver::vital::algorithm_def< compute_track_descriptors >

Subclassed by kwiver::vital::algorithm_impl< burnout_track_descriptors, vital::algo::compute_track_descriptors >

Public Functions

virtual kwiver::vital::track_descriptor_set_sptr compute(kwiver::vital::image_container_sptr image_data, kwiver::vital::track_set_sptr tracks) = 0

Compute track descriptors given an image and tracks.

Return

a set of track descriptors

Parameters

• image_data: contains the image data to process
• tracks: the tracks to extract descriptors around

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::convert_image

An abstract base class for converting base image type.

Inherits from kwiver::vital::algorithm_def< convert_image >

Subclassed by kwiver::vital::algorithm_impl< convert_image, vital::algo::convert_image >, kwiver::vital::algorithm_impl< convert_image_bypass, vital::algo::convert_image >

Public Functions

void set_configuration(kwiver::vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

bool check_configuration(kwiver::vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

Check that the algorithm’s current configuration is valid.

virtual kwiver::vital::image_container_sptr convert(kwiver::vital::image_container_sptr img) const = 0

Convert image base type.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::detect_features

An abstract base class for detecting feature points.

Inherits from kwiver::vital::algorithm_def< detect_features >

Subclassed by kwiver::vital::algorithm_impl< detect_features, vital::algo::detect_features >, kwiver::arrows::ocv::detect_features

Public Functions

virtual kwiver::vital::feature_set_sptr detect(kwiver::vital::image_container_sptr image_data, kwiver::vital::image_container_sptr mask = kwiver::vital::image_container_sptr ()) const = 0

Extract a set of image features from the provided image.

A given mask image should be one-channel (mask->depth() == 1). If the given mask image has more than one channel, only the first will be considered.

Return

a set of image features

Exceptions

• image_size_mismatch_exception: When the given non-zero mask image does not match the size of the dimensions of the given image data.

Parameters

• image_data: contains the image data to process
• mask: Mask image where regions of positive values (boolean true) indicate regions to consider. Only the first channel will be considered.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::detected_object_filter

An abstract base class for filtering sets of detected objects.

A detected object filter accepts a set of detections and produces another set of detections. The output set may be different from the input set. It all depends on the actual implementation. In any case, the input detection set shall be unmodified.

Inherits from kwiver::vital::algorithm_def< detected_object_filter >

Subclassed by kwiver::vital::algorithm_impl< class_probablity_filter, vital::algo::detected_object_filter >

Public Functions

virtual detected_object_set_sptr filter(const detected_object_set_sptr input_set) const = 0

Filter set of detected objects.

This method applies a filter to the input set to create an output set. The input set of detections is unmodified.

Return

Filtered set of detections.

Parameters

• input_set: Set of detections to be filtered.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::detected_object_set_input

Read detected object sets.

This class is the abstract base class for the detected object set writer.

Detection sets from multiple images are stored in a single file with enough information to recreate a unique image identifier, usually the file name, and an associated set of detections.

Inherits from kwiver::vital::algorithm_def< detected_object_set_input >

Subclassed by kwiver::vital::algorithm_impl< detected_object_set_input_csv, vital::algo::detected_object_set_input >, kwiver::vital::algorithm_impl< detected_object_set_input_kw18, vital::algo::detected_object_set_input >

Public Functions

void open(std::string const &filename)

Open a file of detection sets.

This method opens a detection set file for reading.

Parameters

• filename: Name of file to open

Exceptions

• kwiver::vital::path_not_exists: Thrown when the given path does not exist.
• kwiver::vital::path_not_a_file: Thrown when the given path does not point to a file (i.e. it points to a directory).
• kwiver::vital::file_not_found_exception:

void use_stream(std::istream *strm)

Read detections from an existing stream.

This method specifies the input stream to use for reading detections. Using a stream is handy when the detections are available in a stream format.

Parameters

• strm: input stream to use

void close()

Close detection set file.

The currently open detection set file is closed. If there is no currently open file, then this method does nothing.

virtual bool read_set(kwiver::vital::detected_object_set_sptr &set, std::string &image_name) = 0

Read next detected object set.

This method reads the next set of detected objects from the file. False is returned when the end of file is reached.

Return

true if detections are returned, false if end of file.

Parameters

• set: Pointer to the new set of detections. Set may be empty if there are no detections on an image.
• image_name: Name of the image that goes with the detections. This string may be empty depending on the source format.

bool at_eof() const

Determine if input file is at end of file.

This method reports the end of file status for a file open for reading.

Return

true if file is at end.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::detected_object_set_output

Read and write detected object sets.

This class is the abstract base class for the detected object set reader and writer.

Detection sets from multiple images are stored in a single file with enough information to recreate a unique image identifier, usually the file name, and an associated wet of detections.

Inherits from kwiver::vital::algorithm_def< detected_object_set_output >

Subclassed by kwiver::vital::algorithm_impl< detected_object_set_output_csv, vital::algo::detected_object_set_output >, kwiver::vital::algorithm_impl< detected_object_set_output_kw18, vital::algo::detected_object_set_output >, kwiver::vital::algorithm_impl< matlab_detection_output, vital::algo::detected_object_set_output >

Public Functions

void open(std::string const &filename)

Open a file of detection sets.

This method opens a detection set file for writing.

Parameters

• filename: Name of file to open

Exceptions

• kwiver::vital::path_not_exists: Thrown when the given path does not exist.
• kwiver::vital::path_not_a_file: Thrown when the given path does not point to a file (i.e. it points to a directory).

void use_stream(std::ostream *strm)

Write detections to an existing stream.

This method specifies the output stream to use for writing detections. Using a stream is handy when the detections output is available in a stream format.

Parameters

• strm: output stream to use

void close()

Close detection set file.

The currently open detection set file is closed. If there is no currently open file, then this method does nothing.

virtual void write_set(const kwiver::vital::detected_object_set_sptr set, std::string const &image_path) = 0

Write detected object set.

This method writes the specified detected object set and image name to the currently open file.

Parameters

• set: Detected object set
• image_path: File path to image associated with the detections.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::draw_detected_object_set

An abstract base class for algorithms which draw tracks on top of images in various ways, for analyzing results.

Inherits from kwiver::vital::algorithm_def< draw_detected_object_set >

Subclassed by kwiver::vital::algorithm_impl< draw_detected_object_set, vital::algo::draw_detected_object_set >

Public Functions

virtual kwiver::vital::image_container_sptr draw(kwiver::vital::detected_object_set_sptr detected_set, kwiver::vital::image_container_sptr image) = 0

Draw detected object boxes on Image.

This method draws the detections on a copy of the image. The input image is unmodified. The actual boxes that are drawn are controlled by the configuration for the implementation.

Return

Image with boxes and other annotations added.

Parameters

• detected_set: Set of detected objects
• image: Boxes are drawn in this image

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::draw_tracks

An abstract base class for algorithms which draw tracks on top of images in various ways, for analyzing results.

Inherits from kwiver::vital::algorithm_def< draw_tracks >

Subclassed by kwiver::vital::algorithm_impl< draw_tracks, vital::algo::draw_tracks >

Public Functions

virtual kwiver::vital::image_container_sptr draw(kwiver::vital::track_set_sptr display_set, kwiver::vital::image_container_sptr_list image_data, kwiver::vital::track_set_sptr comparison_set = kwiver::vital::track_set_sptr ()) = 0

Draw features tracks on top of the input images.

This process can either be called in an offline fashion, where all tracks and images are provided to the function on the first call, or in an online fashion where only new images are provided on sequential calls. This function can additionally consume a second track set, which can optionally be used to display additional information to provide a comparison between the two track sets.

Return

a pointer to the last image generated

Parameters

• display_set: the main track set to draw
• image_data: a list of images the tracks were computed over
• comparison_set: optional comparison track set

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::dynamic_configuration

Abstract algorithm for getting dynamic configuration values from an external source. This class represents an interface to an external source of configuration values. A typical application would be an external U.I. control that is desired to control the performance of an algorithm by varying some of its configuration values.

Inherits from kwiver::vital::algorithm_def< dynamic_configuration >

Subclassed by kwiver::vital::algorithm_impl< dynamic_config_none, vital::algo::dynamic_configuration >

Public Functions

virtual void set_configuration(config_block_sptr config) = 0

Set this algorithm’s properties via a config block.

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Exceptions

• no_such_configuration_value_exception: Thrown if an expected configuration value is not present.
• algorithm_configuration_exception: Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

• config: The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(config_block_sptr config) const = 0

Check that the algorithm’s configuration config_block is valid.

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Return

true if the configuration check passed and false if it didn’t.

Parameters

• config: The config block to check configuration of.

virtual config_block_sptr get_dynamic_configuration() = 0

Return dynamic configuration values.

This method returns dynamic configuration values. a valid config block is returned even if there are not values being returned.

class kwiver::vital::algo::estimate_canonical_transform

Algorithm for estimating a canonical transform for cameras and landmarks.

A canonical transform is a repeatable transformation that can be recovered from data. In this case we assume at most a similarity transformation. If data sets P1 and P2 are equivalent up to a similarity transformation, then applying a canonical transform to P1 and separately a canonical transform to P2 should bring the data into the same coordinates.

Inherits from kwiver::vital::algorithm_def< estimate_canonical_transform >

Subclassed by kwiver::vital::algorithm_impl< estimate_canonical_transform, vital::algo::estimate_canonical_transform >

Public Functions

virtual kwiver::vital::similarity_d estimate_transform(kwiver::vital::camera_map_sptr const cameras, kwiver::vital::landmark_map_sptr const landmarks) const = 0

Estimate a canonical similarity transform for cameras and points.

Return

An estimated similarity transform mapping the data to the canonical space.

Note

This algorithm does not apply the transformation, it only estimates it.

Parameters

• cameras: The camera map containing all the cameras
• landmarks: The landmark map containing all the 3D landmarks

Exceptions

• algorithm_exception: When the data is insufficient or degenerate.

Public Static Functions

static std::string static_type_name()

Name of this algo definition.

class kwiver::vital::algo::estimate_essential_matrix

An abstract base class for estimating an essential matrix from matching 2D points.

Inherits from kwiver::vital::algorithm_def< estimate_essential_matrix >

Subclassed by kwiver::vital::algorithm_impl< estimate_essential_matrix, vital::algo::estimate_essential_matrix >

Public Functions

essential_matrix_sptr estimate(const kwiver::vital::feature_set_sptr feat1, const kwiver::vital::feature_set_sptr feat2, const kwiver::vital::match_set_sptr matches, const kwiver::vital::camera_intrinsics_sptr cal1, const kwiver::vital::camera_intrinsics_sptr cal2, std::vector<bool> &inliers, double inlier_scale = 1.0) const

Estimate an essential matrix from corresponding features.

Parameters

• feat1: the set of all features from the first image
• feat2: the set of all features from the second image
• matches: the set of correspondences between feat1 and feat2
• cal1: the intrinsic parameters of the first camera
• cal2: the intrinsic parameters of the second camera
• inliers: for each point pair, the value is true if this pair is an inlier to the estimate
• inlier_scale: error distance tolerated for matches to be inliers

essential_matrix_sptr estimate(const kwiver::vital::feature_set_sptr feat1, const kwiver::vital::feature_set_sptr feat2, const kwiver::vital::match_set_sptr matches, const kwiver::vital::camera_intrinsics_sptr cal, std::vector<bool> &inliers, double inlier_scale = 1.0) const

Estimate an essential matrix from corresponding features.

Parameters

• feat1: the set of all features from the first image
• feat2: the set of all features from the second image
• matches: the set of correspondences between feat1 and feat2
• cal: the intrinsic parameters, same for both cameras
• inliers: for each point pair, the value is true if this pair is an inlier to the estimate
• inlier_scale: error distance tolerated for matches to be inliers

virtual kwiver::vital::essential_matrix_sptr estimate(const std::vector<kwiver::vital::vector_2d> &pts1, const std::vector<kwiver::vital::vector_2d> &pts2, const kwiver::vital::camera_intrinsics_sptr cal, std::vector<bool> &inliers, double inlier_scale = 1.0) const

Estimate an essential matrix from corresponding points.

Parameters

• pts1: the vector or corresponding points from the first image
• pts2: the vector of corresponding points from the second image
• cal: the intrinsic parameters, same for both cameras
• inliers: for each point pair, the value is true if this pair is an inlier to the estimate
• inlier_scale: error distance tolerated for matches to be inliers

virtual kwiver::vital::essential_matrix_sptr estimate(const std::vector<kwiver::vital::vector_2d> &pts1, const std::vector<kwiver::vital::vector_2d> &pts2, const kwiver::vital::camera_intrinsics_sptr cal1, const kwiver::vital::camera_intrinsics_sptr cal2, std::vector<bool> &inliers, double inlier_scale = 1.0) const = 0

Estimate an essential matrix from corresponding points.

Parameters

• pts1: the vector or corresponding points from the first image
• pts2: the vector of corresponding points from the second image
• cal1: the intrinsic parameters of the first camera
• cal2: the intrinsic parameters of the second camera
• inliers: for each point pa:wir, the value is true if this pair is an inlier to the estimate
• inlier_scale: error distance tolerated for matches to be inliers

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::estimate_fundamental_matrix

An abstract base class for estimating a fundamental matrix from matching 2D points.

Inherits from kwiver::vital::algorithm_def< estimate_fundamental_matrix >

Subclassed by kwiver::vital::algorithm_impl< estimate_fundamental_matrix, vital::algo::estimate_fundamental_matrix >

Public Functions

fundamental_matrix_sptr estimate(const kwiver::vital::feature_set_sptr feat1, const kwiver::vital::feature_set_sptr feat2, const kwiver::vital::match_set_sptr matches, std::vector<bool> &inliers, double inlier_scale = 1.0) const

Estimate an fundamental matrix from corresponding features.

Parameters

• feat1: the set of all features from the first image
• feat2: the set of all features from the second image
• matches: the set of correspondences between feat1 and feat2
• inliers: for each point pair, the value is true if this pair is an inlier to the estimate
• inlier_scale: error distance tolerated for matches to be inliers

virtual kwiver::vital::fundamental_matrix_sptr estimate(const std::vector<kwiver::vital::vector_2d> &pts1, const std::vector<kwiver::vital::vector_2d> &pts2, std::vector<bool> &inliers, double inlier_scale = 1.0) const = 0

Estimate an fundamental matrix from corresponding points.

Parameters

• pts1: the vector or corresponding points from the first image
• pts2: the vector of corresponding points from the second image
• inliers: for each point pair, the value is true if this pair is an inlier to the estimate
• inlier_scale: error distance tolerated for matches to be inliers

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::estimate_homography

An abstract base class for estimating a homography from matching 2D points.

Inherits from kwiver::vital::algorithm_def< estimate_homography >

Subclassed by kwiver::vital::algorithm_impl< estimate_homography, vital::algo::estimate_homography >

Public Functions

homography_sptr estimate(kwiver::vital::feature_set_sptr feat1, kwiver::vital::feature_set_sptr feat2, kwiver::vital::match_set_sptr matches, std::vector<bool> &inliers, double inlier_scale = 1.0) const

Estimate a homography matrix from corresponding features.

If estimation fails, a NULL-containing sptr is returned

Parameters

• feat1: the set of all features from the source image
• feat2: the set of all features from the destination image
• matches: the set of correspondences between feat1 and feat2
• inliers: for each match in matcher, the value is true if this pair is an inlier to the homography estimate
• inlier_scale: error distance tolerated for matches to be inliers

virtual kwiver::vital::homography_sptr estimate(const std::vector<kwiver::vital::vector_2d> &pts1, const std::vector<kwiver::vital::vector_2d> &pts2, std::vector<bool> &inliers, double inlier_scale = 1.0) const = 0

Estimate a homography matrix from corresponding points.

If estimation fails, a NULL-containing sptr is returned

Parameters

• pts1: the vector or corresponding points from the source image
• pts2: the vector of corresponding points from the destination image
• inliers: for each point pair, the value is true if this pair is an inlier to the homography estimate
• inlier_scale: error distance tolerated for matches to be inliers

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::estimate_similarity_transform

Algorithm for estimating the similarity transform between two point sets.

Inherits from kwiver::vital::algorithm_def< estimate_similarity_transform >

Subclassed by kwiver::vital::algorithm_impl< estimate_similarity_transform, vital::algo::estimate_similarity_transform >

Public Functions

virtual kwiver::vital::similarity_d estimate_transform(std::vector<kwiver::vital::vector_3d> const &from, std::vector<kwiver::vital::vector_3d> const &to) const = 0

Estimate the similarity transform between two corresponding point sets.

Return

An estimated similarity transform mapping 3D points in the from space to points in the to space.

Parameters

• from: List of length N of 3D points in the from space.
• to: List of length N of 3D points in the to space.

Exceptions

• algorithm_exception: When the from and to point sets are misaligned, insufficient or degenerate.

similarity_d estimate_transform(std::vector<kwiver::vital::camera_sptr> const &from, std::vector<kwiver::vital::camera_sptr> const &to) const

Estimate the similarity transform between two corresponding sets of cameras.

Return

An estimated similarity transform mapping camera centers in the from space to camera centers in the to space.

Parameters

• from: List of length N of cameras in the from space.
• to: List of length N of cameras in the to space.

Exceptions

• algorithm_exception: When the from and to point sets are misaligned, insufficient or degenerate.

similarity_d estimate_transform(std::vector<kwiver::vital::landmark_sptr> const &from, std::vector<kwiver::vital::landmark_sptr> const &to) const

Estimate the similarity transform between two corresponding sets of landmarks.

Return

An estinated similarity transform mapping landmark locations in the from space to located in the to space.

Parameters

• from: List of length N of landmarks in the from space.
• to: List of length N of landmarks in the to space.

Exceptions

• algorithm_exception: When the from and to point sets are misaligned, insufficient or degenerate.

similarity_d estimate_transform(kwiver::vital::camera_map_sptr const from, kwiver::vital::camera_map_sptr const to) const

Estimate the similarity transform between two corresponding camera maps.

Cameras with corresponding frame IDs in the two maps are paired for transform estimation. Cameras with no corresponding frame ID in the other map are ignored. An algorithm_exception is thrown if there are no shared frame IDs between the two provided maps (nothing to pair).

Return

An estimated similarity transform mapping camera centers in the from space to camera centers in the to space.

Exceptions

• algorithm_exception: When the from and to point sets are misaligned, insufficient or degenerate.

Parameters

• from: Map of original cameras, sharing N frames with the transformed cameras, where N > 0.
• to: Map of transformed cameras, sharing N frames with the original cameras, where N > 0.

similarity_d estimate_transform(kwiver::vital::landmark_map_sptr const from, kwiver::vital::landmark_map_sptr const to) const

Estimate the similarity transform between two corresponding landmark maps.

Landmarks with corresponding frame IDs in the two maps are paired for transform estimation. Landmarks with no corresponding frame ID in the other map are ignored. An algoirithm_exception is thrown if there are no shared frame IDs between the two provided maps (nothing to pair).

Return

An estimated similarity transform mapping landmark centers in the from space to camera centers in the to space.

Exceptions

• algorithm_exception: When the from and to point sets are misaligned, insufficient or degenerate.

Parameters

• from: Map of original landmarks, sharing N frames with the transformed landmarks, where N > 0.
• to: Map of transformed landmarks, sharing N frames with the original landmarks, where N > 0.

Public Static Functions

static std::string static_type_name()

Name of this algo definition.

class kwiver::vital::algo::extract_descriptors

An abstract base class for extracting feature descriptors.

Inherits from kwiver::vital::algorithm_def< extract_descriptors >

Subclassed by kwiver::vital::algorithm_impl< extract_descriptors, vital::algo::extract_descriptors >, kwiver::arrows::ocv::extract_descriptors

Public Functions

virtual kwiver::vital::descriptor_set_sptr extract(kwiver::vital::image_container_sptr image_data, kwiver::vital::feature_set_sptr features, kwiver::vital::image_container_sptr image_mask = kwiver::vital::image_container_sptr ()) const = 0

Extract from the image a descriptor corresoponding to each feature.

Return

a set of feature descriptors

Parameters

• image_data: contains the image data to process
• features: the feature locations at which descriptors are extracted
• image_mask: Mask image of the same dimensions as image_data where positive values indicate regions of image_data to consider.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::feature_descriptor_io

An abstract base class for reading and writing feature and desriptor sets.

This class represents an abstract interface for reading and writing feature and descriptor sets

Inherits from kwiver::vital::algorithm_def< feature_descriptor_io >

Subclassed by kwiver::vital::algorithm_impl< feature_descriptor_io, vital::algo::feature_descriptor_io >

Public Functions

void load(std::string const &filename, feature_set_sptr &feat, descriptor_set_sptr &desc) const

Load features and descriptors from a file.

Exceptions

• kwiver::vital::path_not_exists: Thrown when the given path does not exist.
• kwiver::vital::path_not_a_file: Thrown when the given path does not point to a file (i.e. it points to a directory).

Parameters

• filename: the path to the file the load
• feat: the set of features to load from the file
• desc: the set of descriptors to load from the file

void save(std::string const &filename, feature_set_sptr feat, descriptor_set_sptr desc) const

Save features and descriptors to a file.

Saves features and/or descriptors to a file. Either feat or desc may be Null, but not both. If both feat and desc are provided then the must be of the same size.

Exceptions

• kwiver::vital::path_not_exists: Thrown when the expected containing directory of the given path does not exist.
• kwiver::vital::path_not_a_directory: Thrown when the expected containing directory of the given path is not actually a directory.

Parameters

• filename: the path to the file to save
• feat: the set of features to write to the file
• desc: the set of descriptors to write to the file

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::filter_features

Abstract base class for feature set filter algorithms.

Inherits from kwiver::vital::algorithm_def< filter_features >

Subclassed by kwiver::vital::algorithm_impl< filter_features_magnitude, vital::algo::filter_features >, kwiver::vital::algorithm_impl< filter_features_scale, vital::algo::filter_features >

Public Functions

feature_set_sptr filter(kwiver::vital::feature_set_sptr input) const

Filter a feature set and return a subset of the features.

The default implementation call the pure virtual function filter(feature_set_sptr feat, std::vector<unsigned int> &indices) const

Return

a filtered version of the feature set (simple_feature_set)

Parameters

• input: The feature set to filter

std::pair<feature_set_sptr, descriptor_set_sptr> filter(kwiver::vital::feature_set_sptr feat, kwiver::vital::descriptor_set_sptr descr) const

Filter a feature_set and its coresponding descriptor_set.

The default implementation calls filter(feature_set_sptr feat, std::vector<unsigned int> &indices) const using with feat and then uses the resulting indices to construct a simple_descriptor_set with the corresponding descriptors.

Return

a pair of the filtered features and descriptors

Parameters

• feat: The feature set to filter
• descr: The parallel descriptor set to filter

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::filter_tracks

Abstract base class for track set filter algorithms.

Inherits from kwiver::vital::algorithm_def< filter_tracks >

Subclassed by kwiver::vital::algorithm_impl< filter_tracks, vital::algo::filter_tracks >

Public Functions

virtual kwiver::vital::track_set_sptr filter(kwiver::vital::track_set_sptr input) const = 0

Filter a track set and return a subset of the tracks.

Return

a filtered version of the track set (simple_track_set)

Parameters

• input: The track set to filter

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::formulate_query

An abstract base class for formulating descriptors for queries.

Inherits from kwiver::vital::algorithm_def< formulate_query >

Subclassed by kwiver::vital::algorithm_impl< formulate_query_core, vital::algo::formulate_query >

Public Functions

void set_configuration(kwiver::vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

bool check_configuration(kwiver::vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

Check that the algorithm’s current configuration is valid.

virtual kwiver::vital::track_descriptor_set_sptr formulate(kwiver::vital::descriptor_request_sptr request) = 0

Formulate query.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::image_filter

Abstract base class for feature set filter algorithms.

Inherits from kwiver::vital::algorithm_def< image_filter >

Subclassed by kwiver::vital::algorithm_impl< matlab_image_filter, vital::algo::image_filter >

Public Functions

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data) = 0

Filter a input image and return resulting image.

This method implements the filtering operation. The resulting image should be the same size as the input image.

Return

a filtered version of the input image

Parameters

• image_data: Image to filter.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::image_io

An abstract base class for reading and writing images.

This class represents an abstract interface for reading and writing images.

Inherits from kwiver::vital::algorithm_def< image_io >

Subclassed by kwiver::vital::algorithm_impl< image_io, vital::algo::image_io >, kwiver::vital::algorithm_impl< image_io_dummy, kwiver::vital::algo::image_io >

Public Functions

image_container_sptr load(std::string const &filename) const

Load image from the file.

Return

an image container refering to the loaded image

Exceptions

• kwiver::vital::path_not_exists: Thrown when the given path does not exist.
• kwiver::vital::path_not_a_file: Thrown when the given path does not point to a file (i.e. it points to a directory).

Parameters

• filename: the path to the file the load

void save(std::string const &filename, kwiver::vital::image_container_sptr data) const

Save image to a file.

Image file format is based on file extension.

Exceptions

• kwiver::vital::path_not_exists: Thrown when the expected containing directory of the given path does not exist.
• kwiver::vital::path_not_a_directory: Thrown when the expected containing directory of the given path is not actually a directory.

Parameters

• filename: the path to the file to save
• data: the image container refering to the image to write

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::image_object_detector

Image object detector base class/.

Inherits from kwiver::vital::algorithm_def< image_object_detector >

Subclassed by kwiver::vital::algorithm_impl< darknet_detector, vital::algo::image_object_detector >, kwiver::vital::algorithm_impl< hough_circle_detector, vital::algo::image_object_detector >, kwiver::vital::algorithm_impl< matlab_image_object_detector, vital::algo::image_object_detector >

Public Functions

virtual detected_object_set_sptr detect(image_container_sptr image_data) const = 0

Find all objects on the provided image.

This method analyzes the supplied image and along with any saved context, returns a vector of detected image objects.

Return

vector of image objects found

Parameters

• image_data: the image pixels

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::initialize_cameras_landmarks

An abstract base class for initialization of cameras and landmarks.

Inherits from kwiver::vital::algorithm_def< initialize_cameras_landmarks >

Subclassed by kwiver::vital::algorithm_impl< initialize_cameras_landmarks, vital::algo::initialize_cameras_landmarks >

Public Types

typedef std::function<bool(kwiver::vital::camera_map_sptr, kwiver::vital::landmark_map_sptr)> callback_t

Typedef for the callback function signature.

Public Functions

virtual void initialize(kwiver::vital::camera_map_sptr &cameras, kwiver::vital::landmark_map_sptr &landmarks, kwiver::vital::feature_track_set_sptr tracks, kwiver::vital::video_metadata_map_sptr metadata = nullptr) const = 0

Initialize the camera and landmark parameters given a set of feature tracks.

The algorithm creates an initial estimate of any missing cameras and landmarks using the available cameras, landmarks, and feature tracks. It may optionally revise the estimates of exisiting cameras and landmarks.

Parameters

• cameras: the cameras to initialize
• landmarks: the landmarks to initialize
• tracks: the feature tracks to use as constraints
• metadata: the frame metadata to use as constraints

void set_callback(callback_t cb)

Set a callback function to report intermediate progress.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::match_features

An abstract base class for matching feature points.

Inherits from kwiver::vital::algorithm_def< match_features >

Subclassed by kwiver::vital::algorithm_impl< match_features, vital::algo::match_features >, kwiver::vital::algorithm_impl< match_features_constrained, vital::algo::match_features >, kwiver::vital::algorithm_impl< match_features_fundamental_matrix, vital::algo::match_features >, kwiver::vital::algorithm_impl< match_features_homography, vital::algo::match_features >, kwiver::arrows::ocv::match_features

Public Functions

virtual kwiver::vital::match_set_sptr match(kwiver::vital::feature_set_sptr feat1, kwiver::vital::descriptor_set_sptr desc1, kwiver::vital::feature_set_sptr feat2, kwiver::vital::descriptor_set_sptr desc2) const = 0

Match one set of features and corresponding descriptors to another.

Return

a set of matching indices from feat1 to feat2

Parameters

• feat1: the first set of features to match
• desc1: the descriptors corresponding to feat1
• feat2: the second set fof features to match
• desc2: the descriptors corresponding to feat2

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::optimize_cameras

Abstract algorithm definition base for optimizing cameras.

Inherits from kwiver::vital::algorithm_def< optimize_cameras >

Subclassed by kwiver::vital::algorithm_impl< optimize_cameras, vital::algo::optimize_cameras >

Public Functions

void optimize(kwiver::vital::camera_map_sptr &cameras, kwiver::vital::feature_track_set_sptr tracks, kwiver::vital::landmark_map_sptr landmarks, kwiver::vital::video_metadata_map_sptr metadata = nullptr) const

Optimize camera parameters given sets of landmarks and feature tracks.

We only optimize cameras that have associating tracks and landmarks in the given maps. The default implementation collects the corresponding features and landmarks for each camera and calls the single camera optimize function.

Exceptions

• invalid_value: When one or more of the given pointer is Null.

Parameters

• cameras: Cameras to optimize.
• tracks: The feature tracks to use as constraints.
• landmarks: The landmarks the cameras are viewing.
• metadata: The optional metadata to constrain the optimization.

virtual void optimize(kwiver::vital::camera_sptr &camera, const std::vector<kwiver::vital::feature_sptr> &features, const std::vector<kwiver::vital::landmark_sptr> &landmarks, kwiver::vital::video_metadata_vector metadata = kwiver::vital::video_metadata_vector()) const = 0

Optimize a single camera given corresponding features and landmarks.

This function assumes that 2D features viewed by this camera have already been put into correspondence with 3D landmarks by aligning them into two parallel vectors

Parameters

• camera: The camera to optimize.
• features: The vector of features observed by camera to use as constraints.
• landmarks: The vector of landmarks corresponding to features.
• metadata: The optional metadata to constrain the optimization.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm definition.

class kwiver::vital::algo::refine_detections

Image object detector base class/.

Inherits from kwiver::vital::algorithm_def< refine_detections >

Subclassed by kwiver::vital::algorithm_impl< refine_detections_write_to_disk, vital::algo::refine_detections >

Public Functions

virtual detected_object_set_sptr refine(image_container_sptr image_data, detected_object_set_sptr detections) const = 0

Refine all object detections on the provided image.

This method analyzes the supplied image and and detections on it, returning a refined set of detections.

Return

vector of image objects refined

Parameters

• image_data: the image pixels
• detections: detected objects

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::split_image

An abstract base class for converting base image type.

Inherits from kwiver::vital::algorithm_def< split_image >

Subclassed by kwiver::vital::algorithm_impl< split_image, vital::algo::split_image >

Public Functions

void set_configuration(kwiver::vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

bool check_configuration(kwiver::vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

Check that the algorithm’s current configuration is valid.

virtual std::vector<kwiver::vital::image_container_sptr> split(kwiver::vital::image_container_sptr img) const = 0

Split image.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::track_descriptor_set_input

Read detected object sets.

This class is the abstract base class for the detected object set writer.

Detection sets from multiple images are stored in a single file with enough information to recreate a unique image identifier, usually the file name, and an associated set of track descriptors.

Inherits from kwiver::vital::algorithm_def< track_descriptor_set_input >

Public Functions

void open(std::string const &filename)

Open a file of track descriptor sets.

This method opens a track descriptor set file for reading.

Parameters

• filename: Name of file to open

Exceptions

• kwiver::vital::path_not_exists: Thrown when the given path does not exist.
• kwiver::vital::path_not_a_file: Thrown when the given path does not point to a file (i.e. it points to a directory).
• kwiver::vital::file_not_found_exception:

void use_stream(std::istream *strm)

Read track descriptors from an existing stream.

This method specifies the input stream to use for reading track descriptors. Using a stream is handy when the track descriptors are available in a stream format.

Parameters

• strm: input stream to use

void close()

Close track descriptor set file.

The currently open track descriptor set file is closed. If there is no currently open file, then this method does nothing.

virtual bool read_set(kwiver::vital::track_descriptor_set_sptr &set, std::string &image_name) = 0

Read next detected object set.

This method reads the next set of detected objects from the file. False is returned when the end of file is reached.

Return

true if track descriptors are returned, false if end of file.

Parameters

• set: Pointer to the new set of track descriptors. Set may be empty if there are no track descriptors on an image.
• image_name: Name of the image that goes with the track descriptors. This string may be empty depending on the source format.

bool at_eof() const

Determine if input file is at end of file.

This method reports the end of file status for a file open for reading.

Return

true if file is at end.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::track_descriptor_set_output

Read and write detected object sets.

This class is the abstract base class for the detected object set reader and writer.

Detection sets from multiple images are stored in a single file with enough information to recreate a unique image identifier, usually the file name, and an associated wet of track descriptors.

Inherits from kwiver::vital::algorithm_def< track_descriptor_set_output >

Subclassed by kwiver::vital::algorithm_impl< track_descriptor_set_output_csv, vital::algo::track_descriptor_set_output >

Public Functions

void open(std::string const &filename)

Open a file of track descriptor sets.

This method opens a track descriptor set file for reading.

Parameters

• filename: Name of file to open

Exceptions

• kwiver::vital::path_not_exists: Thrown when the given path does not exist.
• kwiver::vital::path_not_a_file: Thrown when the given path does not point to a file (i.e. it points to a directory).

void use_stream(std::ostream *strm)

Write track descriptors to an existing stream.

This method specifies the output stream to use for reading track descriptors. Using a stream is handy when the track descriptors are available in a stream format.

Parameters

• strm: output stream to use

void close()

Close track descriptor set file.

The currently open track descriptor set file is closed. If there is no currently open file, then this method does nothing.

virtual void write_set(const kwiver::vital::track_descriptor_set_sptr set, std::string const &image_path) = 0

Write detected object set.

This method writes the specified detected object set and image name to the currently open file.

Parameters

• set: Detected object set
• image_path: File path to image associated with the track descriptors.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::track_features

An abstract base class for tracking feature points.

Inherits from kwiver::vital::algorithm_def< track_features >

Subclassed by kwiver::vital::algorithm_impl< track_features_core, vital::algo::track_features >

Public Functions

virtual feature_track_set_sptr track(feature_track_set_sptr prev_tracks, unsigned int frame_number, image_container_sptr image_data, image_container_sptr mask = image_container_sptr ()) const = 0

Extend a previous set of feature tracks using the current frame.

Return

an updated set of feature tracks including the current frame

Exceptions

• image_size_mismatch_exception: When the given non-zero mask image does not match the size of the dimensions of the given image data.

Parameters

• prev_tracks: the feature tracks from previous tracking steps
• frame_number: the frame number of the current frame
• image_data: the image pixels for the current frame
• mask: Optional mask image that uses positive values to denote regions of the input image to consider for feature tracking. An empty sptr indicates no mask (default value).

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::train_detector

An abstract base class for training object detectors.

Inherits from kwiver::vital::algorithm_def< train_detector >

Subclassed by kwiver::vital::algorithm_impl< darknet_trainer, vital::algo::train_detector >

Public Functions

virtual void train_from_disk(std::vector<std::string> train_image_names, std::vector<kwiver::vital::detected_object_set_sptr> train_groundtruth, std::vector<std::string> test_image_names, std::vector<kwiver::vital::detected_object_set_sptr> test_groundtruth) = 0

Train a detection model given a list of images and detections.

This varient is geared towards offline training.

Parameters

• train_image_list: list of train image filenames
• train_groundtruth: annotations loaded for each image
• test_image_list: list of test image filenames
• test_groundtruth: annotations loaded for each image

void train_from_memory(std::vector<kwiver::vital::image_container_sptr> images, std::vector<kwiver::vital::detected_object_set_sptr> groundtruth)

Train a detection model given images and detections.

This varient is geared towards online training, and is not required to be defined.

Exceptions

• runtime_exception: if not defined.

Parameters

• images: vector of input images
• groundtruth: annotations loaded for each image

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::triangulate_landmarks

An abstract base class for triangulating landmarks.

Inherits from kwiver::vital::algorithm_def< triangulate_landmarks >

Subclassed by kwiver::vital::algorithm_impl< triangulate_landmarks, vital::algo::triangulate_landmarks >

Public Functions

virtual void triangulate(kwiver::vital::camera_map_sptr cameras, kwiver::vital::feature_track_set_sptr tracks, kwiver::vital::landmark_map_sptr &landmarks) const = 0

Triangulate the landmark locations given sets of cameras and feature tracks.

This function only triangulates the landmarks with indices in the landmark map and which have support in the feature tracks and cameras

Parameters

• cameras: the cameras viewing the landmarks
• tracks: the feature tracks to use as constraints
• landmarks: the landmarks to triangulate

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::uuid_factory

Abstract base class for creating uuid’s.

Inherits from kwiver::vital::algorithm_def< uuid_factory >

Subclassed by kwiver::vital::algorithm_impl< uuid_factory_uuid, vital::algo::uuid_factory >

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

class kwiver::vital::algo::video_input

An abstract base class for reading videos.

This class represents an abstract interface for reading videos. Once the video is opened, the frames are returned in order.

Use cases:

1) Reading video from a directory of images.

2) Reading video frames from a list of file names.

3) Reading video from mpeg/video file (one of many formats) (e.g. FMV)

4) Reading video from mpeg/video file (one of many formats) with cropping (e.g. WAMI). This includes Providing geostationary images by cropping to a specific region from an image. This may result in no data if the geo region and image do not intersect.

5) Reading video from network stream. (RTSP) This may result in unexpected end of video conditions and network related disruptions (e.g. missing frames, connection terminating, ...)

A note about the basic capabilities:

HAS_EOV - This capability is set to true if the video source can determine end of video. This is usually the case if the video is being read from a file, but may not be known if the video is coming from a streaming source.

HAS_FRAME_NUMBERS - This capability is set to true if the video source supplies frame numbers. If the video source specifies a frame number, then that number is used when forming a time stamp. If the video does not supply a frame number, the time stamp will not have a frame number.

HAS_FRAME_TIME - This capability is set to true if the video source supplies a frame time. If a frame time is supplied, it is made available in the time stamp for that frame. If the frame time is not supplied, then the timestamp will hot have the time set.

HAS_FRAME_DATA - This capability is set to true if the video source supplies frame images. It may seem strange for a video input algorithm to not supply image data, but happens with a reader that only supplies the metadata.

HAS_ABSOLUTE_FRAME_TIME - This capability is set to true if the video source supplies an absolute, rather than relative frame time. This capability is not set if an absolute frame time can not be found, or if the absolute frame time is configured as “none”.

HAS_METADATA - This capability is set if the video source supplies some type of metadata. The metadata could be in 0601 or 0104 data formats or a different source.

HAS_TIMEOUT - This capability is set if the implementation supports the timeout parameter on the next_frame() method.

All implementations must support the basic traits, in that they are registered with a true or false value. Additional implementation specific (extended) traits may be added. The application should first check to see if a extended trait is registered by calling has_trait() since the actual implementation is set by a configuration entry and not directly known by the application.

Extended capabilities can be created to publish capabilities of non-standard video sources. These capabilities should be namespaced using the name (or abbreviation) of the concrete algorithm followed by the abbreviation of the capability.

Inherits from kwiver::vital::algorithm_def< video_input >

Subclassed by kwiver::vital::algorithm_impl< video_input_filter, vital::algo::video_input >, kwiver::vital::algorithm_impl< video_input_image_list, vital::algo::video_input >, kwiver::vital::algorithm_impl< video_input_pos, vital::algo::video_input >, kwiver::vital::algorithm_impl< video_input_split, vital::algo::video_input >, kwiver::vital::algorithm_impl< vidl_ffmpeg_video_input, vital::algo::video_input >

Public Functions

virtual void open(std::string video_name) = 0

Open a video stream.

This method opens the specified video stream for reading. The format of the name depends on the concrete implementation. It could be a file name or it could be a URI.

Capabilities are set in this call, so they are available after.

Note

Once a video is opened, it starts in an invalid state (i.e. before the first frame of video). You must call next_frame() to step to the first frame of video before calling frame_image().

Parameters

• video_name: Identifier of the video stream.

Exceptions

• exception: if open failed

virtual void close() = 0

Close video stream.

Close the currently opened stream and release resources. Closing a stream that is already closed does not cause a problem.

virtual bool end_of_video() const = 0

Return end of video status.

This method returns the end-of-video status of the input video. true is returned if the last frame has been returned.

This method will always return false for video streams that have no ability to detect end of video, such as network streams.

Return

true if at end of video, false otherwise.

virtual bool good() const = 0

Check whether state of video stream is good.

This method checks the current state of the video stream to see if it is good. A stream is good if it refers to a valid frame such that calls to frame_image() and frame_metadata() are expected to return meaningful data. After calling open() the initial video state is not good until the first call to next_frame().

Return

true if video stream is good, false if not good.

virtual bool next_frame(kwiver::vital::timestamp &ts, uint32_t timeout = 0) = 0

Advance to next frame in video stream.

This method advances the video stream to the next frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing either frame number or time or both, depending on the actual implementation.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the video input is already an end, then calling this method will return false.

Return

true if frame returned, false if end of video.

Parameters

• ts: Time stamp of new frame.
• timeout: Number of seconds to wait. 0 = no timeout.

Exceptions

• video_input_timeout_exception: when the timeout expires.
• video_stream_exception: when there is an error in the video stream.

virtual kwiver::vital::image_container_sptr frame_image() = 0

Get current frame from video stream.

This method returns the image from the current frame. If the video input is already an end, then calling this method will return a null pointer.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same image.

Return

Pointer to image container.

Exceptions

• video_stream_exception: when there is an error in the video stream.

virtual kwiver::vital::video_metadata_vector frame_metadata() = 0

Get metadata collection for current frame.

This method returns the metadata collection for the current frame. It is best to call this after calling next_frame() to make sure the metadata and video are synchronized and that no metadata collections are lost.

Metadata typically occurs less frequently than video frames, so if you call next_frame() and frame_metadata() together while processing a video, there may be times where no metadata is returned. In this case an empty metadata vector will be returned.

Also note that the metadata collection contains a timestamp that can be used to determine where the metadata fits in the video stream.

In video streams without metadata (as determined by the stream capability), this method may return and empty vector, indicating no new metadata has been found.

Calling this method at end of video will return an empty metadata vector.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same metadata.

Return

Vector of metadata pointers.

Exceptions

• video_stream_exception: when there is an error in the video stream.

algorithm_capabilities const &get_implementation_capabilities() const

Return capabilities of concrete implementation.

This method returns the capabilities for the currently opened video.

Return

Reference to supported video capabilities.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.