Images

Image Type

class kwiver::vital::image

The representation of an in-memory image.

Images share memory using the image_memory class. This is effectively a view on an image.

Subclassed by kwiver::vital::image_of< uint16_t >, kwiver::vital::image_of< T >

Public Functions

image(const image_pixel_traits &pt = image_pixel_traits ())

Default Constructor.

Parameters
  • pt: Change the pixel traits of the image

image(size_t width, size_t height, size_t depth = 1, bool interleave = false, const image_pixel_traits &pt = image_pixel_traits ())

Constructor that allocates image memory.

Create a new blank (empty) image of specified size.

Parameters
  • width: Number of pixels in width
  • height: Number of pixel rows
  • depth: Number of image channels
  • pt: data type traits of the image pixels
  • interleave: Set if the pixels are interleaved

image(const void *first_pixel, size_t width, size_t height, size_t depth, ptrdiff_t w_step, ptrdiff_t h_step, ptrdiff_t d_step, const image_pixel_traits &pt = image_pixel_traits ())

Constructor that points at existing memory.

Create a new image from supplied memory.

Parameters
  • first_pixel: Address of the first pixel in the image. This does not have to be the lowest memory address of the image memory.
  • width: Number of pixels wide
  • height: Number of pixels high
  • depth: Number of image channels
  • w_step: pointer increment to get to next pixel column
  • h_step: pointer increment to get to next pixel row
  • d_step: pointer increment to get to next image channel
  • pt: data type traits of the image pixels

image(const image_memory_sptr &mem, const void *first_pixel, size_t width, size_t height, size_t depth, ptrdiff_t w_step, ptrdiff_t h_step, ptrdiff_t d_step, const image_pixel_traits &pt = image_pixel_traits ())

Constructor that shares memory with another image.

Create a new image from existing image.

Parameters
  • mem: Shared memory block to be used
  • first_pixel: Address of the first pixel in the image. This does not have to be the lowest memory address of the image memory.
  • width: Number of pixels wide
  • height: Number of pixels high
  • depth: Number of image channels
  • w_step: pointer increment to get to next pixel column
  • h_step: pointer increment to get to next pixel row
  • d_step: pointer increment to get to next image channel
  • pt: data type traits of the image pixels

image(const image &other)

Copy Constructor.

The new image will share the same memory as the old image

Parameters
  • other: The other image.

const image &operator=(const image &other)

Assignment operator.

const image_memory_sptr &memory() const

Const access to the image memory.

image_memory_sptr memory()

Access to the image memory.

size_t size() const

The size of the image managed data in bytes.

The size of the image data in bytes.

This size includes all allocated image memory, which could be larger than width*height*depth*bytes_per_pixel.

Note
This size only accounts for memory which is owned by the image. If this image was constructed as a view into third party memory then the size is reported as 0.

const void *first_pixel() const

Const access to the pointer to first image pixel.

This may differ from data() if the image is a window into a large image memory chunk.

void *first_pixel()

Access to the pointer to first image pixel.

This may differ from data() if the image is a window into a larger image memory chunk.

size_t width() const

The width of the image in pixels.

size_t height() const

The height of the image in pixels.

size_t depth() const

The depth (or number of channels) of the image.

const image_pixel_traits &pixel_traits() const

The trait of the pixel data type.

ptrdiff_t w_step() const

The the step in memory to next pixel in the width direction.

ptrdiff_t h_step() const

The the step in memory to next pixel in the height direction.

ptrdiff_t d_step() const

The the step in memory to next pixel in the depth direction.

bool is_contiguous() const

Return true if the pixels accessible in this image form a contiguous memory block.

template <typename T>
T &at(unsigned i, unsigned j)

Access pixels in the first channel of the image.

Parameters
  • i: width position (x)
  • j: height position (y)

template <typename T>
const T &at(unsigned i, unsigned j) const

Const access pixels in the first channel of the image.

template <typename T>
T &at(unsigned i, unsigned j, unsigned k)

Access pixels in the image (width, height, channel)

template <typename T>
const T &at(unsigned i, unsigned j, unsigned k) const

Const access pixels in the image (width, height, channel)

void copy_from(const image &other)

Deep copy the image data from another image into this one.

void set_size(size_t width, size_t height, size_t depth)

Set the size of the image.

If the size has not changed, do nothing. Otherwise, allocate new memory matching the new size.

Parameters
  • width: a new image width
  • height: a new image height
  • depth: a new image depth

Time Stamp

class kwiver::vital::timestamp

Frame time.

This class represents a timestamp for a single video frame. The time is represented in seconds and frame numbers start at one.

A timestamp has the notion of valid time and valid frame. This is useful when dealing with interpolated timestamps. In this case, a timestamp may have a time, but no frame.

When comparing timestamps, they must be from the same domain. If not, then they are not comparable and all relative operators return false.

If both timestamps have a time, then they are ordered by that value. If both do not have time but both have frame numbers, they are ordered by frame number. If the timestamps do not have some way of being compared, all relational operators return false.

Public Functions

timestamp()

Default constructor.

Created an invalid timestamp.

timestamp(time_t t, frame_t f)

Constructor.

Creates a valid timestamp with specified time and frame number.

Parameters
  • t: Time for timestamp
  • f: Frame number for timestamp

bool is_valid() const

Is timestamp valid.

Both the time and frame must be set for a timestamp to be totally valid.

Return
true if both time and frame are valid

bool has_valid_time() const

Timestamp has valid time.

Indicates that the time has been set for this timestamp.

Return
true if time has been set

bool has_valid_frame() const

Timestamp has valid frame number.

Indicates that the frame number has been set for this timestamp.

Return
true if frame number has been set

time_t get_time_usec() const

Get time from timestamp.

The time portion of the timestamp is returned in micro-seconds. The value will be undetermined if the timestamp does not have a valid time.

See
has_valid_time()
Return
Frame time in micro-seconds

double get_time_seconds() const

Get time in seconds.

The time portion of the timestamp is returned in seconds and fractions.

Return
time in seconds.

frame_t get_frame() const

Get frame number from timestamp.

The frame number value from the timestamp is returned. The first frame in a sequence is usually one. The frame number will be undetermined if the timestamp does not have a valid frame number set.

See
has_valid_frame()
Return
Frame number.

timestamp &set_time_usec(time_t t)

Set time portion of timestamp.

Parameters
  • t: Time for frame.

timestamp &set_time_seconds(double t)

Set time portion of timestamp.

Parameters
  • t: Time for frame.

timestamp &set_frame(frame_t f)

Set frame portion of timestamp.

Parameters
  • f: Frame number

timestamp &set_invalid()

Set timestamp totally invalid.

Both the frame and time are set to invalid

timestamp &set_time_domain_index(int dom)

Set time domain index for this timestamp.

Return
Reference to this object.
Parameters
  • dom: Time domain index

std::string pretty_print() const

Format object in a readable manner.

This method formats a time stamp in a readable and recognizable manner suitable form debugging and logging.

Return
formatted timestamp

Image Container Type

class kwiver::vital::image_container

An abstract representation of an image container.

This class provides an interface for passing image data between algorithms. It is intended to be a wrapper for image classes in third-party libraries and facilitate conversion between various representations. It provides limited access to the underlying data and is not intended for direct use in image processing algorithms.

Subclassed by kwiver::arrows::ocv::image_container, kwiver::arrows::vcl::image_container, kwiver::arrows::vxl::image_container, kwiver::vital::simple_image_container

Public Functions

virtual ~image_container()

Destructor.

virtual size_t size() const = 0

The size of the image data in bytes.

This size includes all allocated image memory, which could be larger than width*height*depth.

virtual size_t width() const = 0

The width of the image in pixels.

virtual size_t height() const = 0

The height of the image in pixels.

virtual size_t depth() const = 0

The depth (or number of channels) of the image.

virtual image get_image() const = 0

Get and in-memory image class to access the data.

virtual video_metadata_sptr get_metadata() const

Get metadata associated with this image.

virtual void set_metadata(video_metadata_sptr md)

Set metadata associated with this image.

Image I/O Algorithm

Instantiate with:

kwiver::vital::algo::image_io_sptr img_io = kwiver::vital::algo::image_io::create("<impl_name>");
Arrow & Configuration <impl_name> options CMake Flag to Enable
OpenCV ocv KWIVER_ENABLE_OPENCV
VXL vxl KWIVER_ENABLE_VXL
class kwiver::vital::algo::image_io

An abstract base class for reading and writing images.

This class represents an abstract interface for reading and writing images.

Inherits from kwiver::vital::algorithm_def< image_io >

Subclassed by kwiver::vital::algorithm_impl< image_io, vital::algo::image_io >, kwiver::vital::algorithm_impl< image_io_dummy, kwiver::vital::algo::image_io >

Public Functions

image_container_sptr load(std::string const &filename) const

Load image from the file.

Return
an image container refering to the loaded image
Exceptions
  • kwiver::vital::path_not_exists: Thrown when the given path does not exist.
  • kwiver::vital::path_not_a_file: Thrown when the given path does not point to a file (i.e. it points to a directory).
Parameters
  • filename: the path to the file the load

void save(std::string const &filename, kwiver::vital::image_container_sptr data) const

Save image to a file.

Image file format is based on file extension.

Exceptions
  • kwiver::vital::path_not_exists: Thrown when the expected containing directory of the given path does not exist.
  • kwiver::vital::path_not_a_directory: Thrown when the expected containing directory of the given path is not actually a directory.
Parameters
  • filename: the path to the file to save
  • data: the image container refering to the image to write

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

Image Filter Algorithm

Instantiate with:

kwiver::vital::algo::image_filter_sptr img_filter = kwiver::vital::algo::image_filter::create("<impl_name>");
Arrow & Configuration <impl_name> options CMake Flag to Enable
N/A N/A N/A

** Currently there are no arrows implementing the image_filter algorithm **

class kwiver::vital::algo::image_filter

Abstract base class for feature set filter algorithms.

Inherits from kwiver::vital::algorithm_def< image_filter >

Subclassed by kwiver::vital::algorithm_impl< matlab_image_filter, vital::algo::image_filter >

Public Functions

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data) = 0

Filter a input image and return resulting image.

This method implements the filtering operation. The resulting image should be the same size as the input image.

Return
a filtered version of the input image
Parameters
  • image_data: Image to filter.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

Split Image Algorithm

Instantiate with:

kwiver::vital::algo::split_image_sptr img_filter = kwiver::vital::algo::split_image::create("<impl_name>");
Arrow & Configuration <impl_name> options CMake Flag to Enable
OpenCV ocv KWIVER_ENABLE_OPENCV
VXL vxl KWIVER_ENABLE_VXL
class kwiver::vital::algo::split_image

An abstract base class for converting base image type.

Inherits from kwiver::vital::algorithm_def< split_image >

Subclassed by kwiver::vital::algorithm_impl< split_image, vital::algo::split_image >

Public Functions

void set_configuration(kwiver::vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

bool check_configuration(kwiver::vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

Check that the algorithm’s current configuration is valid.

virtual std::vector<kwiver::vital::image_container_sptr> split(kwiver::vital::image_container_sptr img) const = 0

Split image.

Public Static Functions

static std::string static_type_name()

Return the name of this algorithm.

Code Example

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
#include "vital/types/image.h"
#include "vital/types/image_container.h"

#include "vital/algo/image_io.h"
#include "vital/algo/image_filter.h"
#include "vital/algo/split_image.h"

#include "vital/plugin_loader/plugin_manager.h"

// We will be calling some OpenCV code, so we need to include
// some OpenCV related files
#include <opencv2/highgui/highgui.hpp>
#include "arrows/ocv/image_container.h"

void how_to_part_01_images()
{
  // Note that the use of _sptr in objet typing.
  // All vital objects (types, algorithms, etc.) provide a shared_pointer typedef
  // This shared pointer typedef is used through out kwiver to elimate the need of memory ownership managers

  // All algorithms are implemented/encapsulated in an arrow, and operate on vital classes
  // There are various algorithms (arrows) that kwiver provides that you can use to analyze imagry
  // In this example, while we will look at a few algorithms, this example highlights the vital data types used by algorithms 
  // These vital data types can then be used as inputs or outputs for algorithms.
  // The vital data types are a sort of common 'glue' between dispart algorithms allowing them to work together.

  // Image I/O algorithms are derived from the kwiver::vital::image_io algorithm interface

  // While we could instantiate a particular algorithm object directly with this code 
  // kwiver::arrows::ocv::image_io ocv_io;
  // kwiver::arrows::vxl::image_io vxl_io;
  // This would require our application to include specific headers be include in our code
  // and require our application to directly link to OpenCV and cause a dependency

  // A key feature of the KWIVER architecture is the ability to dynamically load available algorithms at runtime.
  // This ability allow you to write your application with a set of basic data types and algorithm interfaces and
  // then dynamically replace or reconfigure algorithms at run time without needing to recompile
  // New algorithms can be dropped on disk at and KWIVER can run them
  // The first thing to do is to tell kwiver to load up all it's plugins (which includes all the algorithms)
  kwiver::vital::plugin_manager::instance().load_all_plugins();

  // Refer to this page : http://kwiver.readthedocs.io/en/latest/vital/images.html 
  // Documenting the types and algorithms associated with images:
  //               Various implementations of the algorithm,
  //               The string to use to specify creation of a specific implementation, 
  //               The KWIVER CMake option that builds the specific implementation

  ///////////////
  // Image I/O //
  ///////////////

  // The main image libraries used in KWIVER are the OpenCV and VXL libraries
  kwiver::vital::algo::image_io_sptr ocv_io = kwiver::vital::algo::image_io::create("ocv");
  kwiver::vital::algo::image_io_sptr vxl_io = kwiver::vital::algo::image_io::create("vxl");
  
  // The image_io interface is simple, and has a load and save method
  // These methods will operate on the vital object image_container
  // The image_container is intended to be a wrapper for image to facilitate conversion between
  // various representations. It provides limited access to the underlying
  // data and is not intended for direct use in image processing algorithms.
  kwiver::vital::image_container_sptr ocv_img = ocv_io->load("./cat.jpg");
  kwiver::vital::image_container_sptr vxl_img = vxl_io->load("./cat.jpg");

  // Let's use OpenCV to display the images
  // NOTE, this requires that our application CMakeLists properly find_package(OpenCV)
  // And that we tell our application CMake targets about OpenCV (See the CMakeLists.txt for this file)
  cv::Mat mat;
  // First, convert the image to an OpenCV image object
  mat = kwiver::arrows::ocv::image_container::vital_to_ocv(ocv_img->get_image());
  cv::namedWindow("Image loaded by OpenCV", cv::WINDOW_AUTOSIZE);// Create a window for display.
  cv::imshow("Image loaded by OpenCV", mat);                     // Show our image inside it.
  cv::waitKey(5);
  Sleep(2000);                                                   // Wait for 2s
  cvDestroyWindow("Image loaded by OpenCV");

  // We can do the same, even if the image was originally loaded with VXL
  mat = kwiver::arrows::ocv::image_container::vital_to_ocv(vxl_img->get_image());
  cv::namedWindow("Image loaded by VXL", cv::WINDOW_AUTOSIZE);// Create a window for display.
  cv::imshow("Image loaded by VXL", mat);                     // Show our image inside it.
  cv::waitKey(5);
  Sleep(2000);                                                // Wait for 2s
  cvDestroyWindow("Image loaded by VXL");

  //////////////////
  // Image Filter //
  //////////////////

  // Currently, there is no arrow implementing image filtering
  //kwiver::vital::algo::image_filter_sptr _filter = kwiver::vital::algo::image_filter::create("<impl_name>");
  
  /////////////////
  // Split Image //
  /////////////////

  // These algorithms split an image in half (left and right)
  kwiver::vital::algo::split_image_sptr ocv_split = kwiver::vital::algo::split_image::create("ocv");
  kwiver::vital::algo::split_image_sptr vxl_split = kwiver::vital::algo::split_image::create("vxl");

  std::vector<kwiver::vital::image_container_sptr> ocv_imgs = ocv_split->split(vxl_img);
  for (kwiver::vital::image_container_sptr i : ocv_imgs)
  {
    mat = kwiver::arrows::ocv::image_container::vital_to_ocv(i->get_image());
    cv::namedWindow("OpenCV Split Image", cv::WINDOW_AUTOSIZE);// Create a window for display.
    cv::imshow("OpenCV Split Image", mat);                     // Show our image inside it.
    cv::waitKey(5);
    Sleep(2000);                                               // Wait for 2s
    cvDestroyWindow("OpenCV Split Image");
  }

  std::vector<kwiver::vital::image_container_sptr> vxl_imgs = ocv_split->split(ocv_img);
  for (kwiver::vital::image_container_sptr i : vxl_imgs)
  {
    mat = kwiver::arrows::ocv::image_container::vital_to_ocv(i->get_image());
    cv::namedWindow("VXL Split Image", cv::WINDOW_AUTOSIZE);// Create a window for display.
    cv::imshow("VXL Split Image", mat);                     // Show our image inside it.
    cv::waitKey(5);
    Sleep(2000);                                            // Wait for 2s
    cvDestroyWindow("VXL Split Image");
  }

}