Sprokit Architecture

Sprokit is a “Stream Processing Toolkit” that provides infrastructure for chaining together algorithms into pipelines for processing streaming data sources. The most common use case of Sprokit is for video processing, but Sprokit is data type agnostic and could be used for any type of streaming data. Sprokit allows the user to dynamically connect and configure a pipeline by chaining together processing nodes called “processes” into a directed graph with data sources and sinks. Sprokit schedules the jobs to run each process and keep data flowing through pipeline. Sprokit also allows processes written in Python to be interconnected with those written in C++.

• Getting Started with sprokit
  • Python Processes
• Process
• How To Make a Process
• Plugins
• Pipeline design
  • Overview
  • Type Safety
  • Introspection
  • Thread safety
  • Error Handling
  • Control Flow
  • Data Flow
  • Ports
  • Packets
  • Pipeline Execution
• Pipeline Declaration Files
  • Configuration Entries
    • Configuration entry attributes
  • Macro Substitution
    • Macro Providers
    • LOCAL Macro Provider
    • ENV Macro Provider
    • CONFIG Macro Provider
    • SYSENV Macro Provider
  • Block Specification
  • Including Files
  • Relativepath Modifier
  • Configuration Section
    • Examples
  • Process definition Section
    • Specification
    • Examples
    • Non-blocking processes
    • Static port values
    • Instrumenting Processes
  • Connection Definition
    • Examples
  • Pipeline Edge Configuration
  • Scheduler configuration
    • Example
  • Clusters Definition File
    • config specifier
    • Input mapping
    • Output mapping
• Pipeline Example
• How To Make a Pipeline