Plugins

FLOWCAST is developed in plugin model. Plugins response to general functional needs as well as to specific requirements related to data processing issues.

• Collectors

  Collectors are collecting information from various systems. Specific collector types require specific configuration attributes (e.g. path to collect files from, interval - how often to collect files). Collector always produces single output stream of events, that can be consumed by processors or distributors.

  Basic suite includes:

  • From file - Picks up a file from local path
  • From directory - Picks up all files from a directory, in lexical order, ignoring subdirectories
  • From HTTP - Reads HTTP resource with GET method e.g download JSON, text file or image

• Processors

  Processors are processing (e.g modifying fields, removing duplicates, taking some other actions) information inside the Flow. Specific processor types require specific configuration attributes (e.g name of fields to be removed or added). Processor can consume multiple input stream of events from collectors or other processors. Processor can produce multiple output streams of events to be consumed by other processors or distributors.

  Basic suite includes:

  • Route data - Split output from the collector to multiple distributors
  • Parse JSON - Processes JSON input using jq syntax
  • Dedicated API parser - Customized plugin, e.g. constructs DLS text from any API response

• Distributors

  Distributors are distributing information to various systems. Specific distributor types require specific configuration attributes (e.g path where to save files, API url and tokens). Distributor always consumes single input stream of events from a collector or processor.

  Basic suite includes:

  • To file - Saving data into a file on a local file system
  • To PanedaAPI DLS - Sends data to PanedaAPI DLS endpoint (DAB+ related)
  • To PanedaAPI SLS - Sends data to PanedaAPI SLS endpoint (DAB+ related)

Coming soon