Cygnus is a connector in charge of persisting certain sources of data in certain configured third-party storages, creating a historical view of such data.

Internally, Cygnus is based on Apache Flume, a technology addressing the design and execution of data collection and persistence agents. An agent is basically composed of a listener or source in charge of receiving the data, a channel where the source puts the data once it has been transformed into a Flume event, and a sink, which takes Flume events from the channel in order to persist the data within its body into a third-party storage.

Cygnus is designed to run a specific Flume agent per source of data.

Current stable release is able to persist the following sources of data in the following third-party storages:

  • NGSI-like context data in:
    • HDFS, the Hadoop distributed file system.
    • MySQL, the well-know relational database manager.
    • CKAN, an Open Data platform.
    • MongoDB, the NoSQL document-oriented database.
    • FIWARE Comet (STH), a Short-Term Historic database built on top of MongoDB.

Documentation and API

FIWRE Cygnus at Github.