fiware-cygnus.readthedocs.io Open in urlscan Pro
104.16.254.120  Public Scan

Form analysis
1 forms found in the DOM

GET ./search.html

<form id="rtd-search-form" class="wy-form" action="./search.html" method="get">
  <input type="text" name="q" placeholder="Search docs" title="Type search term here">
</form>

Text Content

fiware-cygnus

 * Home
   * Welcome
   * Cygnus place in FIWARE architecture
   * Further documentation
   * Licensing
   * Reporting issues and contact information

 * Architecture

cygnus-common library

 * Installation and Administration Guide
   * Introduction
   * Hardware requirements
   * Installation from sources
   * Installation via docker
   * Cygnus agent configuration
   * Flume environment configuration
   * log4j configuration
   * Running as a process
   * YAFS
   * Testing
   * Logs and alarms
   * Original API
   * v1 API (beta)
   * Sanity checks
   * Diagnosis procedures
   * Reporting issues and contact information
   * Error persistence
 * Backends catalogue
   * Introduction
   * CKAN backend
   * DynamoDB backend
   * HDFS backend
   * Hive backend
   * Http backend
   * Kafka backend
   * MongoDB backend
   * Orion backend
   * CartoDB backend
   * Elasticsearch backend
   * SQL backend

cygnus-ngsi agent

 * Quick Start Guide
 * Installation and Administration Guide
   * Introduction
   * Installation via docker
   * Installation from sources
   * NGSI agent configuration
   * Configuration examples
   * Testing
   * Backends as short-term historics
   * Name mappings
   * Multitenancy
   * Reliability
   * Performance tips
   * IPv6 support
   * Deprecated and removed functionality
   * Sanity checks
   * Diagnosis procedures
   * Reporting issues and contact information
 * User and Programmer Guide
   * Introduction
   * Connecting Orion Context Broker and Cygnus
   * Adding a new sink
   * Reporting issues and contact information
   * Debugging Cygnus with IntelliJ & Docker
 * Flume extensions catalogue
   * Introduction
   * NGSIRestHandler
   * NGSICartoDBSink
   * NGSICKANSink
   * NGSIDynamoDBSink
   * NGSIKafkaSink
   * NGSIHDFSSink
   * NGSIMongoSink
   * NGSIMySQLSink
   * NGSIPostgreSQLSink
   * NGSIPostGISSink
   * NGSIOracleSink
   * NGSISTHSink
   * NGSIOrionSink
   * NGSIElasticsearchSink
   * NGSIArcgisFeatureTableSink
   * NGSITestSink
   * NGSINameMappingsInterceptor
   * LastDataFunctionality
   * RoundRobinChannelSelector
   * Reporting issues and contact information
 * Integration examples
   * Integration with Apache Spark
   * Integration with Apache Kafka

cygnus-twitter agent

 * Installation and Administration Guide
   * Introduction
   * Installation via docker
   * Installation from sources
   * Twitter agent configuration
   * Running from cmd
   * Testing
   * Logs and alarms
   * Reporting issues and contact information
 * User and Programmer Guide
   * Programmer Guide
   * Reporting issues and contact information
 * Flume extensions catalogue
   * Introduction
   * TwitterSource
   * TwitterHDFSink
   * Reporting issues and contact information
 * Quick Start Guide

cygnus-ngsi-ld agent

 * Quick Start Guide
 * Installation and Administration Guide
   * Introduction
   * Installation via docker
   * Installation from sources
   * NGSI-LD agent configuration
   * Testing
 * Flume extensions catalogue
   * Introduction
   * NGSIRestHandler
   * NGSIPostgreSQLSink
   * NGSIPostGISSink
   * NGSICKANSink
   * Reporting issues and contact information
 * User and Programmer Guide
   * Programmer Guide
   * Reporting issues and contact information

fiware-cygnus
 * Docs »
 * Home
 * Edit on GitHub

--------------------------------------------------------------------------------


CYGNUS




WELCOME

This project is part of FIWARE, being part of the Cosmos Ecosystem.

Cygnus is a connector in charge of persisting certain sources of data in certain
configured third-party storages, creating a historical view of such data.

Internally, Cygnus is based on Apache Flume, a technology addressing the design
and execution of data collection and persistence agents. An agent is basically
composed of a listener or source in charge of receiving the data, a channel
where the source puts the data once it has been transformed into a Flume event,
and a sink, which takes Flume events from the channel in order to persist the
data within its body into a third-party storage.

Cygnus is designed to run a specific Flume agent per source of data.

Current stable release is able to persist the following sources of data in the
following third-party storages:

 * NGSI-like context data in:
   * HDFS, the Hadoop distributed file system.
   * MySQL, the well-know relational database manager.
   * CKAN, an Open Data platform.
   * MongoDB, the NoSQL document-oriented database.
   * STH Comet, a Short-Term Historic database built on top of MongoDB.
   * Kafka, the publish-subscribe messaging broker.
   * DynamoDB, a cloud-based NoSQL database by Amazon Web Services.
   * PostgreSQL, the well-know relational database manager.
   * Carto, the database specialized in geolocated data.
   * Elasticsearch, the distributed full-text search engine with JSON documents.
   * Arcgis, the Arcgis is a geographic information system (GIS).
 * Twitter data in:
   * HDFS, the Hadoop distributed file system.

IMPORTANT NOTE: for the time being, cygnus-ngsi and cygus-twitter agents cannot
be installed in the same base path, because of an incompatibility with the
required version of the httpclient library. Of course, if you are going to use
just one of the agents, there is no problem at all.


CYGNUS PLACE IN FIWARE ARCHITECTURE

Cygnus (more specifically, cygnus-ngsi agent) plays the role of a connector
between Orion Context Broker (which is a NGSI source of data) and many FIWARE
storages such as CKAN, Cosmos Big Data (Hadoop) and STH Comet. Of course, as
previously said, you may add MySQL, Kafka, Carto, etc as other non FIWARE
storages to the FIWARE architecture.




FURTHER DOCUMENTATION

The per agent Quick Start Guide found at readthedocs.org provides a good
documentation summary (cygnus-ngsi, cygnus-twitter).

Nevertheless, both the Installation and Administration Guide and the User and
Programmer Guide for each agent also found at readthedocs.org cover more
advanced topics.

The per agent Flume Extensions Catalogue completes the available documentation
for Cygnus (cygnus-ngsi, cygnus-twitter).

Other interesting links are:

 * Our Apiary Documentation if you want to know how to use our API methods for
   Cygnus.
 * cygnus-ngsi integration examples .
 * cygnus-ngsi introductory course in FIWARE Academy.
 * The Contributing Guidelines if your aim is to extend Cygnus.


LICENSING

Cygnus is licensed under Affero General Public License (GPL) version 3. You can
find a copy of this license in the repository.


REPORTING ISSUES AND CONTACT INFORMATION

Any doubt you may have, please refer to the Cygnus Core Team.

Next

--------------------------------------------------------------------------------


Built with MkDocs using a theme provided by Read the Docs.
75GB Bandwidth | 20,000 Uploads | 50,000 Transformations | 50GB Storage >> Get
Started Free.
www.filestack.com
Ads by EthicalAds
×