2 This document describes how to use HSQLDB, HBase, and Cassandra data stores to
3 capture time series data using Time Series Data Repository (TSDR) features
4 in OpenDaylight. This document contains configuration, administration, management,
5 usage, and troubleshooting sections for the features.
8 The Time Series Data Repository (TSDR) project in OpenDaylight (ODL) creates a
9 framework for collecting, storing, querying, and maintaining time series data.
10 TSDR provides the framework for plugging in proper data collectors to collect
11 various time series data and store the data into
12 TSDR Data Stores. With a common data model and generic TSDR data persistence
13 APIs, the user can choose various data stores to be plugged into the TSDR
14 persistence framework. Currently, three types of data stores are supported:
15 HSQLDB relational database, HBase NoSQL database, and Cassandra NoSQL database.
17 With the capabilities of data collection, storage, query, aggregation, and
18 purging provided by TSDR, network administrators can leverage various data
19 driven appliations built on top of TSDR for security risk detection,
20 performance analysis, operational configuration optimization, traffic
21 engineering, and network analytics with automated intelligence.
25 TSDR has the following major components:
27 * Data Collection Service
28 * Data Storage Service
29 * TSDR Persistence Layer with data stores as plugins
32 * Grafana integration for time series data visualization
33 * Data Aggregation Service
34 * Data Purging Service
36 The Data Collection Service handles the collection of time series data into TSDR and
37 hands it over to the Data Storage Service. The Data Storage Service stores the data
38 into TSDR through the TSDR Persistence Layer. The TSDR Persistence Layer provides generic
39 Service APIs allowing various data stores to be plugged in. The Data Aggregation
40 Service aggregates time series fine-grained raw data into course-grained roll-up
41 data to control the size of the data. The Data Purging Service periodically purges
42 both fine-grained raw data and course-granined aggregated data according to
43 user-defined schedules.
45 We have implemented The Data Collection Service, Data Storage Service, TSDR
46 Persistence Layer, TSDR HSQLDB Data Store, TSDR HBase Data Store, and TSDR Cassandra
47 Datastore. Among these services and components, time series data is communicated
48 using a common TSDR data model, which is designed and implemented for the
49 abstraction of time series data commonalities. With these functions, TSDR is
50 able to collect the data from the data sources and store them into one of
51 the TSDR data stores: HSQLDB Data Store, HBase Data Store or Cassandra Data
52 Store. Besides a simple query command from Karaf console to retrieve data from the
53 TSDR data stores, we also provided a Data Query Service for the user to use REST API
54 to query the data from the data stores. Moreover, the user can use Grafana, which is
55 a time series visualization tool to view the data stored in TSDR in various charting
58 === Configuring TSDR Data Stores
59 ==== To Configure HSQLDB Data Store
61 The HSQLDB based storage files get stored automatically in <karaf install folder>/tsdr/
62 directory. If you want to change the default storage location, the configuration
63 file to change can be found in <karaf install folder>/etc directory. The filename
64 is org.ops4j.datasource-metric.cfg. Change the last portion of the url=jdbc:hsqldb:./tsdr/metric
65 to point to different directory.
67 ==== To Configure HBase Data Store
69 After installing HBase Server on the same machine as OpenDaylight, if the user accepts the default configuration of the HBase Data Store, the user can directly proceed with the installation of HBase Data Store from Karaf console.
71 Optionally, the user can configure TSDR HBase Data Store following HBase Data Store Configuration Procedure.
73 * HBase Data Store Configuration Steps
75 ** Open the file etc/tsdr-persistence-hbase.peroperties under karaf distribution directory.
76 ** Edit the following parameters:
79 *** HBase client connection pool size
80 *** HBase client write buffer size
82 After the configuration of HBase Data Store is complete, proceed with the installation of HBase Data Store from Karaf console.
84 * HBase Data Store Installation Steps
86 ** Start Karaf Console
87 ** Run the following commands from Karaf Console:
88 feature:install odl-tsdr-hbase
90 ==== To Configure Cassandra Data Store
92 Currently, there's no configuration needed for Cassandra Data Store. The user can use Cassandra data store directly after installing the feature from Karaf console.
94 Additionally separate commands have been implemented to install various data collectors.
96 === Administering or Managing TSDR Data Stores
97 ==== To Administer HSQLDB Data Store
99 Once the TSDR default datastore feature (odl-tsdr-hsqldb-all) is enabled, the TSDR captured OpenFlow statistics metrics can be accessed from Karaf Console by executing the command
101 tsdr:list <metric-category> <starttimestamp> <endtimestamp>
105 * <metric-category> = any one of the following categories FlowGroupStats, FlowMeterStats, FlowStats, FlowTableStats, PortStats, QueueStats
106 * <starttimestamp> = to filter the list of metrics starting this timestamp
107 * <endtimestamp> = to filter the list of metrics ending this timestamp
108 * <starttimestamp> and <endtimestamp> are optional.
109 * Maximum 1000 records will be displayed.
111 ==== To Administer HBase Data Store
113 * Using Karaf Command to retrieve data from HBase Data Store
115 The user first need to install hbase data store from karaf console:
117 feature:install odl-tsdr-hbase
119 The user can retrieve the data from HBase data store using the following commands from Karaf console:
122 tsdr:list <CategoryName> <StartTime> <EndTime>
124 Typing tab will get the context prompt of the arguments when typeing the command in Karaf console.
126 ==== To Administer Cassandra Data Store
128 The user first needs to install Cassandra data store from Karaf console:
130 feature:install odl-tsdr-cassandra
132 Then the user can retrieve the data from Cassandra data store using the following commands from Karaf console:
135 tsdr:list <CategoryName> <StartTime> <EndTime>
137 Typing tab will get the context prompt of the arguments when typeing the command in Karaf console.
139 === Installing TSDR Data Collectors
141 When the user uses HSQLDB data store and installed "odl-tsdr-hsqldb-all" feature from Karaf console, besides the HSQLDB data store, OpenFlow data collector is also installed with this command. However, if the user needs to use other collectors, such as NetFlow Collector, Syslog Collector, SNMP Collector, and Controller Metrics Collector, the user needs to install them with separate commands. If the user uses HBase or Cassandra data store, no collectors will be installed when the data store is installed. Instead, the user needs to install each collector separately using feature install command from Karaf console.
143 The following is the list of supported TSDR data collectors with the associated feature install commands:
145 * OpenFlow Data Collector
147 feature:install odl-tsdr-openflow-statistics-collector
149 * SNMP Data Collector
151 feature:install odl-tsdr-snmp-data-collector
153 * NetFlow Data Collector
155 feature:install odl-tsdr-netflow-statistics-collector
157 * sFlow Data Collector
158 feature:install odl-tsdr-sflow-statistics-colletor
160 * Syslog Data Collector
162 feature:install odl-tsdr-syslog-collector
164 * Controller Metrics Collector
166 feature:install odl-tsdr-controller-metrics-collector
168 In order to use controller metrics collector, the user needs to install Sigar library.
170 The following is the instructions for installing Sigar library on Ubuntu:
172 *** Install back end library by "sudo apt-get install libhyperic-sigar-java"
173 *** Execute "export LD_LIBRARY_PATH=/usr/lib/jni/:/usr/lib:/usr/local/lib" to set the path of the JNI (you can add this to the ".bashrc" in your home directory)
174 *** Download the file "sigar-1.6.4.jar". It might be also in your ".m2" directory under "~/.m2/resources/org/fusesource/sigar/1.6.4"
175 *** Create the directory "org/fusesource/sigar/1.6.4" under the "system" directory in your controller home directory and place the "sigar-1.6.4.jar" there
177 === Configuring TSDR Data Collectors
179 * SNMP Data Collector Device Credential Configuration
181 After installing SNMP Data Collector, a configuration file under etc/ directory of ODL distribution is generated: etc/tsdr.snmp.cfg is created.
183 The following is a sample tsdr.snmp.cfg file:
185 credentials=[192.168.0.2,public],[192.168.0.3,public]
187 The above credentials indicate that TSDR SNMP Collector is going to connect to two devices. The IPAddress and Read community string of these two devices are (192.168.0.2, public), and (192.168.0.3) respectively.
189 The user can make changes to this configuration file any time during runtime. The configuration will be picked up by TSDR in the next cycle of data collection.
191 ==== Polling interval configuration for SNMP Collector and OpenFlow Stats Collector
193 The default polling interval of SNMP Collector and OpenFlow Stats Collector is 30 seconds and 15 seconds respectively. The user can change the polling interval through restconf APIs at any time. The new polling interval will be picked up by TSDR in the next collection cycle.
195 * Retrieve Polling Interval API for SNMP Collector
196 ** URL: http://localhost:8181/restconf/config/tsdr-snmp-data-collector:TSDRSnmpDataCollectorConfig
199 * Update Polling Interval API for SNMP Collector
200 ** URL: http://localhost:8181/restconf/operations/tsdr-snmp-data-collector:setPollingInterval
202 ** Content Type: application/json
211 * Retrieve Polling Interval API for OpenFlowStats Collector
212 ** URL: http://localhost:8181/restconf/config/tsdr-openflow-statistics-collector:TSDROSCConfig
215 * Update Polling Interval API for OpenFlowStats Collector
216 ** URL: http://localhost:8181/restconf/operations/tsdr-openflow-statistics-collector:setPollingInterval
218 ** Content Type: application/json
227 === Querying TSDR from REST APIs
229 TSDR provides two REST APIs for querying data stored in TSDR data stores.
231 * Query of TSDR Metrics
232 ** URL: http://localhost:8181/tsdr/metrics/query
235 *** tsdrkey=[NID=][DC=][MN=][RK=]
237 The TSDRKey format indicates the NodeID(NID), DataCategory(DC), MetricName(MN), and RecordKey(RK) of the monitored objects.
238 For example, the following is a valid tsdrkey:
239 [NID=openflow:1][DC=FLOWSTATS][MN=PacketCount][RK=Node:openflow:1,Table:0,Flow:3]
240 The following is also a valid tsdrkey:
241 tsdrkey=[NID=][DC=FLOWSTATS][MN=][RK=]
242 In the case when the sections in the tsdrkey is empty, the query will return all the records in the TSDR data store that matches the filled tsdrkey. In the above example, the query will return all the data in FLOWSTATS data category.
243 The query will return only the first 1000 records that match the query criteria.
245 *** from=<time_in_seconds>
246 *** until=<time_in_seconds>
248 The following is an example curl command for querying metric data from TSDR data store:
250 curl -G -v -H "Accept: application/json" -H "Content-Type: application/json" "http://localhost:8181/tsdr/metrics/query" --data-urlencode "tsdrkey=[NID=][DC=FLOWSTATS][MN=][RK=]" --data-urlencode "from=0" --data-urlencode "until=240000000000"|more
252 * Query of TSDR Log type of data
253 ** URL:http://localhost:8181/tsdr/logs/query
256 *** tsdrkey=tsdrkey=[NID=][DC=][RK=]
258 The TSDRKey format indicates the NodeID(NID), DataCategory(DC), and RecordKey(RK) of the monitored objects.
259 For example, the following is a valid tsdrkey:
260 [NID=openflow:1][DC=NETFLOW][RK]
261 The query will return only the first 1000 records that match the query criteria.
263 *** from=<time_in_seconds>
264 *** until=<time_in_seconds>
266 The following is an example curl command for querying log type of data from TSDR data store:
268 curl -G -v -H "Accept: application/json" -H "Content-Type: application/json" "http://localhost:8181/tsdr/logs/query" --data-urlencode "tsdrkey=[NID=][DC=NETFLOW][RK=]" --data-urlencode "from=0" --data-urlencode "until=240000000000"|more
270 === Grafana integration with TSDR
272 TSDR provides northbound integration with Grafana time series data visualization tool. All the metric type of data stored in TSDR data store can be visualized using Grafana.
274 For the detailed instruction about how to install and configure Grafana to work with TSDR, please refer to the following link:
276 https://wiki.opendaylight.org/view/Grafana_Integration_with_TSDR_Step-by-Step
278 === Purging Service configuration
280 After the data stores are installed from Karaf console, the purging service will be installed as well. A configuration file called tsdr.data.purge.cfg will be generated under etc/ directory of ODL distribution.
282 The following is the sample default content of the tsdr.data.purge.cfg file:
285 data_purge_enabled=true
286 data_purge_time=23:59:59
287 data_purge_interval_in_minutes=1440
288 retention_time_in_hours=168
290 The host indicates the IPAddress of the data store. In the case when the data store is together with ODL controller, 127.0.0.1 should be the right value for the host IP. The other attributes are self-explained. The user can change those attributes at any time. The configuration change will be picked up right away by TSDR Purging service at runtime.
292 === How to use TSDR to collect, store, and view OpenFlow Interface Statistics
295 This tutorial describes an example of using TSDR to collect, store, and view one type of time series data in OpenDaylight environment.
299 You would need to have the following as prerequisits:
301 * One or multiple OpenFlow enabled switches. Alternatively, you can use mininet to simulate such a switch.
302 * Successfully installed OpenDaylight Controller.
303 * Successfully installed HBase Data Store following TSDR HBase Data Store Installation Guide.
304 * Connect the OpenFlow enabled switch(es) to OpenDaylight Controller.
306 ==== Target Environment
307 HBase data store is only supported in Linux operation system.
311 * Start OpenDaylight.
313 * Connect OpenFlow enabled switch(es) to the controller.
315 ** If using mininet, run the following commands from mininet command line:
317 *** mn --topo single,3 --controller 'remote,ip=172.17.252.210,port=6653' --switch ovsk,protocols=OpenFlow13
320 * Install tsdr hbase feature from Karaf:
322 ** feature:install odl-tsdr-hbase
324 * Install OpenFlow Statistics Collector from Karaf:
326 ** feature:install odl-tsdr-openflow-statistics-collector
328 * run the following command from Karaf console:
330 ** tsdr:list PORTSTATS
332 You should be able to see the interface statistics of the switch(es) from the HBase Data Store. If there are too many rows, you can use "tsdr:list InterfaceStats|more" to view it page by page.
334 By tabbing after "tsdr:list", you will see all the supported data categories. For example, "tsdr:list FlowStats" will output the Flow statistics data collected from the switch(es).
339 All TSDR features and components write logging information including information messages, warnings, errors and debug messages into karaf.log.
341 ==== HBase and Cassandra logs
343 For HBase and Cassandra data stores, the database level logs are written into HBase log and Cassandra logs.
346 *** HBase log is under <HBase-installation-directory>/logs/.
349 *** Cassandra log is under {cassandra.logdir}/system.log. The default {cassandra.logdir} is /var/log/cassandra/.
353 TSDR gets the data from a variety of sources, which can be secured in different ways.
356 *** The OpenFlow data can be configured with Transport Layer Security (TLS) since the OpenFlow Plugin that TSDR depends on provides this security support.
359 *** The SNMP version3 has security support. However, since ODL SNMP Plugin that TSDR depends on does not support version 3, we (TSDR) will not have security support at this moment.
362 *** NetFlow, which cannot be configured with security so we recommend making sure it flows only over a secured management network.
365 *** Syslog, which cannot be configured with security so we recommend making sure it flows only over a secured management network.
367 === Support multiple data stores simultaneously at runtime
369 TSDR supports running multiple data stores simultaneously at runtim. For example, it is possible to configure TSDR to push log type of data into Cassandra data store, while pushing metrics type of data into HBase.
371 When you install one TSDR data store from karaf console, such as using feature:install odl-tsdr-hsqldb, a properties file will be generated under <Karaf-distribution-directory>/etc/. For example, when you install hsqldb, a file called tsdr-persistence-hsqldb.properties is generated under that directory.
373 By default, all the types of data are supported in the data store. For example, the default content of tsdr-persistence-hsqldb.properties is as follows:
375 metric-persistency=true
377 binary-persistency=true
379 When the user would like to use different data stores to support different types of data, he/she could enable or disable a particular type of data persistence in the data stores by configuring the properties file accordingly.
381 For example, if the user would like to store the log type of data in HBase, and store the metric and binary type of data in Cassandra, he/she needs to install both hbase and cassandra data stores from Karaf console. Then the user needs to modify the properties file under <Karaf-distribution-directory>/etc as follows:
383 * tsdr-persistence-hbase.properties
385 metric-persistency=false
387 binary-persistency=true
390 * tsdr-persistence-cassandra.properties
392 metric-psersistency=true
393 log-persistency=false
394 binary-persistency=false