1 OpenDaylight Performance Test Suite
2 ===================================
4 The performance tests in this suite use ODL's RESTCONF API to add/delete
5 flows into/from ODL's configuration data store. The test suite is meant for
6 manual performance testing, where a user can tests different ways/scale to
9 The test suite contains the following scripts:
11 - inventory_crawler.py:
12 Retrieves all nodes from either the config or operational data store
13 and prints a flow summary to the console. Depending on the print level
14 specified in a command line option, the summary can shows the overall
15 number of flows in the network, the number of flows in each node, or
16 detailed data for each flow
18 - flow_config_blaster.py:
19 Adds and deletes ("blasts") flows into ODL's config space. Command line
20 options control the number of "blaster" threads, the number of blast
21 cycles, the number of flows blasted in each cycle by each thread, etc.
22 flow_config_blaster.py provides the FlowConfigBlaster class that is
23 reusable in other tests, such as in flow_add_delete_test.py
25 - flow_config_blaster_fle.py:
26 "FlowConfigBlaster Floodlight Edition" - the same as flow_config_blaster,
27 but for the Floodlight controller.
30 Cleans up the config data store by deleting the entire inventory.
32 - flow_add_delete_test.py:
33 Adds/deletes ("blasts") flows into ODL's config space. Similar to the
34 flow_config_blaster (in fact, the flow_config_blaster is used in this
35 test), but has more advanced handling of the add/delete cycles. The
36 test executes in three steps:
37 1. The specified number of flows is added in the 'add cycle' (uses
38 flow_config_blaster to blast flows)
39 2. The network is polled for flow statistics from the network
40 (using the inventory_crawler) to make sure that all flows have been
41 properly programmed into the network and stats can properly read them
42 3. The flows are deleted in the flow cycle (either in 'bulk' using the
43 config_cleanup script or one by one using the flow_config_blaster)
50 The Inventory Crawler:
51 ======================
52 To see the command line options, type:
53 > ./inventory_crawler --help
55 usage: inventory_crawler.py [-h] [--odlhost ODLHOST] [--odlport ODLPORT]
57 [--datastore {operational,config}] [--no-auth]
61 -h, --help show this help message and exit
62 --odlhost ODLHOST host where odl controller is running (default is
64 --odlport ODLPORT port on which odl's RESTCONF is listening (default is
66 --plevel PLEVEL Print Level: 0 - Summary (stats only); 1 - Node names;
67 2 - Node details;3 - Flow details
68 --datastore {operational,config}
69 Which data store to crawl; default operational
70 --no-auth Do not use authenticated access to REST (default)
71 --auth Use authenticated access to REST (username: 'admin',
73 --debug List nodes that have not provided proper statistics
78 To show a summary of all flows shown in the network, type:
79 > ./inventory_crawler.py --plevel=1 --auth
81 NOTE: REST authentication is turned by default in ODL Helium. To use
82 authenticated REST, you have to specify the '--auth' switch in the command
85 To show a summary of all flows shown in the config data store, type:
86 > ./inventory_crawler.py --plevel=1 --datastore=config --auth
90 The Flow Config Blaster:
91 ========================
92 To see the command line options, type:
93 > ./flow_config_blaster.py --help
95 usage: flow_config_blaster.py [-h] [--host HOST] [--port PORT]
96 [--cycles CYCLES] [--threads THREADS]
97 [--flows FLOWS] [--nodes NODES] [--delay DELAY]
98 [--delete] [--no-delete] [--auth]
99 [--startflow STARTFLOW] [--file FILE]
101 Flow programming performance test: First adds and then deletes flows into the
102 config tree, as specified by optional parameters.
105 -h, --help show this help message and exit
106 --host HOST Host where odl controller is running (default is
108 --port PORT Port on which odl's RESTCONF is listening (default is
110 --cycles CYCLES Number of flow add/delete cycles; default 1. Both Flow
111 Adds and Flow Deletes are performed in cycles.
112 <THREADS> worker threads are started in each cycle and
113 the cycle ends when all threads finish. Another cycle
114 is started when the previous cycle finished.
115 --threads THREADS Number of request worker threads to start in each
116 cycle; default=1. Each thread will add/delete <FLOWS>
118 --flows FLOWS Number of flows that will be added/deleted by each
119 worker thread in each cycle; default 10
120 --nodes NODES Number of nodes if mininet is not connected;
121 default=16. If mininet is connected, flows will be
122 evenly distributed (programmed) into connected nodes.
123 --delay DELAY Time (in seconds) to wait between the add and delete
125 --delete Delete all added flows one by one, benchmark delete
127 --no-delete Do not perform the delete cycle.
128 --auth Use the ODL default username/password 'admin'/'admin'
129 to authenticate access to REST; default: no
131 --startflow STARTFLOW
132 The starting Flow ID; default=0
133 --file FILE File from which to read the JSON flow template;
134 default: no file, use a built in template.
136 NOTE: The 'startflow' command line parameter is used with multiple
137 flow_config_blasters blasting flows at the same ODL instance. With Python's
138 GIL any given blaster can not use more than one CPU even when multiple blaster
139 threads are specified. Therefore, multiple blaster processes must be used to
140 test ODL's performance limits. The 'startflow' parameter gives each blaster
141 process its own flow id space so that each injects unique flows into ODL's
144 NOTE: You don't have to be connected to mininet (or another openflow network,
145 for that matter) to use this script. If ODL is connected to an openflow
146 network, flow_config_blaster will evenly distribute flows across the network.
147 If ODL is not connected to a network, flows are only stored in the config
148 data store (i.e. nodes that may connect at some point in the future are in
149 effect "preconfigured"). The not-connected mode can be used to test the
150 performance of the data store and the REST subsystems. The 'nodes' parameter
151 determines how many nodes are used in the non-connected mode.
155 To put 5000 flows into ODL running on the same node as the script type:
156 > ./flow_config_blaster.py --flows=5000 --auth --no-delete
158 To use 5 threads to put 5000 flows into ODL running on the same node as
160 > ./flow_config_blaster.py --threads=5 --flows=1000 --auth --no-delete
162 NOTE: each thread will put 1000 flows, and all 5 threads will work
165 To first put and then delete 5000 flows into ODL running on the same node
167 > ./flow_config_blaster.py --flows=5000 --auth
169 To use 5 threads to first put and then delete 5000 flows into ODL running
170 on the same node as the script type:
171 > ./flow_config_blaster.py --threads=5 --flows=1000 --auth
173 NOTE: 5 threads are used to both add and delete flows
175 To use 5 threads to first put and then delete 5000 flows into ODL in 10
176 add/delete cycles type:
177 > ./flow_config_blaster.py --threads=5 --flows=100 --cycles=10 --auth
179 NOTE: 5 threads are used to both add and delete flows.
181 NOTE: Both Add and Delete are performed in 10 cycles. 5 worker threads
182 are started in each cycle and the cycle ends when all threads finish.
183 Cycles are useful to determine performance degradation with increasing
184 number of flows in the datastore and in the network.
186 To put and then delete 1000 flows with nicira match and action extensions,
188 >./flow_config_blaster.py --flows=1000 --auth --file=./nicira-ext-all.json
190 NOTE: json for flow adds will be taken from the file 'nicira-ext-all.json'
196 To see the command line options, type:
197 > ./config_cleanup.py --help
199 usage: config_cleanup.py [-h] [--odlhost ODLHOST] [--odlport ODLPORT]
202 Cleans up the config space
205 -h, --help show this help message and exit
206 --odlhost ODLHOST host where odl controller is running (default is
208 --odlport ODLPORT port on which odl's RESTCONF is listening (default is
210 --no-auth Do not use authenticated access to REST (default)
211 --auth Use authenticated access to REST (username: 'admin',
216 The Flow Add/Delete Test
217 ========================
218 To see the command line options, type:
219 >./flow_add_delete_test.py --help
221 usage: flow_add_delete_test.py [-h] [--host HOST] [--port PORT]
222 [--flows FLOWS] [--cycles CYCLES]
223 [--threads THREADS] [--nodes NODES]
224 [--delay DELAY] [--timeout TIMEOUT] [--delete]
225 [--bulk-delete] [--auth]
226 [--startflow STARTFLOW]
228 Flow programming performance test: First adds and then deletes flows into the
229 config tree, as specified by optional parameters.
232 -h, --help show this help message and exit
233 --host HOST Host where odl controller is running (default is
235 --port PORT Port on which odl's RESTCONF is listening (default is
237 --cycles CYCLES Number of flow add/delete cycles; default 1. Both Flow
238 Adds and Flow Deletes are performed in cycles.
239 <THREADS> worker threads are started in each cycle and
240 the cycle ends when all threads finish. Another cycle
241 is started when the previous cycle finished.
242 --threads THREADS Number of request worker threads to start in each
243 cycle; default=1. Each thread will add/delete <FLOWS>
245 --flows FLOWS Number of flows that will be added/deleted by each
246 worker thread in each cycle; default 10
247 --nodes NODES Number of nodes if mininet is not connected;
248 default=16. If mininet is connected, flows will be
249 evenly distributed (programmed) into connected nodes.
250 --delay DELAY Time (seconds) to between inventory polls when waiting
251 for stats to catch up; default=1
252 --timeout TIMEOUT The maximum time (seconds) to wait between the add and
253 delete cycles; default=100
254 --delete Delete all added flows one by one, benchmark delete
256 --bulk-delete Delete all flows in bulk; default=False
257 --auth Use authenticated access to REST (username: 'admin',
258 password: 'admin'); default=False
259 --startflow STARTFLOW
260 The starting Flow ID; default=0
261 --file FILE File from which to read the JSON flow template;
262 default: no file, use a built in template.
266 To put 5000 flows into ODL, then wait for stats to catch up and then delete
267 the flows in bulk (using config_cleanup):
268 > ./flow_add_delete_test.py --flows=5000 --auth --no-delete --bulk-delete