OpenDaylight Performance Test Suite =================================== The performance tests in this suite use ODL's RESTCONF API to add/delete flows into/from ODL's configuration data store. The test suite is meant for manual performance testing, where a user can tests different ways/scale to add and delete flows. The test suite contains the following scripts: - inventory_crawler.py: Retrieves all nodes from either the config or operational data store and prints a flow summary to the console. Depending on the print level specified in a command line option, the summary can shows the overall number of flows in the network, the number of flows in each node, or detailed data for each flow - flow_config_blaster.py: Adds and deletes ("blasts") flows into ODL's config space. Command line options control the number of "blaster" threads, the number of blast cycles, the number of flows blasted in each cycle by each thread, etc. flow_config_blaster.py provides the FlowConfigBlaster class that is reusable in other tests, such as in flow_add_delete_test.py - flow_config_blaster_fle.py: "FlowConfigBlaster Floodlight Edition" - the same as flow_config_blaster, but for the Floodlight controller. - config_cleanup.py: Cleans up the config data store by deleting the entire inventory. - flow_add_delete_test.py: Adds/deletes ("blasts") flows into ODL's config space. Similar to the flow_config_blaster (in fact, the flow_config_blaster is used in this test), but has more advanced handling of the add/delete cycles. The test executes in three steps: 1. The specified number of flows is added in the 'add cycle' (uses flow_config_blaster to blast flows) 2. The network is polled for flow statistics from the network (using the inventory_crawler) to make sure that all flows have been properly programmed into the network and stats can properly read them 3. The flows are deleted in the flow cycle (either in 'bulk' using the config_cleanup script or one by one using the flow_config_blaster) Prerequisites: - requests - netaddr The Inventory Crawler: ====================== To see the command line options, type: > ./inventory_crawler --help usage: inventory_crawler.py [-h] [--odlhost ODLHOST] [--odlport ODLPORT] [--plevel PLEVEL] [--datastore {operational,config}] [--no-auth] [--auth] [--debug] optional arguments: -h, --help show this help message and exit --odlhost ODLHOST host where odl controller is running (default is 127.0.0.1) --odlport ODLPORT port on which odl's RESTCONF is listening (default is 8181) --plevel PLEVEL Print Level: 0 - Summary (stats only); 1 - Node names; 2 - Node details;3 - Flow details --datastore {operational,config} Which data store to crawl; default operational --no-auth Do not use authenticated access to REST (default) --auth Use authenticated access to REST (username: 'admin', password: 'admin'). --debug List nodes that have not provided proper statistics data Examples: --------- To show a summary of all flows shown in the network, type: > ./inventory_crawler.py --plevel=1 --auth NOTE: REST authentication is turned by default in ODL Helium. To use authenticated REST, you have to specify the '--auth' switch in the command line. To show a summary of all flows shown in the config data store, type: > ./inventory_crawler.py --plevel=1 --datastore=config --auth The Flow Config Blaster: ======================== To see the command line options, type: > ./flow_config_blaster.py --help usage: flow_config_blaster.py [-h] [--host HOST] [--port PORT] [--cycles CYCLES] [--threads THREADS] [--flows FLOWS] [--nodes NODES] [--delay DELAY] [--delete] [--no-delete] [--auth] [--startflow STARTFLOW] [--file FILE] Flow programming performance test: First adds and then deletes flows into the config tree, as specified by optional parameters. optional arguments: -h, --help show this help message and exit --host HOST Host where odl controller is running (default is 127.0.0.1) --port PORT Port on which odl's RESTCONF is listening (default is 8181) --cycles CYCLES Number of flow add/delete cycles; default 1. Both Flow Adds and Flow Deletes are performed in cycles. worker threads are started in each cycle and the cycle ends when all threads finish. Another cycle is started when the previous cycle finished. --threads THREADS Number of request worker threads to start in each cycle; default=1. Each thread will add/delete flows. --flows FLOWS Number of flows that will be added/deleted by each worker thread in each cycle; default 10 --nodes NODES Number of nodes if mininet is not connected; default=16. If mininet is connected, flows will be evenly distributed (programmed) into connected nodes. --delay DELAY Time (in seconds) to wait between the add and delete cycles; default=0 --delete Delete all added flows one by one, benchmark delete performance. --no-delete Do not perform the delete cycle. --auth Use the ODL default username/password 'admin'/'admin' to authenticate access to REST; default: no authentication --startflow STARTFLOW The starting Flow ID; default=0 --file FILE File from which to read the JSON flow template; default: no file, use a built in template. NOTE: The 'startflow' command line parameter is used with multiple flow_config_blasters blasting flows at the same ODL instance. With Python's GIL any given blaster can not use more than one CPU even when multiple blaster threads are specified. Therefore, multiple blaster processes must be used to test ODL's performance limits. The 'startflow' parameter gives each blaster process its own flow id space so that each injects unique flows into ODL's config data store. NOTE: You don't have to be connected to mininet (or another openflow network, for that matter) to use this script. If ODL is connected to an openflow network, flow_config_blaster will evenly distribute flows across the network. If ODL is not connected to a network, flows are only stored in the config data store (i.e. nodes that may connect at some point in the future are in effect "preconfigured"). The not-connected mode can be used to test the performance of the data store and the REST subsystems. The 'nodes' parameter determines how many nodes are used in the non-connected mode. Examples: --------- To put 5000 flows into ODL running on the same node as the script type: > ./flow_config_blaster.py --flows=5000 --auth --no-delete To use 5 threads to put 5000 flows into ODL running on the same node as the script type: > ./flow_config_blaster.py --threads=5 --flows=1000 --auth --no-delete NOTE: each thread will put 1000 flows, and all 5 threads will work simultaneously. To first put and then delete 5000 flows into ODL running on the same node as the script type: > ./flow_config_blaster.py --flows=5000 --auth To use 5 threads to first put and then delete 5000 flows into ODL running on the same node as the script type: > ./flow_config_blaster.py --threads=5 --flows=1000 --auth NOTE: 5 threads are used to both add and delete flows To use 5 threads to first put and then delete 5000 flows into ODL in 10 add/delete cycles type: > ./flow_config_blaster.py --threads=5 --flows=100 --cycles=10 --auth NOTE: 5 threads are used to both add and delete flows. NOTE: Both Add and Delete are performed in 10 cycles. 5 worker threads are started in each cycle and the cycle ends when all threads finish. Cycles are useful to determine performance degradation with increasing number of flows in the datastore and in the network. To put and then delete 1000 flows with nicira match and action extensions, type: >./flow_config_blaster.py --flows=1000 --auth --file=./nicira-ext-all.json NOTE: json for flow adds will be taken from the file 'nicira-ext-all.json' The Config Cleanup ================== To see the command line options, type: > ./config_cleanup.py --help usage: config_cleanup.py [-h] [--odlhost ODLHOST] [--odlport ODLPORT] [--no-auth] [--auth] Cleans up the config space optional arguments: -h, --help show this help message and exit --odlhost ODLHOST host where odl controller is running (default is 127.0.0.1) --odlport ODLPORT port on which odl's RESTCONF is listening (default is 8181) --no-auth Do not use authenticated access to REST (default) --auth Use authenticated access to REST (username: 'admin', password: 'admin'). The Flow Add/Delete Test ======================== To see the command line options, type: >./flow_add_delete_test.py --help usage: flow_add_delete_test.py [-h] [--host HOST] [--port PORT] [--flows FLOWS] [--cycles CYCLES] [--threads THREADS] [--nodes NODES] [--delay DELAY] [--timeout TIMEOUT] [--delete] [--bulk-delete] [--auth] [--startflow STARTFLOW] Flow programming performance test: First adds and then deletes flows into the config tree, as specified by optional parameters. optional arguments: -h, --help show this help message and exit --host HOST Host where odl controller is running (default is 127.0.0.1) --port PORT Port on which odl's RESTCONF is listening (default is 8181) --cycles CYCLES Number of flow add/delete cycles; default 1. Both Flow Adds and Flow Deletes are performed in cycles. worker threads are started in each cycle and the cycle ends when all threads finish. Another cycle is started when the previous cycle finished. --threads THREADS Number of request worker threads to start in each cycle; default=1. Each thread will add/delete flows. --flows FLOWS Number of flows that will be added/deleted by each worker thread in each cycle; default 10 --nodes NODES Number of nodes if mininet is not connected; default=16. If mininet is connected, flows will be evenly distributed (programmed) into connected nodes. --delay DELAY Time (seconds) to between inventory polls when waiting for stats to catch up; default=1 --timeout TIMEOUT The maximum time (seconds) to wait between the add and delete cycles; default=100 --delete Delete all added flows one by one, benchmark delete performance. --bulk-delete Delete all flows in bulk; default=False --auth Use authenticated access to REST (username: 'admin', password: 'admin'); default=False --startflow STARTFLOW The starting Flow ID; default=0 --file FILE File from which to read the JSON flow template; default: no file, use a built in template. Examples: --------- To put 5000 flows into ODL, then wait for stats to catch up and then delete the flows in bulk (using config_cleanup): > ./flow_add_delete_test.py --flows=5000 --auth --no-delete --bulk-delete