2 *NETCONF testtool is a set of standalone runnable jars that can:*
4 * Simulate NETCONF devices (suitable for scale testing)
5 * Stress/Performance test NETCONF devices
6 * Stress/Performance test RESTCONF devices
8 These jars are part of OpenDaylight's controller project and are built from the
9 NETCONF codebase in OpenDaylight.
11 TIP: Download testtool from OpenDaylight Nexus at: https://nexus.opendaylight.org/content/repositories/public/org/opendaylight/netconf/netconf-testtool/1.0.2-Beryllium-SR2/
13 *Nexus contains 3 executable tools:*
15 * executable.jar - device simulator
16 * stress.client.tar.gz - NETCONF stress/performance measuring tool
17 * perf-client.jar - RESTCONF stress/performance measuring tool
19 TIP: Each executable tool provides help. Just invoke +java -jar
20 <name-of-the-tool.jar> --help+
22 ==== NETCONF device simulator
24 NETCONF testtool (or NETCONF device simulator) is a tool that
26 * Simulates 1 or more NETCONF devices
27 * Is suitable for scale, performance or crud testing
28 * Uses core implementation of NETCONF server from OpenDaylight
29 * Generates configuration files for controller so that the OpenDaylight distribution (Karaf) can easily connect to all simulated devices
30 * Provides broad configuration options
31 * Can start a fully fledged MD-SAL datastore
32 * Supports notifications
34 ===== Building testtool
36 . Check out latest NETCONF repository from https://git.opendaylight.org/gerrit/#/admin/projects/netconf[git]
37 . Move into the `opendaylight/netconf/tools/netconf-testtool/` folder
38 . Build testtool using the `mvn clean install` command
40 ===== Downloading testtool
42 Netconf-testtool is now part of default maven build profile for controller and
43 can be also downloaded from nexus. The executable jar for testtool can be found at:
44 https://nexus.opendaylight.org/content/repositories/public/org/opendaylight/netconf/netconf-testtool/1.0.2-Beryllium-SR2/[nexus-artifacts]
46 ===== Running testtool
48 . After successfully building or downloading, move into the `opendaylight/netconf/tools/netconf-testtool/target/` folder and there is file `netconf-testtool-1.1.0-SNAPSHOT-executable.jar` (or if downloaded from nexus just take that jar file)
49 . Execute this file using, e.g.:
51 java -jar netconf-testtool-1.1.0-SNAPSHOT-executable.jar
53 This execution runs the testtool with default for all parameters and you should see this log output from the testtool :
55 10:31:08.206 [main] INFO o.o.c.n.t.t.NetconfDeviceSimulator - Starting 1, SSH simulated devices starting on port 17830
56 10:31:08.675 [main] INFO o.o.c.n.t.t.NetconfDeviceSimulator - All simulated devices started successfully from port 17830 to 17830
58 ====== Default Parameters
60 The default parameters for testtool are:
63 * Run 1 simulated device
64 * Device port is 17830
65 * YANG modules used by device are only: ietf-netconf-monitoring, ietf-yang-types, ietf-inet-types (these modules are required for device in order to support NETCONF monitoring and are included in the netconf-testtool)
66 * Connection timeout is set to 30 minutes (quite high, but when testing with 10000 devices it might take some time for all of them to fully establish a connection)
67 * Debug level is set to false
68 * No distribution is modified to connect automatically to the NETCONF testtool
70 ===== Verifying testtool
72 To verify that the simulated device is up and running, we can try to connect to
73 it using command line ssh tool. Execute this command to connect to the device:
75 ssh admin@localhost -p 17830 -s netconf
77 Just accept the server with yes (if required) and provide any password (testtool
78 accepts all users with all passwords). You should see the hello message sent by simulated device.
83 usage: netconf testool [-h] [--device-count DEVICES-COUNT] [--devices-per-port DEVICES-PER-PORT] [--schemas-dir SCHEMAS-DIR] [--notification-file NOTIFICATION-FILE]
84 [--initial-config-xml-file INITIAL-CONFIG-XML-FILE] [--starting-port STARTING-PORT] [--generate-config-connection-timeout GENERATE-CONFIG-CONNECTION-TIMEOUT]
85 [--generate-config-address GENERATE-CONFIG-ADDRESS] [--generate-configs-batch-size GENERATE-CONFIGS-BATCH-SIZE] [--distribution-folder DISTRO-FOLDER] [--ssh SSH] [--exi EXI]
86 [--debug DEBUG] [--md-sal MD-SAL]
88 NETCONF device simulator. Detailed info can be found at https://wiki.opendaylight.org/view/OpenDaylight_Controller:Netconf:Testtool#Building_testtool
91 -h, --help show this help message and exit
92 --device-count DEVICES-COUNT
93 Number of simulated netconf devices to spin. This is the number of actual ports open for the devices.
94 --devices-per-port DEVICES-PER-PORT
95 Amount of config files generated per port to spoof more devices then are actually running
96 --schemas-dir SCHEMAS-DIR
97 Directory containing yang schemas to describe simulated devices. Some schemas e.g. netconf monitoring and inet types are included by default
98 --notification-file NOTIFICATION-FILE
99 Xml file containing notifications that should be sent to clients after create subscription is called
100 --initial-config-xml-file INITIAL-CONFIG-XML-FILE
101 Xml file containing initial simulatted configuration to be returned via get-config rpc
102 --starting-port STARTING-PORT
103 First port for simulated device. Each other device will have previous+1 port number
104 --generate-config-connection-timeout GENERATE-CONFIG-CONNECTION-TIMEOUT
105 Timeout to be generated in initial config files
106 --generate-config-address GENERATE-CONFIG-ADDRESS
107 Address to be placed in generated configs
108 --generate-configs-batch-size GENERATE-CONFIGS-BATCH-SIZE
109 Number of connector configs per generated file
110 --distribution-folder DISTRO-FOLDER
111 Directory where the karaf distribution for controller is located
112 --ssh SSH Whether to use ssh for transport or just pure tcp
113 --exi EXI Whether to use exi to transport xml content
114 --debug DEBUG Whether to use debug log level instead of INFO
115 --md-sal MD-SAL Whether to use md-sal datastore instead of default simulated datastore.
118 ===== Supported operations
120 Testtool default simple datastore supported operations:
122 get-schema:: returns YANG schemas loaded from user specified directory,
123 edit-config:: always returns OK and stores the XML from the input in a local variable available for get-config and get RPC. Every edit-config replaces the previous data,
124 commit:: always returns OK, but does not actually commit the data,
125 get-config:: returns local XML stored by edit-config,
126 get:: returns local XML stored by edit-config with netconf-state subtree, but also supports filtering.
127 (un)lock:: returns always OK with no lock guarantee
128 create-subscription:: returns always OK and after the operation is triggered, provided NETCONF notifications (if any) are fed to the client. No filtering or stream recognition is supported.
130 Note: when operation="delete" is present in the payload for edit-config, it will wipe its local store to simulate the removal of data.
132 When using the MD-SAL datastore testtool behaves more like normal NETCONF server
133 and is suitable for crud testing. create-subscription is not supported when
134 testtool is running with the MD-SAL datastore.
136 ===== Notification support
138 Testtool supports notifications via the --notification-file switch. To trigger the notification feed, create-subscription operation has to be invoked.
139 The XML file provided should look like this example file:
142 <?xml version='1.0' encoding='UTF-8' standalone='yes'?>
145 <!-- Notifications are processed in the order they are defined in XML -->
147 <!-- Notification that is sent only once right after create-subscription is called -->
149 <!-- Content of each notification entry must contain the entire notification with event time. Event time can be hardcoded, or generated by testtool if XXXX is set as eventtime in this XML -->
151 <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
152 <eventTime>2011-01-04T12:30:46</eventTime>
153 <random-notification xmlns="http://www.opendaylight.org/netconf/event:1.0">
154 <random-content>single no delay</random-content>
155 </random-notification>
160 <!-- Repeated Notification that is sent 5 times with 2 second delay inbetween -->
162 <!-- Delay in seconds from previous notification -->
164 <!-- Number of times this notification should be repeated -->
167 <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
168 <eventTime>XXXX</eventTime>
169 <random-notification xmlns="http://www.opendaylight.org/netconf/event:1.0">
170 <random-content>scheduled 5 times 10 seconds each</random-content>
171 </random-notification>
176 <!-- Single notification that is sent only once right after the previous notification -->
180 <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
181 <eventTime>XXXX</eventTime>
182 <random-notification xmlns="http://www.opendaylight.org/netconf/event:1.0">
183 <random-content>single with delay</random-content>
184 </random-notification>
192 ===== Connecting testtool with controller Karaf distribution
194 ====== Auto connect to OpenDaylight
196 It is possible to make OpenDaylight auto connect to the simulated
197 devices spawned by testtool (so user does not have to post a configuration for
198 every NETCONF connector via RESTCONF). The testtool is able to modify the OpenDaylight
199 distribution to auto connect to the simulated devices after feature
200 +odl-netconf-connector-all+ is installed.
201 When running testtool, issue this command (just point the testool to the distribution:
203 java -jar netconf-testtool-1.1.0-SNAPSHOT-executable.jar --device-count 10 --distribution-folder ~/distribution-karaf-0.4.0-SNAPSHOT/ --debug true
205 With the distribution-folder parameter, the testtool will modify the distribution
206 to include configuration for netconf-connector to connect to all simulated devices.
207 So there is no need to spawn netconf-connectors via RESTCONF.
209 ====== Running testtool and OpenDaylight on different machines
211 The testtool binds by default to 0.0.0.0 so it should be accessible from remote
212 machines. However you need to set the parameter "generate-config-address"
213 (when using autoconnect) to the address of machine where testtool will be run
214 so OpenDaylight can connect. The default value is localhost.
216 ===== Executing operations via RESTCONF on a mounted simulated device
218 Simulated devices support basic RPCs for editing their config. This part shows how to edit data for simulated device via RESTCONF.
220 ====== Test YANG schema
222 The controller and RESTCONF assume that the data that can be manipulated for
223 mounted device is described by a YANG schema. For demonstration, we will define
229 namespace "urn:opendaylight:test";
232 revision "2014-10-17";
244 Save this schema in file called test@2014-10-17.yang and store it a directory called test-schemas/, e.g., your home folder.
246 ====== Editing data for simulated device
248 * Start the device with following command:
250 java -jar netconf-testtool-1.1.0-SNAPSHOT-executable.jar --device-count 10 --distribution-folder ~/distribution-karaf-0.4.0-SNAPSHOT/ --debug true --schemas-dir ~/test-schemas/
253 * Install odl-netconf-connector-all feature
254 * Install odl-restconf feature
255 * Check that you can see config data for simulated device by executing GET request to
257 http://localhost:8181/restconf/config/network-topology:network-topology/topology/topology-netconf/node/17830-sim-device/yang-ext:mount/
259 * The data should be just and empty data container
260 * Now execute edit-config request by executing a POST request to:
262 http://localhost:8181/restconf/config/network-topology:network-topology/topology/topology-netconf/node/17830-sim-device/yang-ext:mount
266 Accept application/xml
267 Content-Type application/xml
272 <cont xmlns="urn:opendaylight:test">
277 * Check that you can see modified config data for simulated device by executing GET request to
279 http://localhost:8181/restconf/config/network-topology:network-topology/topology/topology-netconf/node/17830-sim-device/yang-ext:mount/
281 * Check that you can see the same modified data in operational for simulated device by executing GET request to
283 http://localhost:8181/restconf/operational/network-topology:network-topology/topology/topology-netconf/node/17830-sim-device/yang-ext:mount/
285 WARNING: Data will be mirrored in operational datastore only when using the default
290 ====== Slow creation of devices on virtual machines
292 When testtool seems to take unusually long time to create the devices use this flag when running it:
294 -Dorg.apache.sshd.registerBouncyCastle=false
296 ====== Too many files open
298 When testtool or OpenDaylight starts to fail with TooManyFilesOpen exception, you need to increase the limit of open files in your OS. To find out the limit in linux execute:
302 Example sufficient configuration in linux:
305 core file size (blocks, -c) 0
306 data seg size (kbytes, -d) unlimited
307 scheduling priority (-e) 0
308 file size (blocks, -f) unlimited
309 pending signals (-i) 63338
310 max locked memory (kbytes, -l) 64
311 max memory size (kbytes, -m) unlimited
312 open files (-n) 500000
313 pipe size (512 bytes, -p) 8
314 POSIX message queues (bytes, -q) 819200
315 real-time priority (-r) 0
316 stack size (kbytes, -s) 8192
317 cpu time (seconds, -t) unlimited
318 max user processes (-u) 63338
319 virtual memory (kbytes, -v) unlimited
320 file locks (-x) unlimited
323 To set these limits edit file: /etc/security/limits.conf, for example:
328 root hard nofile 500000
329 root soft nofile 500000
334 The testtool might end unexpectedly with a simple message: "Killed". This means
335 that the OS killed the tool due to too much memory consumed or too many threads
336 spawned. To find out the reason on linux you can use following command:
338 dmesg | egrep -i -B100 'killed process'
340 Also take a look at this file: /proc/sys/kernel/threads-max. It limits the
341 number of threads spawned by a process. Sufficient (but probably much more than
342 enough) value is, e.g., 126676
344 ==== NETCONF stress/performance measuring tool
345 This is basically a NETCONF client that puts NETCONF servers under
346 heavy load of NETCONF RPCs and measures the time until a configurable
347 amount of them is processed.
350 TODO add a guide on how to do this with OpenDaylight
353 ==== RESTCONF stress-performance measuring tool
354 Very similar to NETCONF stress tool with the difference of using
355 RESTCONF protocol instead of NETCONF.
358 TODO add a guide on how to do this with OpenDaylight