14 **NETCONF testtool is a set of standalone runnable jars that can:**
16 - Simulate NETCONF devices (suitable for scale testing)
18 - Stress/Performance test NETCONF devices
20 - Stress/Performance test RESTCONF devices
22 These jars are part of OpenDaylight’s controller project and are built
23 from the NETCONF codebase in OpenDaylight.
25 **Nexus contains 3 executable tools:**
27 - executable.jar - device simulator
29 - stress.client.tar.gz - NETCONF stress/performance measuring tool
31 - perf-client.jar - RESTCONF stress/performance measuring tool
35 Each executable tool provides help. Just invoke ``java -jar
36 <name-of-the-tool.jar> --help``
38 NETCONF device simulator
39 ~~~~~~~~~~~~~~~~~~~~~~~~
41 NETCONF testtool (or NETCONF device simulator) is a tool that
43 - Simulates 1 or more NETCONF devices
45 - Is suitable for scale, performance or crud testing
47 - Uses core implementation of NETCONF server from OpenDaylight
49 - Allows for automatic sending of connector configurations through RESTCONF to the controller. This makes it easy for
50 the OpenDaylight distribution (Karaf) to connect to all simulated devices once the simulator starts.
52 - Provides broad configuration options
54 - Can start a fully fledged MD-SAL datastore
56 - Supports notifications
61 1. Check out latest NETCONF repository from
62 `git <https://git.opendaylight.org/gerrit/admin/repos/netconf>`__
64 2. Move into the ``opendaylight/netconf/tools/netconf-testtool/`` folder
66 3. Build testtool using the ``mvn clean install`` command
71 Netconf-testtool is now part of default maven build profile for
72 controller and can be also downloaded from nexus. The executable jars for
73 testtool can be found by release at this parent directory:
74 `nexus-artifacts <https://nexus.opendaylight.org/content/repositories/public/org/opendaylight/netconf/netconf-testtool/>`__
79 1. After successfully building or downloading, move into the
80 ``opendaylight/netconf/tools/netconf-testtool/target/`` folder and
81 there is file ``netconf-testtool-[VERSION]-executable.jar`` (or
82 if downloaded from nexus just take that \*-executable.jar file)
84 2. Execute this file using, e.g.:
88 java -jar netconf-testtool-[VERSION]-executable.jar
90 This execution runs the testtool with default for all parameters and
91 you should see this log output from the testtool :
95 10:31:08.206 [main] INFO o.o.c.n.t.t.NetconfDeviceSimulator - Starting 1, SSH simulated devices starting on port 17830
96 10:31:08.675 [main] INFO o.o.c.n.t.t.NetconfDeviceSimulator - All simulated devices started successfully from port 17830 to 17830
101 The default parameters for testtool are:
105 - Run 1 simulated device
107 - Device port is 17830
109 - YANG modules used by device are only: ietf-netconf-monitoring,
110 ietf-yang-types, ietf-inet-types (these modules are required for
111 device in order to support NETCONF monitoring and are included in the
114 - Connection timeout is set to 30 minutes (quite high, but when testing
115 with 10000 devices it might take some time for all of them to fully
116 establish a connection)
118 - Debug level is set to false
120 - Other default parameters can be seen with ``--help`` option
125 To verify that the simulated device is up and running, we can try to
126 connect to it using command line ssh tool. Execute this command to
127 connect to the device:
131 ssh admin@localhost -p 17830 -s netconf
133 Just accept the server with yes (if required) and provide any password
134 (testtool accepts all users with all passwords). You should see the
135 hello message sent by simulated device.
142 usage: netconf testtool [-h] [--edit-content EDIT-CONTENT] [--async-requests {true,false}]
143 [--thread-amount THREAD-AMOUNT] [--throttle THROTTLE]
144 [--controller-auth-username CONTROLLER-AUTH-USERNAME]
145 [--controller-auth-password CONTROLLER-AUTH-PASSWORD] [--controller-ip CONTROLLER-IP]
146 [--controller-port CONTROLLER-PORT] [--device-count DEVICES-COUNT]
147 [--devices-per-port DEVICES-PER-PORT] [--schemas-dir SCHEMAS-DIR]
148 [--notification-file NOTIFICATION-FILE]
149 [--initial-config-xml-file INITIAL-CONFIG-XML-FILE] [--starting-port STARTING-PORT]
150 [--generate-config-connection-timeout GENERATE-CONFIG-CONNECTION-TIMEOUT]
151 [--generate-config-address GENERATE-CONFIG-ADDRESS]
152 [--generate-configs-batch-size GENERATE-CONFIGS-BATCH-SIZE]
153 [--distribution-folder DISTRO-FOLDER] [--ssh {true,false}] [--exi {true,false}]
154 [--debug {true,false}] [--md-sal {true,false}] [--time-out TIME-OUT] [--ip IP]
155 [--thread-pool-size THREAD-POOL-SIZE] [--rpc-config RPC-CONFIG]
160 -h, --help show this help message and exit
161 --edit-content EDIT-CONTENT
162 --async-requests {true,false}
164 --thread-amount THREAD-AMOUNT
165 The number of threads to use for configuring devices. (default: 1)
166 --throttle THROTTLE Maximum amount of async requests that can be open at a time, with mutltiple threads
167 this gets divided among all threads (default: 5000)
168 --controller-auth-username CONTROLLER-AUTH-USERNAME
169 Username for HTTP basic authentication to destination controller. (default: admin)
170 --controller-auth-password CONTROLLER-AUTH-PASSWORD
171 Password for HTTP basic authentication to destination controller. (default: admin)
172 --controller-ip CONTROLLER-IP
173 Ip of controller if available it will be used for spawning netconf connectors via
174 topology configuration as a part of URI(http://<controller-ip>:<controller-
175 port>/rests/data/...) otherwise it will just start simulated devices and skip the
176 execution of PATCH requests
177 --controller-port CONTROLLER-PORT
178 Port of controller if available it will be used for spawning netconf connectors via
179 topology configuration as a part of URI(http://<controller-ip>:<controller-
180 port>/rests/data/...) otherwise it will just start simulated devices and skip the
181 execution of PATCH requests
182 --device-count DEVICES-COUNT
183 Number of simulated netconf devices to spin. This is the number of actual ports open
184 for the devices. (default: 1)
185 --devices-per-port DEVICES-PER-PORT
186 Amount of config files generated per port to spoof more devices than are actually
188 --schemas-dir SCHEMAS-DIR
189 Directory containing yang schemas to describe simulated devices. Some schemas e.g.
190 netconf monitoring and inet types are included by default
191 --notification-file NOTIFICATION-FILE
192 Xml file containing notifications that should be sent to clients after create
193 subscription is called
194 --initial-config-xml-file INITIAL-CONFIG-XML-FILE
195 Xml file containing initial simulatted configuration to be returned via get-config rpc
196 --starting-port STARTING-PORT
197 First port for simulated device. Each other device will have previous+1 port number
199 --generate-config-connection-timeout GENERATE-CONFIG-CONNECTION-TIMEOUT
200 Timeout to be generated in initial config files (default: 1800000)
201 --generate-config-address GENERATE-CONFIG-ADDRESS
202 Address to be placed in generated configs (default: 127.0.0.1)
203 --generate-configs-batch-size GENERATE-CONFIGS-BATCH-SIZE
204 Number of connector configs per generated file (default: 1)
205 --distribution-folder DISTRO-FOLDER
206 Directory where the karaf distribution for controller is located
207 --ssh {true,false} Whether to use ssh for transport or just pure tcp (default: true)
208 --exi {true,false} Whether to use exi to transport xml content (default: true)
209 --debug {true,false} Whether to use debug log level instead of INFO (default: false)
210 --md-sal {true,false} Whether to use md-sal datastore instead of default simulated datastore. (default: false)
211 --time-out TIME-OUT the maximum time in seconds for executing each PATCH request (default: 20)
212 --ip IP Ip address which will be used for creating a socket address.It can either be a machine
213 name, such as java.sun.com, or a textual representation of its IP address. (default:
215 --thread-pool-size THREAD-POOL-SIZE
216 The number of threads to keep in the pool, when creating a device simulator. Even if
217 they are idle. (default: 8)
218 --rpc-config RPC-CONFIG
219 Rpc config file. It can be used to define custom rpc behavior, or override the default
220 one.Usable for testing buggy device behavior.
226 Testtool default simple datastore supported operations:
229 returns YANG schemas loaded from user specified directory,
232 always returns OK and stores the XML from the input in a local
233 variable available for get-config and get RPC. Every edit-config
234 replaces the previous data,
237 always returns OK, but does not actually commit the data,
240 returns local XML stored by edit-config,
243 returns local XML stored by edit-config with netconf-state subtree,
244 but also supports filtering.
247 returns always OK with no lock guarantee
250 returns always OK and after the operation is triggered, provided
251 NETCONF notifications (if any) are fed to the client. No filtering
252 or stream recognition is supported.
254 Note: when operation="delete" is present in the payload for edit-config,
255 it will wipe its local store to simulate the removal of data.
257 When using the MD-SAL datastore testtool behaves more like normal
258 NETCONF server and is suitable for crud testing. create-subscription is
259 not supported when testtool is running with the MD-SAL datastore.
264 Testtool supports notifications via the ``--notification-file`` switch. To
265 trigger the notification feed, create-subscription operation has to be
266 invoked. The XML file provided should look like this example file:
270 <?xml version='1.0' encoding='UTF-8' standalone='yes'?>
273 <!-- Notifications are processed in the order they are defined in XML -->
275 <!-- Notification that is sent only once right after create-subscription is called -->
277 <!-- Content of each notification entry must contain the entire notification with event time. Event time can be hardcoded, or generated by testtool if XXXX is set as eventtime in this XML -->
279 <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
280 <eventTime>2011-01-04T12:30:46</eventTime>
281 <random-notification xmlns="http://www.opendaylight.org/netconf/event:1.0">
282 <random-content>single no delay</random-content>
283 </random-notification>
288 <!-- Repeated Notification that is sent 5 times with 2 second delay inbetween -->
290 <!-- Delay in seconds from previous notification -->
292 <!-- Number of times this notification should be repeated -->
295 <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
296 <eventTime>XXXX</eventTime>
297 <random-notification xmlns="http://www.opendaylight.org/netconf/event:1.0">
298 <random-content>scheduled 5 times 10 seconds each</random-content>
299 </random-notification>
304 <!-- Single notification that is sent only once right after the previous notification -->
308 <notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
309 <eventTime>XXXX</eventTime>
310 <random-notification xmlns="http://www.opendaylight.org/netconf/event:1.0">
311 <random-content>single with delay</random-content>
312 </random-notification>
319 Connecting testtool with controller Karaf distribution
320 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
322 Auto connect to OpenDaylight
323 ''''''''''''''''''''''''''''
325 You can set up the testtool to automatically connect to the Controller. When you provide
326 the ``--controller-ip`` and ``--controller-port`` parameters, the testtool will send a PATCH request to the Controller
327 netconf-topology with the device connector configuration.
331 java -jar netconf-testtool-[VERSION]-executable.jar --device-count 10 --controller-ip 127.0.0.1 --controller-port 8181 --debug true
334 Running testtool and OpenDaylight on different machines
335 '''''''''''''''''''''''''''''''''''''''''''''''''''''''
337 The testtool binds by default to 0.0.0.0 so it should be accessible from
338 remote machines. However you need to set the parameter
339 "generate-config-address" (when using autoconnect) to the address of
340 machine where testtool will be run so OpenDaylight can connect. The
341 default value is localhost.
343 Executing operations via RESTCONF on a mounted simulated device
344 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
346 Simulated devices support basic RPCs for editing their config. This part
347 shows how to edit data for simulated device via RESTCONF.
352 The controller and RESTCONF assume that the data that can be manipulated
353 for mounted device is described by a YANG schema. For demonstration, we
354 will define a simple YANG model:
360 namespace "urn:opendaylight:test";
362 revision "2014-10-17";
371 Save this schema in file called ``test@2014-10-17.yang`` and store it a
372 directory called test-schemas/, e.g., your home folder.
374 Editing data for simulated device
375 '''''''''''''''''''''''''''''''''
379 - Install odl-netconf-topology and odl-restconf-nb features
381 - Start the device with following command:
385 java -jar netconf-testtool-[VERSION]-executable.jar --controller-ip 127.0.0.1 --controller-port 8181 --debug true --schemas-dir ~/test-schemas/
387 - Check that you can see config data for simulated device by executing GET request to:
391 http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount?content=config
393 - The data should be just and empty data container
395 - Now execute edit-config request by executing a POST request to:
399 http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount
405 Accept application/xml
406 Content-Type application/xml
412 <cont xmlns="urn:opendaylight:test">
415 - Response should be 201 with empty body
417 - Check that you can see modified config data for simulated device by
418 executing GET request to
422 http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount?content=config
424 - Check that you can see the same modified data in operational for
425 simulated device by executing GET request to
429 http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount?content=nonconfig
433 Data will be mirrored in operational datastore only when using the
434 default simple datastore.
437 Testing User defined RPC
438 ^^^^^^^^^^^^^^^^^^^^^^^^
440 The NETCONF test-tool allows using custom RPC. Custom RPC needs to be defined in yang model provide to test-tool along
441 with parameter ``--schemas-dir``.
443 The input and output of the custom RPC should be provided with ``--rpc-config`` parameter as a path to the file containing
444 definition of input and output. The format of the custom RPC file is xml as shown below.
446 Example YANG model file (stored in folder ~/test-schemas/):
451 namespace "urn:example-ops:reboot";
454 import ietf-yang-types {
458 revision "2016-07-07" {
459 description "Initial version.";
460 reference "example document.";
464 description "Reboot operation.";
470 description "Delay in seconds.";
474 description "Log message.";
481 Example payload (RPC config file ~/tmp/customrpc.xml):
488 <reboot xmlns="urn:example-ops:reboot">
490 <message>message</message>
494 <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
501 Start the device with following command:
505 java -jar netconf-testtool-[VERSION]-executable.jar --controller-ip 127.0.0.1 --controller-port 8181 --schemas-dir ~/test-schemas/ --rpc-config ~/tmp/customrpc.xml --debug=true
513 POST http://localhost:8181/rests/operations/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount/example-ops:reboot
519 <?xml version="1.0" encoding="UTF-8" ?>
520 <input xmlns="urn:example-ops:reboot">
522 <message>message</message>
525 If successful the response should be 204.
531 A working example of user defined RPC can be found in TestToolTest.java class of the tools[netconf-testtool] project.
537 Slow creation of devices on virtual machines
538 ''''''''''''''''''''''''''''''''''''''''''''
540 When testtool seems to take unusually long time to create the devices
541 use this flag when running it:
545 -Dorg.apache.sshd.registerBouncyCastle=false
550 When testtool or OpenDaylight starts to fail with TooManyFilesOpen
551 exception, you need to increase the limit of open files in your OS. To
552 find out the limit in linux execute:
558 Example sufficient configuration in linux:
562 core file size (blocks, -c) 0
563 data seg size (kbytes, -d) unlimited
564 scheduling priority (-e) 0
565 file size (blocks, -f) unlimited
566 pending signals (-i) 63338
567 max locked memory (kbytes, -l) 64
568 max memory size (kbytes, -m) unlimited
569 open files (-n) 500000
570 pipe size (512 bytes, -p) 8
571 POSIX message queues (bytes, -q) 819200
572 real-time priority (-r) 0
573 stack size (kbytes, -s) 8192
574 cpu time (seconds, -t) unlimited
575 max user processes (-u) 63338
576 virtual memory (kbytes, -v) unlimited
577 file locks (-x) unlimited
579 To set these limits edit file: /etc/security/limits.conf, for example:
585 root hard nofile 500000
586 root soft nofile 500000
591 The testtool might end unexpectedly with a simple message: "Killed".
592 This means that the OS killed the tool due to too much memory consumed
593 or too many threads spawned. To find out the reason on linux you can use
598 dmesg | egrep -i -B100 'killed process'
600 Also take a look at this file: /proc/sys/kernel/threads-max. It limits
601 the number of threads spawned by a process. Sufficient (but probably
602 much more than enough) value is, e.g., 126676