2 Documentation netconf-restperfclient update performance test suite (clustered setup).
4 ... Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.
6 ... This program and the accompanying materials are made available under the
7 ... terms of the Eclipse Public License v1.0 which accompanies this distribution,
8 ... and is available at http://www.eclipse.org/legal/epl-v10.html
11 ... Perform given count of update operations on device data mounted onto a
12 ... netconf connector (using the netconf-testtool-restperfclient tool) and
13 ... see how much time it took. More exactly, it sends the data to a restconf
14 ... mountpoint of the netconf connector belonging to the device, which turns
15 ... out to turn the first request sent to a "create" request and the
16 ... remaining requests to "update" requests (due to how the testtool device
17 ... behavior is implemented).
19 ... The difference from the "single node" test suite (see
20 ... ../scale/performance.robot) is that the device is configured and the data
21 ... on it created using one node in the cluster and the update operations are
22 ... issued on a different node. This forces the cluster nodes to communicate
23 ... with each other about the data to be sent to the device.
24 Suite Setup Setup_Everything
25 Suite Teardown Teardown_Everything
26 Test Setup SetupUtils.Setup_Test_With_Logging_And_Fast_Failing
27 Test Teardown SetupUtils.Teardown_Test_Show_Bugs_And_Start_Fast_Failing_If_Test_Failed
28 Default Tags 1node netconf critical
30 Library RequestsLibrary
31 Library OperatingSystem
33 Library SSHLibrary timeout=10s
34 Resource ${CURDIR}/../../../libraries/KarafKeywords.robot
35 Resource ${CURDIR}/../../../libraries/NetconfKeywords.robot
36 Resource ${CURDIR}/../../../libraries/NexusKeywords.robot
37 Resource ${CURDIR}/../../../libraries/RestPerfClient.robot
38 Resource ${CURDIR}/../../../libraries/SetupUtils.robot
39 Resource ${CURDIR}/../../../libraries/Utils.robot
40 Variables ${CURDIR}/../../../variables/Variables.py
43 ${DEVICE_NAME} ${FIRST_TESTTOOL_PORT}-sim-device
44 ${REQUEST_COUNT} 16384
45 ${directory_with_crud_templates} ${CURDIR}/../../../variables/netconf/CRUD
46 ${DEVICE_DATA_CONNECT_TIMEOUT} 60s
50 [Documentation] Deploy and start test tool, then wait for all its devices to become online.
52 SSHLibrary.Switch_Connection ${testtool}
53 NetconfKeywords.Install_And_Start_Testtool device-count=1 debug=false schemas=${CURDIR}/../../../variables/netconf/CRUD/schemas
55 Configure_Device_On_Netconf
56 [Documentation] Configure the testtool device on Netconf connector, using node 1.
57 NetconfKeywords.Configure_Device_In_Netconf ${DEVICE_NAME} device_type=configure-via-topology session=node1
59 Wait_For_Device_To_Become_Connected
60 [Documentation] Wait until the device becomes available through Netconf on node 1.
61 NetconfKeywords.Wait_Device_Connected ${DEVICE_NAME} session=node1
63 Wait_For_Device_Data_To_Be_Seen
64 [Documentation] Wait until the device data show up at node 2.
65 BuiltIn.Wait_Until_Keyword_Succeeds ${DEVICE_DATA_CONNECT_TIMEOUT} 1s Check_Data_Present
68 [Documentation] Send some sample test data into the device through node 2 and check that the request went OK.
69 ${template_as_string}= BuiltIn.Set_Variable {'DEVICE_NAME': '${DEVICE_NAME}'}
70 TemplatedRequests.Post_As_Xml_Templated ${directory_with_crud_templates}${/}cars ${template_as_string} session=node2
73 [Documentation] Deploy and execute restperfclient, asking it to send the specified amount of requests to the netconf connector of the device through node 3.
74 ... The duration of this test case is the main performance metric.
75 ${url}= BuiltIn.Set_Variable /restconf/config/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount/car:cars
76 RestPerfClient.Invoke_Restperfclient ${TESTTOOL_DEVICE_TIMEOUT} ${url} ip=${ODL_SYSTEM_3_IP}
78 Check_For_Failed_Requests
79 [Documentation] Make sure there are no failed requests in the restperfclient log.
80 ... This is a separate test case to distinguish between restperfclient
81 ... failure and failed requests. Failed requests are rejected because
82 ... we don't want to test performance of ODL rejecting our requests.
83 ... If this test case fails, then the duration of Run_Restperfclient
84 ... cannot be trusted to show the real performance of the cluster.
85 ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out
86 BuiltIn.Should_Be_Equal '${result}' ''
87 ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed
88 BuiltIn.Should_Be_Equal '${result}' ''
89 ${result}= RestPerfClient.Grep_Restperfclient_Log Status code
90 BuiltIn.Should_Be_Equal '${result}' ''
93 [Documentation] Deconfigure the testtool device on Netconf connector using node 1.
94 [Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
95 RestPerfClient.Collect_From_Restperfclient
96 NetconfKeywords.Remove_Device_From_Netconf ${DEVICE_NAME} session=node1
100 [Documentation] Setup everything needed for the test cases.
101 # Calculate and set the value of the timeout
102 ${value}= BuiltIn.Evaluate ${REQUEST_COUNT}/10+20
103 Utils.Set_User_Configurable_Variable_Default TESTTOOL_DEVICE_TIMEOUT ${value} s
104 # Setup resources used by the suite.
105 SetupUtils.Setup_Utils_For_Setup_And_Teardown
106 NetconfKeywords.Setup_Netconf_Keywords
107 RestPerfClient.Setup_Restperfclient
108 # Connect to the tools system (testtool)
109 ${testtool}= SSHKeywords.Open_Connection_To_Tools_System
110 BuiltIn.Set_Suite_Variable ${testtool} ${testtool}
112 RequestsLibrary.Create_Session node1 http://${ODL_SYSTEM_1_IP}:${RESTCONFPORT} headers=${HEADERS_XML} auth=${AUTH}
113 RequestsLibrary.Create_Session node2 http://${ODL_SYSTEM_2_IP}:${RESTCONFPORT} headers=${HEADERS_XML} auth=${AUTH}
116 ${url}= Builtin.Set_Variable ${CONFIG_API}/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount
117 ${data}= TemplatedRequests.Get_As_Xml_From_Uri ${url} session=node2
118 BuiltIn.Should_Be_Equal_As_Strings ${data} <data xmlns="${ODL_NETCONF_NAMESPACE}"></data>
121 [Documentation] Teardown the test infrastructure, perform cleanup and release all resources.
122 RequestsLibrary.Delete_All_Sessions
123 RestPerfClient.Teardown_Restperfclient
124 SSHLibrary.Switch_Connection ${testtool}
125 BuiltIn.Run_Keyword_And_Ignore_Error NetconfKeywords.Stop_Testtool