Library Collections
Library DateTime
Library RequestsLibrary
+Library SSHLibrary
Resource NetconfViaRestconf.robot
Resource NexusKeywords.robot
Resource SSHKeywords.robot
--- /dev/null
+*** Settings ***
+Documentation RestPerfClient handling singleton resource.
+...
+... Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.
+...
+... This program and the accompanying materials are made available under the
+... terms of the Eclipse Public License v1.0 which accompanies this distribution,
+... and is available at http://www.eclipse.org/legal/epl-v10.html
+...
+... This singleton manages RestPerfClient invocation, tracks the log file
+... produced by the invocation, allows the test suite to easily search this
+... log file and collect it once done.
+...
+... TODO: Currently only one RestPerfClient invocation running at a time is
+... supported. Support for multiple concurrently running RestPerfClient
+... invocations might be needed for example when performance testing cluster
+... nodes. However no such suites are planned for now.
+...
+... FIXME: There may be suites which want to use this Resource without
+... NetconfKeywords, in which case NexusKeywords will not be initialized
+... and Setup_Restperfclient will fail. Fixing this problem will require
+... updating NexusKeywords initialization (which may break other suites)
+... and currently all suites using this use also NetconfKeywords so this
+... was postponed. Workaround for the problem: Initialize NexusKeywords
+... manually before initializing this resource.
+Library SSHLibrary
+Resource ${CURDIR}/NexusKeywords.robot
+Resource ${CURDIR}/SSHKeywords.robot
+Resource ${CURDIR}/Utils.robot
+
+*** Variables ***
+${RestPerfClient__restperfclientlog} ${EMPTY}
+
+*** Keywords ***
+Setup_Restperfclient
+ [Documentation] Deploy RestPerfClient and determine the Java command to use to call it.
+ ... Open a SSH connection through which the RestPerfClient will be
+ ... invoked, deploy RestPerfClient and the data files it needs to do
+ ... its work and initialize the internal state for the remaining
+ ... keywords.
+ ${connection}= SSHKeywords.Open_Connection_To_Tools_System
+ BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclient} ${connection}
+ SSHLibrary.Put_File ${CURDIR}/../variables/netconf/RestPerfClient/request1.json
+ ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client
+ ${prefix}= NexusKeywords.Compose_Full_Java_Command -Xmx1G -XX:MaxPermSize=256M -jar ${filename}
+ BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclient_invocation_command_prefix} ${prefix}
+
+Invoke_Restperfclient
+ [Arguments] ${timeout} ${url} ${testcase}=${EMPTY} ${ip}=${ODL_SYSTEM_IP} ${port}=${RESTCONFPORT} ${count}=${REQUEST_COUNT}
+ ... ${async}=false ${user}=${ODL_RESTCONF_USER} ${password}=${ODL_RESTCONF_PASSWORD}
+ [Documentation] Invoke RestPerfClient on the specified URL with the specified timeout.
+ ... Assemble the RestPerfClient invocation commad, setup the specified
+ ... timeout for the SSH connection, invoke the assembled command and
+ ... then check that RestPerfClient finished its run correctly.
+ ${logname}= Utils.Get_Log_File_Name restperfclient ${testcase}
+ BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclientlog} ${logname}
+ ${options}= BuiltIn.Set_Variable --ip ${ip} --port ${port} --edits ${count}
+ ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json --async-requests ${async}
+ ${options}= BuiltIn.Set_Variable ${options} --auth ${user} ${password}
+ ${options}= BuiltIn.Set_Variable ${options} --destination ${url}
+ ${command}= BuiltIn.Set_Variable ${RestPerfClient__restperfclient_invocation_command_prefix} ${options}
+ BuiltIn.Log Running restperfclient: ${command}
+ SSHLibrary.Switch_Connection ${RestPerfClient__restperfclient}
+ SSHLibrary.Set_Client_Configuration timeout=${timeout}
+ Set_Known_Bug_Id 5413
+ Execute_Command_Passes ${command} >${RestPerfClient__restperfclientlog} 2>&1
+ Set_Unknown_Bug_Id
+ ${result}= Grep_Restperfclient_Log FINISHED. Execution time:
+ BuiltIn.Should_Not_Be_Equal '${result}' ''
+
+Grep_Restperfclient_Log
+ [Documentation] Search for the specified string in the log file produced by latest invocation of RestPerfClient
+ [Arguments] ${pattern}
+ BuiltIn.Should_Not_Be_Equal '${RestPerfClient__restperfclientlog}' ''
+ ${result}= SSHLibrary.Execute_Command grep '${pattern}' ${RestPerfClient__restperfclientlog}
+ [Return] ${result}
+
+Get_Current_Log_Name
+ [Return] ${RestPerfClient__restperfclientlog}
+
+Collect_From_Restperfclient
+ [Documentation] Collect useful data produced by restperfclient
+ BuiltIn.Should_Not_Be_Equal '${RestPerfClient__restperfclientlog}' ''
+ SSHLibrary.Get_File ${RestPerfClient__restperfclientlog}
+ BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclientlog} ${EMPTY}
+
+Teardown_Restperfclient
+ [Documentation] Free resources allocated during the RestPerfClient setup
+ SSHLibrary.Switch_Connection ${RestPerfClient__restperfclient}
+ SSHLibrary.Close_Connection
Resource ${CURDIR}/../../../libraries/NetconfKeywords.robot
Resource ${CURDIR}/../../../libraries/NetconfViaRestconf.robot
Resource ${CURDIR}/../../../libraries/NexusKeywords.robot
+Resource ${CURDIR}/../../../libraries/RestPerfClient.robot
Resource ${CURDIR}/../../../libraries/SetupUtils.robot
Resource ${CURDIR}/../../../libraries/Utils.robot
Variables ${CURDIR}/../../../variables/Variables.py
Run_RestPerfClient_Directly_On_MDSAL
[Documentation] Deploy and execute restperfclient, asking it to send the specified amount of requests to the MDSAL via Restconf.
[Timeout] ${DIRECT_MDSAL_TIMEOUT_FOR_TESTCASE}
- ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient direct
- BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog}
- SSHLibrary.Switch_Connection ${restperfclient}
- SSHLibrary.Put_File ${CURDIR}/../../../variables/netconf/RestPerfClient/request1.json
- ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client
- SSHLibrary.Set_Client_Configuration timeout=${DIRECT_MDSAL_TIMEOUT}
- ${options}= BuiltIn.Set_Variable --ip ${ODL_SYSTEM_IP} --port ${RESTCONFPORT} --edits ${REQUEST_COUNT}
- ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json --async-requests false
- ${options}= BuiltIn.Set_Variable ${options} --auth ${ODL_RESTCONF_USER} ${ODL_RESTCONF_PASSWORD}
- ${options}= BuiltIn.Set_Variable ${options} --destination
- ${prefix}= NexusKeywords.Compose_Full_Java_Command -Xmx1G -XX:MaxPermSize=256M -jar ${filename} ${options}
- BuiltIn.Set_Suite_Variable ${command_prefix} ${prefix}
- ${command} BuiltIn.Set_Variable ${command_prefix} /restconf/config/car:cars
- BuiltIn.Log Running restperfclient: ${command}
- Set_Known_Bug_Id 5413
- Execute_Command_Passes ${command} >${restperfclientlog} 2>&1
- Set_Unknown_Bug_Id
- ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog}
- BuiltIn.Should_Not_Be_Equal '${result}' ''
+ ${url}= BuiltIn.Set_Variable /restconf/config/car:cars
+ RestPerfClient.Invoke_Restperfclient ${DIRECT_MDSAL_TIMEOUT} ${url} testcase=direct
Check_For_Failed_Direct_MDSAL_Requests
[Documentation] Make sure there are no failed requests in the restperfclient log.
... failure and failed requests. Failed requests are rejected because
... we don't want to test performance of ODL rejecting our requests.
[Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
- ${result}= SSHLibrary.Execute_Command grep "thread timed out" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out
BuiltIn.Should_Be_Equal '${result}' ''
- ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed
BuiltIn.Should_Be_Equal '${result}' ''
- ${result}= SSHLibrary.Execute_Command grep "Status code" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Status code
BuiltIn.Should_Be_Equal '${result}' ''
Cleanup_And_Collect_For_Direct_Access
[Documentation] Cleanup the test data produced by the direct MDSAL access.
[Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+ RestPerfClient.Collect_From_Restperfclient
NetconfViaRestconf.Delete_Xml_Template_Folder_Via_Restconf ${DIRECTORY_WITH_TEMPLATE_FOLDERS}${/}cars-delete {}
- SSHLibrary.Get_File ${restperfclientlog}
Create_Test_Data_For_Connector_Access
[Documentation] Create the test data container again so it is ready for the netconf connector test.
Run_RestPerfClient_Through_Netconf_Connector
[Documentation] Ask RestPerfClient to send the requests to the MDSAL mapped via a netconf connector.
[Timeout] ${NETCONF_CONNECTOR_MDSAL_TIMEOUT_FOR_TESTCASE}
- ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient netconf-connector
- BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog}
- SSHLibrary.Switch_Connection ${restperfclient}
- SSHLibrary.Set_Client_Configuration timeout=${NETCONF_CONNECTOR_MDSAL_TIMEOUT}
- ${command} BuiltIn.Set_Variable ${command_prefix} /restconf/config/network-topology:network-topology/topology/topology-netconf/node/odl-mdsal-northbound-via-netconf-connector/yang-ext:mount/car:cars
- BuiltIn.Log Running restperfclient: ${command}
- Set_Known_Bug_Id 5413
- Execute_Command_Passes ${command} >${restperfclientlog} 2>&1
- Set_Unknown_Bug_Id
- ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog}
- BuiltIn.Should_Not_Be_Equal '${result}' ''
+ ${url}= BuiltIn.Set_Variable /restconf/config/network-topology:network-topology/topology/topology-netconf/node/odl-mdsal-northbound-via-netconf-connector/yang-ext:mount/car:cars
+ RestPerfClient.Invoke_Restperfclient ${NETCONF_CONNECTOR_MDSAL_TIMEOUT} ${url} testcase=netconf-connector
Check_For_Failed_Netconf_Connector_Requests
[Documentation] Make sure there are no failed requests in the restperfclient log.
... failure and failed requests. Failed requests are rejected because
... we don't want to test performance of ODL rejecting our requests.
[Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+ ${restperfclientlog}= RestPerfClient.Get_Current_Log_Name
Set_Known_Bug_Id 5581
- ${result}= SSHLibrary.Execute_Command grep "thread timed out" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out
BuiltIn.Should_Be_Equal '${result}' ''
Set_Unknown_Bug_Id
- ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed
BuiltIn.Should_Be_Equal '${result}' ''
- ${result}= SSHLibrary.Execute_Command grep "Status code" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Status code
BuiltIn.Should_Be_Equal '${result}' ''
Deconfigure_ODL_From_Netconf
Cleanup_And_Collect_For_Connector_Access
[Documentation] Delete the test data produced by the Netconf connector MDSAL access.
[Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+ RestPerfClient.Collect_From_Restperfclient
NetconfViaRestconf.Delete_Xml_Template_Folder_Via_Restconf ${DIRECTORY_WITH_TEMPLATE_FOLDERS}${/}cars-delete {}
- SSHLibrary.Get_File ${restperfclientlog}
*** Keywords ***
Setup_Everything
# Setup resources used by the suite.
SetupUtils.Setup_Utils_For_Setup_And_Teardown
NetconfKeywords.Setup_Netconf_Keywords
- # Connect to the tools system (rest-perf-client)
- ${restperfclient}= SSHKeywords.Open_Connection_To_Tools_System
- BuiltIn.Set_Suite_Variable ${restperfclient} ${restperfclient}
- # Initialize artifact deployment infrastructure.
- ${testtool}= SSHLibrary.Get Connection
- BuiltIn.Set_Suite_Variable ${testtool} ${testtool.index}
+ RestPerfClient.Setup_Restperfclient
# Calculate timeouts
${value}= BuiltIn.Evaluate ${REQUEST_COUNT}/50+10
Utils.Set_User_Configurable_Variable_Default DIRECT_MDSAL_TIMEOUT ${value} s
[Documentation] Teardown the test infrastructure, perform cleanup and release all resources.
Teardown_Netconf_Via_Restconf
RequestsLibrary.Delete_All_Sessions
- SSHLibrary.Switch_Connection ${testtool}
- SSHLibrary.Close_Connection
+ RestPerfClient.Teardown_Restperfclient
Resource ${CURDIR}/../../../libraries/NetconfKeywords.robot
Resource ${CURDIR}/../../../libraries/NetconfViaRestconf.robot
Resource ${CURDIR}/../../../libraries/NexusKeywords.robot
+Resource ${CURDIR}/../../../libraries/RestPerfClient.robot
Resource ${CURDIR}/../../../libraries/SetupUtils.robot
Resource ${CURDIR}/../../../libraries/Utils.robot
Variables ${CURDIR}/../../../variables/Variables.py
${template_as_string}= BuiltIn.Set_Variable {'DEVICE_NAME': '${DEVICE_NAME}'}
NetconfViaRestconf.Post_Xml_Template_Folder_Via_Restconf ${directory_with_crud_templates}${/}cars ${template_as_string}
-Deploy_And_Run_RestPerfClient
+Run_Restperfclient
[Documentation] Deploy and execute restperfclient, asking it to send the specified amount of requests to the netconf connector of the device.
[Timeout] ${TESTTOOL_DEVICE_TIMEOUT_FOR_TESTCASE}
- SSHLibrary.Switch_Connection ${restperfclient}
- SSHLibrary.Put_File ${CURDIR}/../../../variables/netconf/RestPerfClient/request1.json
- ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client
- ${timeout}= BuiltIn.Evaluate ${REQUEST_COUNT}/100+10
- SSHLibrary.Set_Client_Configuration timeout=${timeout}
- ${options}= BuiltIn.Set_Variable --ip ${ODL_SYSTEM_IP} --port ${RESTCONFPORT} --edits ${REQUEST_COUNT}
- ${options}= BuiltIn.Set_Variable ${options} --destination /restconf/config/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount/car:cars
- ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json
- ${options}= BuiltIn.Set_Variable ${options} --auth ${ODL_RESTCONF_USER} ${ODL_RESTCONF_PASSWORD}
- ${command}= NexusKeywords.Compose_Full_Java_Command -Xmx1G -XX:MaxPermSize=256M -jar ${filename} ${options}
- BuiltIn.Log Running restperfclient: ${command}
- ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient
- BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog}
- SSHKeywords.Execute_Command_Passes ${command} >${restperfclientlog} 2>&1
- SSHLibrary.Get_File ${restperfclientlog}
- ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog}
- BuiltIn.Should_Not_Be_Equal '${result}' ''
+ ${url}= BuiltIn.Set_Variable /restconf/config/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount/car:cars
+ RestPerfClient.Invoke_Restperfclient ${TESTTOOL_DEVICE_TIMEOUT} ${url} async=true
Check_For_Failed_Requests
[Documentation] Make sure there are no failed requests in the restperfclient log.
... This is a separate test case to distinguish between restperfclient
... failure and failed requests. Failed requests are rejected because
... we don't want to test performance of ODL rejecting our requests.
- ${result}= SSHLibrary.Execute_Command grep "thread timed out" ${restperfclientlog}
+ ${restperfclientlog}= RestPerfClient.Get_Current_Log_Name
+ ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out
BuiltIn.Should_Be_Equal '${result}' ''
- ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed
BuiltIn.Should_Be_Equal '${result}' ''
- ${result}= SSHLibrary.Execute_Command grep "Status code" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Status code
BuiltIn.Should_Be_Equal '${result}' ''
-Deconfigure_Device_From_Netconf
+Cleanup_And_Collect
[Documentation] Deconfigure the testtool device on Netconf connector.
[Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+ RestPerfClient.Collect_From_Restperfclient
NetconfKeywords.Remove_Device_From_Netconf ${DEVICE_NAME}
*** Keywords ***
# Setup resources used by the suite.
SetupUtils.Setup_Utils_For_Setup_And_Teardown
NetconfKeywords.Setup_Netconf_Keywords
- # Connect to the tools system (rest-perf-client)
- ${restperfclient}= SSHKeywords.Open_Connection_To_Tools_System
- BuiltIn.Set_Suite_Variable ${restperfclient} ${restperfclient}
- ${testtool}= SSHLibrary.Get Connection
- BuiltIn.Set_Suite_Variable ${testtool} ${testtool.index}
+ RestPerfClient.Setup_Restperfclient
+ # Connect to the tools system (testtool)
+ ${testtool}= SSHKeywords.Open_Connection_To_Tools_System
+ BuiltIn.Set_Suite_Variable ${testtool} ${testtool}
Teardown_Everything
[Documentation] Teardown the test infrastructure, perform cleanup and release all resources.
Teardown_Netconf_Via_Restconf
RequestsLibrary.Delete_All_Sessions
+ RestPerfClient.Teardown_Restperfclient
SSHLibrary.Switch_Connection ${testtool}
BuiltIn.Run_Keyword_And_Ignore_Error NetconfKeywords.Stop_Testtool
Code Review / integration / test.git / commitdiff