From: Jozef Behran Date: Wed, 23 Mar 2016 07:52:25 +0000 (+0100) Subject: Remove restperfclient invocation code triplication X-Git-Tag: release/lithium-sr4~88 X-Git-Url: https://git.opendaylight.org/gerrit/gitweb?a=commitdiff_plain;h=refs%2Fchanges%2F85%2F36585%2F18;p=integration%2Ftest.git Remove restperfclient invocation code triplication Move the triplicated restperfclient invocation code into its own bunch of keywords and hide some implementation details from the test suite files. This makes the test suite files easier to read and as a result it is also easier to add additional code into the restperfclient invocation process. Change-Id: I7a64b15e72c5fcf7766a768b44d6ebc3dd5434b2 Signed-off-by: Jozef Behran --- diff --git a/csit/libraries/NetconfKeywords.robot b/csit/libraries/NetconfKeywords.robot index 234f99d8f1..f6610e8060 100644 --- a/csit/libraries/NetconfKeywords.robot +++ b/csit/libraries/NetconfKeywords.robot @@ -14,6 +14,7 @@ Documentation Perform complex operations on netconf. Library Collections Library DateTime Library RequestsLibrary +Library SSHLibrary Resource NetconfViaRestconf.robot Resource NexusKeywords.robot Resource SSHKeywords.robot diff --git a/csit/libraries/RestPerfClient.robot b/csit/libraries/RestPerfClient.robot new file mode 100644 index 0000000000..5a1ca5878e --- /dev/null +++ b/csit/libraries/RestPerfClient.robot @@ -0,0 +1,90 @@ +*** Settings *** +Documentation RestPerfClient handling singleton resource. +... +... Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved. +... +... This program and the accompanying materials are made available under the +... terms of the Eclipse Public License v1.0 which accompanies this distribution, +... and is available at http://www.eclipse.org/legal/epl-v10.html +... +... This singleton manages RestPerfClient invocation, tracks the log file +... produced by the invocation, allows the test suite to easily search this +... log file and collect it once done. +... +... TODO: Currently only one RestPerfClient invocation running at a time is +... supported. Support for multiple concurrently running RestPerfClient +... invocations might be needed for example when performance testing cluster +... nodes. However no such suites are planned for now. +... +... FIXME: There may be suites which want to use this Resource without +... NetconfKeywords, in which case NexusKeywords will not be initialized +... and Setup_Restperfclient will fail. Fixing this problem will require +... updating NexusKeywords initialization (which may break other suites) +... and currently all suites using this use also NetconfKeywords so this +... was postponed. Workaround for the problem: Initialize NexusKeywords +... manually before initializing this resource. +Library SSHLibrary +Resource ${CURDIR}/NexusKeywords.robot +Resource ${CURDIR}/SSHKeywords.robot +Resource ${CURDIR}/Utils.robot + +*** Variables *** +${RestPerfClient__restperfclientlog} ${EMPTY} + +*** Keywords *** +Setup_Restperfclient + [Documentation] Deploy RestPerfClient and determine the Java command to use to call it. + ... Open a SSH connection through which the RestPerfClient will be + ... invoked, deploy RestPerfClient and the data files it needs to do + ... its work and initialize the internal state for the remaining + ... keywords. + ${connection}= SSHKeywords.Open_Connection_To_Tools_System + BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclient} ${connection} + SSHLibrary.Put_File ${CURDIR}/../variables/netconf/RestPerfClient/request1.json + ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client + ${prefix}= NexusKeywords.Compose_Full_Java_Command -Xmx1G -XX:MaxPermSize=256M -jar ${filename} + BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclient_invocation_command_prefix} ${prefix} + +Invoke_Restperfclient + [Arguments] ${timeout} ${url} ${testcase}=${EMPTY} ${ip}=${ODL_SYSTEM_IP} ${port}=${RESTCONFPORT} ${count}=${REQUEST_COUNT} + ... ${async}=false ${user}=${ODL_RESTCONF_USER} ${password}=${ODL_RESTCONF_PASSWORD} + [Documentation] Invoke RestPerfClient on the specified URL with the specified timeout. + ... Assemble the RestPerfClient invocation commad, setup the specified + ... timeout for the SSH connection, invoke the assembled command and + ... then check that RestPerfClient finished its run correctly. + ${logname}= Utils.Get_Log_File_Name restperfclient ${testcase} + BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclientlog} ${logname} + ${options}= BuiltIn.Set_Variable --ip ${ip} --port ${port} --edits ${count} + ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json --async-requests ${async} + ${options}= BuiltIn.Set_Variable ${options} --auth ${user} ${password} + ${options}= BuiltIn.Set_Variable ${options} --destination ${url} + ${command}= BuiltIn.Set_Variable ${RestPerfClient__restperfclient_invocation_command_prefix} ${options} + BuiltIn.Log Running restperfclient: ${command} + SSHLibrary.Switch_Connection ${RestPerfClient__restperfclient} + SSHLibrary.Set_Client_Configuration timeout=${timeout} + Set_Known_Bug_Id 5413 + Execute_Command_Passes ${command} >${RestPerfClient__restperfclientlog} 2>&1 + Set_Unknown_Bug_Id + ${result}= Grep_Restperfclient_Log FINISHED. Execution time: + BuiltIn.Should_Not_Be_Equal '${result}' '' + +Grep_Restperfclient_Log + [Documentation] Search for the specified string in the log file produced by latest invocation of RestPerfClient + [Arguments] ${pattern} + BuiltIn.Should_Not_Be_Equal '${RestPerfClient__restperfclientlog}' '' + ${result}= SSHLibrary.Execute_Command grep '${pattern}' ${RestPerfClient__restperfclientlog} + [Return] ${result} + +Get_Current_Log_Name + [Return] ${RestPerfClient__restperfclientlog} + +Collect_From_Restperfclient + [Documentation] Collect useful data produced by restperfclient + BuiltIn.Should_Not_Be_Equal '${RestPerfClient__restperfclientlog}' '' + SSHLibrary.Get_File ${RestPerfClient__restperfclientlog} + BuiltIn.Set_Suite_Variable ${RestPerfClient__restperfclientlog} ${EMPTY} + +Teardown_Restperfclient + [Documentation] Free resources allocated during the RestPerfClient setup + SSHLibrary.Switch_Connection ${RestPerfClient__restperfclient} + SSHLibrary.Close_Connection diff --git a/csit/suites/netconf/restperfclient/mdsal.robot b/csit/suites/netconf/restperfclient/mdsal.robot index ae7923b1bb..284877c5cf 100644 --- a/csit/suites/netconf/restperfclient/mdsal.robot +++ b/csit/suites/netconf/restperfclient/mdsal.robot @@ -27,6 +27,7 @@ Library SSHLibrary timeout=10s Resource ${CURDIR}/../../../libraries/NetconfKeywords.robot Resource ${CURDIR}/../../../libraries/NetconfViaRestconf.robot Resource ${CURDIR}/../../../libraries/NexusKeywords.robot +Resource ${CURDIR}/../../../libraries/RestPerfClient.robot Resource ${CURDIR}/../../../libraries/SetupUtils.robot Resource ${CURDIR}/../../../libraries/Utils.robot Variables ${CURDIR}/../../../variables/Variables.py @@ -43,25 +44,8 @@ Create_Test_Data_For_Direct_Access Run_RestPerfClient_Directly_On_MDSAL [Documentation] Deploy and execute restperfclient, asking it to send the specified amount of requests to the MDSAL via Restconf. [Timeout] ${DIRECT_MDSAL_TIMEOUT_FOR_TESTCASE} - ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient direct - BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog} - SSHLibrary.Switch_Connection ${restperfclient} - SSHLibrary.Put_File ${CURDIR}/../../../variables/netconf/RestPerfClient/request1.json - ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client - SSHLibrary.Set_Client_Configuration timeout=${DIRECT_MDSAL_TIMEOUT} - ${options}= BuiltIn.Set_Variable --ip ${ODL_SYSTEM_IP} --port ${RESTCONFPORT} --edits ${REQUEST_COUNT} - ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json --async-requests false - ${options}= BuiltIn.Set_Variable ${options} --auth ${ODL_RESTCONF_USER} ${ODL_RESTCONF_PASSWORD} - ${options}= BuiltIn.Set_Variable ${options} --destination - ${prefix}= NexusKeywords.Compose_Full_Java_Command -Xmx1G -XX:MaxPermSize=256M -jar ${filename} ${options} - BuiltIn.Set_Suite_Variable ${command_prefix} ${prefix} - ${command} BuiltIn.Set_Variable ${command_prefix} /restconf/config/car:cars - BuiltIn.Log Running restperfclient: ${command} - Set_Known_Bug_Id 5413 - Execute_Command_Passes ${command} >${restperfclientlog} 2>&1 - Set_Unknown_Bug_Id - ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog} - BuiltIn.Should_Not_Be_Equal '${result}' '' + ${url}= BuiltIn.Set_Variable /restconf/config/car:cars + RestPerfClient.Invoke_Restperfclient ${DIRECT_MDSAL_TIMEOUT} ${url} testcase=direct Check_For_Failed_Direct_MDSAL_Requests [Documentation] Make sure there are no failed requests in the restperfclient log. @@ -69,18 +53,18 @@ Check_For_Failed_Direct_MDSAL_Requests ... failure and failed requests. Failed requests are rejected because ... we don't want to test performance of ODL rejecting our requests. [Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing - ${result}= SSHLibrary.Execute_Command grep "thread timed out" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out BuiltIn.Should_Be_Equal '${result}' '' - ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed BuiltIn.Should_Be_Equal '${result}' '' - ${result}= SSHLibrary.Execute_Command grep "Status code" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log Status code BuiltIn.Should_Be_Equal '${result}' '' Cleanup_And_Collect_For_Direct_Access [Documentation] Cleanup the test data produced by the direct MDSAL access. [Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing + RestPerfClient.Collect_From_Restperfclient NetconfViaRestconf.Delete_Xml_Template_Folder_Via_Restconf ${DIRECTORY_WITH_TEMPLATE_FOLDERS}${/}cars-delete {} - SSHLibrary.Get_File ${restperfclientlog} Create_Test_Data_For_Connector_Access [Documentation] Create the test data container again so it is ready for the netconf connector test. @@ -95,17 +79,8 @@ Configure_ODL_As_A_Device_On_Netconf Run_RestPerfClient_Through_Netconf_Connector [Documentation] Ask RestPerfClient to send the requests to the MDSAL mapped via a netconf connector. [Timeout] ${NETCONF_CONNECTOR_MDSAL_TIMEOUT_FOR_TESTCASE} - ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient netconf-connector - BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog} - SSHLibrary.Switch_Connection ${restperfclient} - SSHLibrary.Set_Client_Configuration timeout=${NETCONF_CONNECTOR_MDSAL_TIMEOUT} - ${command} BuiltIn.Set_Variable ${command_prefix} /restconf/config/network-topology:network-topology/topology/topology-netconf/node/odl-mdsal-northbound-via-netconf-connector/yang-ext:mount/car:cars - BuiltIn.Log Running restperfclient: ${command} - Set_Known_Bug_Id 5413 - Execute_Command_Passes ${command} >${restperfclientlog} 2>&1 - Set_Unknown_Bug_Id - ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog} - BuiltIn.Should_Not_Be_Equal '${result}' '' + ${url}= BuiltIn.Set_Variable /restconf/config/network-topology:network-topology/topology/topology-netconf/node/odl-mdsal-northbound-via-netconf-connector/yang-ext:mount/car:cars + RestPerfClient.Invoke_Restperfclient ${NETCONF_CONNECTOR_MDSAL_TIMEOUT} ${url} testcase=netconf-connector Check_For_Failed_Netconf_Connector_Requests [Documentation] Make sure there are no failed requests in the restperfclient log. @@ -113,13 +88,14 @@ Check_For_Failed_Netconf_Connector_Requests ... failure and failed requests. Failed requests are rejected because ... we don't want to test performance of ODL rejecting our requests. [Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing + ${restperfclientlog}= RestPerfClient.Get_Current_Log_Name Set_Known_Bug_Id 5581 - ${result}= SSHLibrary.Execute_Command grep "thread timed out" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out BuiltIn.Should_Be_Equal '${result}' '' Set_Unknown_Bug_Id - ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed BuiltIn.Should_Be_Equal '${result}' '' - ${result}= SSHLibrary.Execute_Command grep "Status code" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log Status code BuiltIn.Should_Be_Equal '${result}' '' Deconfigure_ODL_From_Netconf @@ -130,8 +106,8 @@ Deconfigure_ODL_From_Netconf Cleanup_And_Collect_For_Connector_Access [Documentation] Delete the test data produced by the Netconf connector MDSAL access. [Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing + RestPerfClient.Collect_From_Restperfclient NetconfViaRestconf.Delete_Xml_Template_Folder_Via_Restconf ${DIRECTORY_WITH_TEMPLATE_FOLDERS}${/}cars-delete {} - SSHLibrary.Get_File ${restperfclientlog} *** Keywords *** Setup_Everything @@ -139,12 +115,7 @@ Setup_Everything # Setup resources used by the suite. SetupUtils.Setup_Utils_For_Setup_And_Teardown NetconfKeywords.Setup_Netconf_Keywords - # Connect to the tools system (rest-perf-client) - ${restperfclient}= SSHKeywords.Open_Connection_To_Tools_System - BuiltIn.Set_Suite_Variable ${restperfclient} ${restperfclient} - # Initialize artifact deployment infrastructure. - ${testtool}= SSHLibrary.Get Connection - BuiltIn.Set_Suite_Variable ${testtool} ${testtool.index} + RestPerfClient.Setup_Restperfclient # Calculate timeouts ${value}= BuiltIn.Evaluate ${REQUEST_COUNT}/50+10 Utils.Set_User_Configurable_Variable_Default DIRECT_MDSAL_TIMEOUT ${value} s @@ -159,5 +130,4 @@ Teardown_Everything [Documentation] Teardown the test infrastructure, perform cleanup and release all resources. Teardown_Netconf_Via_Restconf RequestsLibrary.Delete_All_Sessions - SSHLibrary.Switch_Connection ${testtool} - SSHLibrary.Close_Connection + RestPerfClient.Teardown_Restperfclient diff --git a/csit/suites/netconf/restperfclient/performance.robot b/csit/suites/netconf/restperfclient/performance.robot index 7c24c8ddf5..9e42341696 100644 --- a/csit/suites/netconf/restperfclient/performance.robot +++ b/csit/suites/netconf/restperfclient/performance.robot @@ -28,6 +28,7 @@ Resource ${CURDIR}/../../../libraries/KarafKeywords.robot Resource ${CURDIR}/../../../libraries/NetconfKeywords.robot Resource ${CURDIR}/../../../libraries/NetconfViaRestconf.robot Resource ${CURDIR}/../../../libraries/NexusKeywords.robot +Resource ${CURDIR}/../../../libraries/RestPerfClient.robot Resource ${CURDIR}/../../../libraries/SetupUtils.robot Resource ${CURDIR}/../../../libraries/Utils.robot Variables ${CURDIR}/../../../variables/Variables.py @@ -57,42 +58,29 @@ Create_Device_Data ${template_as_string}= BuiltIn.Set_Variable {'DEVICE_NAME': '${DEVICE_NAME}'} NetconfViaRestconf.Post_Xml_Template_Folder_Via_Restconf ${directory_with_crud_templates}${/}cars ${template_as_string} -Deploy_And_Run_RestPerfClient +Run_Restperfclient [Documentation] Deploy and execute restperfclient, asking it to send the specified amount of requests to the netconf connector of the device. [Timeout] ${TESTTOOL_DEVICE_TIMEOUT_FOR_TESTCASE} - SSHLibrary.Switch_Connection ${restperfclient} - SSHLibrary.Put_File ${CURDIR}/../../../variables/netconf/RestPerfClient/request1.json - ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client - ${timeout}= BuiltIn.Evaluate ${REQUEST_COUNT}/100+10 - SSHLibrary.Set_Client_Configuration timeout=${timeout} - ${options}= BuiltIn.Set_Variable --ip ${ODL_SYSTEM_IP} --port ${RESTCONFPORT} --edits ${REQUEST_COUNT} - ${options}= BuiltIn.Set_Variable ${options} --destination /restconf/config/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount/car:cars - ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json - ${options}= BuiltIn.Set_Variable ${options} --auth ${ODL_RESTCONF_USER} ${ODL_RESTCONF_PASSWORD} - ${command}= NexusKeywords.Compose_Full_Java_Command -Xmx1G -XX:MaxPermSize=256M -jar ${filename} ${options} - BuiltIn.Log Running restperfclient: ${command} - ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient - BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog} - SSHKeywords.Execute_Command_Passes ${command} >${restperfclientlog} 2>&1 - SSHLibrary.Get_File ${restperfclientlog} - ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog} - BuiltIn.Should_Not_Be_Equal '${result}' '' + ${url}= BuiltIn.Set_Variable /restconf/config/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount/car:cars + RestPerfClient.Invoke_Restperfclient ${TESTTOOL_DEVICE_TIMEOUT} ${url} async=true Check_For_Failed_Requests [Documentation] Make sure there are no failed requests in the restperfclient log. ... This is a separate test case to distinguish between restperfclient ... failure and failed requests. Failed requests are rejected because ... we don't want to test performance of ODL rejecting our requests. - ${result}= SSHLibrary.Execute_Command grep "thread timed out" ${restperfclientlog} + ${restperfclientlog}= RestPerfClient.Get_Current_Log_Name + ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out BuiltIn.Should_Be_Equal '${result}' '' - ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed BuiltIn.Should_Be_Equal '${result}' '' - ${result}= SSHLibrary.Execute_Command grep "Status code" ${restperfclientlog} + ${result}= RestPerfClient.Grep_Restperfclient_Log Status code BuiltIn.Should_Be_Equal '${result}' '' -Deconfigure_Device_From_Netconf +Cleanup_And_Collect [Documentation] Deconfigure the testtool device on Netconf connector. [Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing + RestPerfClient.Collect_From_Restperfclient NetconfKeywords.Remove_Device_From_Netconf ${DEVICE_NAME} *** Keywords *** @@ -106,15 +94,15 @@ Setup_Everything # Setup resources used by the suite. SetupUtils.Setup_Utils_For_Setup_And_Teardown NetconfKeywords.Setup_Netconf_Keywords - # Connect to the tools system (rest-perf-client) - ${restperfclient}= SSHKeywords.Open_Connection_To_Tools_System - BuiltIn.Set_Suite_Variable ${restperfclient} ${restperfclient} - ${testtool}= SSHLibrary.Get Connection - BuiltIn.Set_Suite_Variable ${testtool} ${testtool.index} + RestPerfClient.Setup_Restperfclient + # Connect to the tools system (testtool) + ${testtool}= SSHKeywords.Open_Connection_To_Tools_System + BuiltIn.Set_Suite_Variable ${testtool} ${testtool} Teardown_Everything [Documentation] Teardown the test infrastructure, perform cleanup and release all resources. Teardown_Netconf_Via_Restconf RequestsLibrary.Delete_All_Sessions + RestPerfClient.Teardown_Restperfclient SSHLibrary.Switch_Connection ${testtool} BuiltIn.Run_Keyword_And_Ignore_Error NetconfKeywords.Stop_Testtool