From: Jozef Behran Date: Wed, 2 Dec 2015 09:35:42 +0000 (+0100) Subject: RestPerfClient performance test suite X-Git-Tag: release/beryllium-sr1~78 X-Git-Url: https://git.opendaylight.org/gerrit/gitweb?p=integration%2Ftest.git;a=commitdiff_plain;h=31db4aad404a6618a6ec3a0f3f969ce4d3264846 RestPerfClient performance test suite Deploy restperfclient and ask it to bombard one testtool device mounted on a netconf connector with configuration update requests. The count of these requests is configurable via a Robot variable. Change-Id: I4550a9ce4f6ac2e3233097c0c1246fee4861b14c Signed-off-by: Jozef Behran --- diff --git a/csit/suites/netconf/restperfclient/performance.robot b/csit/suites/netconf/restperfclient/performance.robot new file mode 100644 index 0000000000..7403726883 --- /dev/null +++ b/csit/suites/netconf/restperfclient/performance.robot @@ -0,0 +1,128 @@ +*** Settings *** +Documentation netconf-restperfclient Update performance test suite. +... +... Copyright (c) 2015 Cisco Systems, Inc. and others. All rights reserved. +... +... This program and the accompanying materials are made available under the +... terms of the Eclipse Public License v1.0 which accompanies this distribution, +... and is available at http://www.eclipse.org/legal/epl-v10.html +... +... +... Perform given count of update operations on device data mounted onto a +... netconf connector (using the netconf-testtool-restperfclient tool) and +... see how much time it took. More exactly, it sends the data to a restconf +... mountpoint of the netconf connector belonging to the device, which turns +... out to turn the first request sent to a "create" request and the +... remaining requests to "update" requests (due to how the testtool device +... behavior is implemented). +... +... TODO: The "Wait_Until_Prompt" keyword shall probably be turned into a +... reusable piece and moved into SSHKeywords. There is a bunch of other +... test suites (e.g. PCEP, BGP) which contain the same or similar pieces of +... code. +Suite Setup Setup_Everything +Suite Teardown Teardown_Everything +Test Setup SetupUtils.Setup_Test_With_Logging_And_Fast_Failing +Test Teardown FailFast.Start_Failing_Fast_If_This_Failed +Library Collections +Library RequestsLibrary +Library OperatingSystem +Library String +Library SSHLibrary timeout=10s +Resource ${CURDIR}/../../../libraries/FailFast.robot +Resource ${CURDIR}/../../../libraries/KarafKeywords.robot +Resource ${CURDIR}/../../../libraries/NetconfKeywords.robot +Resource ${CURDIR}/../../../libraries/NetconfViaRestconf.robot +Resource ${CURDIR}/../../../libraries/NexusKeywords.robot +Resource ${CURDIR}/../../../libraries/SetupUtils.robot +Resource ${CURDIR}/../../../libraries/Utils.robot +Variables ${CURDIR}/../../../variables/Variables.py + +*** Variables *** +${DIRECTORY_WITH_CRUD_TEMPLATES} ${CURDIR}/../../../variables/netconf/CRUD +${DIRECTORY_WITH_TEMPLATE_FOLDERS} ${CURDIR}/../../../variables/netconf/RestPerfClient +${DEVICE_NAME} ${FIRST_TESTTOOL_PORT}-sim-device +${REQUEST_COUNT} 65536 + +*** Test Cases *** +Start_Testtool + [Documentation] Deploy and start test tool, then wait for all its devices to become online. + # Start test tool + SSHLibrary.Switch_Connection ${testtool} + NetconfKeywords.Install_And_Start_Testtool device-count=1 schemas=${CURDIR}/../../../variables/netconf/CRUD/schemas mdsal=false debug=false + +Configure_Device_On_Netconf + [Documentation] Configure the testtool device on Netconf connector. + NetconfKeywords.Configure_Device_In_Netconf ${DEVICE_NAME} + +Wait_For_Device_To_Become_Connected + [Documentation] Wait until the device becomes available through Netconf. + NetconfKeywords.Wait_Device_Connected ${DEVICE_NAME} + +Create_Device_Data + [Documentation] Send some sample test data into the device and check that the request went OK. + ${template_as_string}= BuiltIn.Set_Variable {'DEVICE_NAME': '${DEVICE_NAME}'} + NetconfViaRestconf.Post_Xml_Template_Folder_Via_Restconf ${DIRECTORY_WITH_CRUD_TEMPLATES}${/}cars ${template_as_string} + +Deploy_And_Run_RestPerfClient + [Documentation] Deploy and execute restperfclient, asking it to send the specified amount of requests to the netconf connector of the device. + SSHLibrary.Switch_Connection ${restperfclient} + SSHLibrary.Put_File ${CURDIR}/../../../variables/netconf/RestPerfClient/request1.json + ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client + ${timeout}= BuiltIn.Evaluate ${REQUEST_COUNT}/100+10 + SSHLibrary.Set_Client_Configuration timeout=${timeout} + ${options}= BuiltIn.Set_Variable --ip ${ODL_SYSTEM_IP} --port ${RESTCONFPORT} --edits ${REQUEST_COUNT} + ${options}= BuiltIn.Set_Variable ${options} --destination /restconf/config/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount/car:cars + ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json + ${options}= BuiltIn.Set_Variable ${options} --auth ${ODL_RESTCONF_USER} ${ODL_RESTCONF_PASSWORD} + ${command} BuiltIn.Set_Variable java -Xmx1G -XX:MaxPermSize=256M -jar ${filename} ${options} + BuiltIn.Log Running restperfclient: ${command} + ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient + BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog} + Execute_Command_Passes ${command} >${restperfclientlog} 2>&1 + SSHLibrary.Get_File ${restperfclientlog} + ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog} + BuiltIn.Should_Not_Be_Equal '${result}' '' + +Check_For_Failed_Requests + [Documentation] Make sure there are no failed requests in the restperfclient log. + ... This is a separate test case to distinguish between restperfclient + ... failure and failed requests. Failed requests are rejected because + ... we don't want to test performance of ODL rejecting our requests. + ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog} + BuiltIn.Should_Be_Equal '${result}' '' + +Deconfigure_Device_From_Netconf + [Documentation] Deconfigure the testtool device on Netconf connector. + [Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing + NetconfKeywords.Remove_Device_From_Netconf ${DEVICE_NAME} + +*** Keywords *** +Setup_Everything + [Documentation] Setup everything needed for the test cases. + # Setup resources used by the suite. + SetupUtils.Setup_Utils_For_Setup_And_Teardown + RequestsLibrary.Create_Session operational http://${ODL_SYSTEM_IP}:${RESTCONFPORT}${OPERATIONAL_API} auth=${AUTH} + NetconfKeywords.Setup_Netconf_Keywords + # Connect to the tools system (rest-perf-client) + ${restperfclient}= SSHKeywords.Open_Connection_To_Tools_System + BuiltIn.Set_Suite_Variable ${restperfclient} ${restperfclient} + # Initialize artifact deployment infrastructure. + NexusKeywords.Initialize_Artifact_Deployment_And_Usage + ${testtool}= SSHLibrary.Get Connection + BuiltIn.Set_Suite_Variable ${testtool} ${testtool.index} + +Teardown_Everything + [Documentation] Teardown the test infrastructure, perform cleanup and release all resources. + Teardown_Netconf_Via_Restconf + RequestsLibrary.Delete_All_Sessions + SSHLibrary.Switch_Connection ${testtool} + BuiltIn.Run_Keyword_And_Ignore_Error NetconfKeywords.Stop_Testtool + +Wait_Until_Prompt + [Documentation] Wait until prompt appears or timeout occurs. When timeout occurs, send Ctrl-C and then wait for the prompt again. + ... This is necessary because the restperfclient can crash and hang, requiring that Ctrl-C character to get rid of it for good. + ${status} ${result}= BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Read_Until_Prompt + Return_From_Keyword_If '${status}' == 'PASS' + Utils.Write_Bare_Ctrl_C + SSHLibrary.Read_Until_Prompt diff --git a/csit/testplans/netconf-scale.txt b/csit/testplans/netconf-scale.txt index 08d21b2e1a..522e07002e 100644 --- a/csit/testplans/netconf-scale.txt +++ b/csit/testplans/netconf-scale.txt @@ -6,5 +6,6 @@ # Place the suites in run order: integration/test/csit/suites/netconf/ready +integration/test/csit/suites/netconf/restperfclient/performance.robot integration/test/csit/suites/netconf/scale/getmulti.robot integration/test/csit/suites/netconf/scale/getsingle.robot diff --git a/csit/variables/Variables.py b/csit/variables/Variables.py index 1c384bcec7..5ccea6037e 100644 --- a/csit/variables/Variables.py +++ b/csit/variables/Variables.py @@ -43,6 +43,10 @@ PROTOCOL_LOG_LEVEL = BGPCEP_LOG_LEVEL ODL_BGP_PORT = '1790' BGP_TOOL_PORT = '17900' +# Restconf variables +ODL_RESTCONF_USER = 'admin' +ODL_RESTCONF_PASSWORD = 'admin' + # Netconf variables ODL_NETCONF_PORT = '2830' ODL_NETCONF_USER = 'admin' diff --git a/csit/variables/netconf/RestPerfClient/request1.json b/csit/variables/netconf/RestPerfClient/request1.json new file mode 100644 index 0000000000..b963b1f41b --- /dev/null +++ b/csit/variables/netconf/RestPerfClient/request1.json @@ -0,0 +1,9 @@ +{ + "car:car-entry": { + "car:id":"http://www.example.com/cars/{PEERID}/{INTLEAF}", + "car:model":"TestCar", + "car:manufacturer":"{PEERID}", + "car:year":"2016", + "car:category":"test" + } +}