*** Settings ***
Documentation netconf-restperfclient Update performance test suite.
...
-... Copyright (c) 2015 Cisco Systems, Inc. and others. All rights reserved.
+... Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.
...
... This program and the accompanying materials are made available under the
... terms of the Eclipse Public License v1.0 which accompanies this distribution,
... out to turn the first request sent to a "create" request and the
... remaining requests to "update" requests (due to how the testtool device
... behavior is implemented).
-...
-... TODO: The "Wait_Until_Prompt" keyword shall probably be turned into a
-... reusable piece and moved into SSHKeywords. There is a bunch of other
-... test suites (e.g. PCEP, BGP) which contain the same or similar pieces of
-... code.
Suite Setup Setup_Everything
Suite Teardown Teardown_Everything
Test Setup SetupUtils.Setup_Test_With_Logging_And_Fast_Failing
-Test Teardown FailFast.Start_Failing_Fast_If_This_Failed
+Test Teardown SetupUtils.Teardown_Test_Show_Bugs_And_Start_Fast_Failing_If_Test_Failed
Library Collections
Library RequestsLibrary
Library OperatingSystem
Library String
Library SSHLibrary timeout=10s
-Resource ${CURDIR}/../../../libraries/FailFast.robot
Resource ${CURDIR}/../../../libraries/KarafKeywords.robot
Resource ${CURDIR}/../../../libraries/NetconfKeywords.robot
-Resource ${CURDIR}/../../../libraries/NetconfViaRestconf.robot
Resource ${CURDIR}/../../../libraries/NexusKeywords.robot
+Resource ${CURDIR}/../../../libraries/RestPerfClient.robot
Resource ${CURDIR}/../../../libraries/SetupUtils.robot
+Resource ${CURDIR}/../../../libraries/TemplatedRequests.robot
Resource ${CURDIR}/../../../libraries/Utils.robot
Variables ${CURDIR}/../../../variables/Variables.py
*** Variables ***
-${DIRECTORY_WITH_CRUD_TEMPLATES} ${CURDIR}/../../../variables/netconf/CRUD
-${DIRECTORY_WITH_TEMPLATE_FOLDERS} ${CURDIR}/../../../variables/netconf/RestPerfClient
${DEVICE_NAME} ${FIRST_TESTTOOL_PORT}-sim-device
-${REQUEST_COUNT} 65536
+${REQUEST_COUNT} 16384
+${directory_with_crud_templates} ${CURDIR}/../../../variables/netconf/CRUD
+${device_type} full-uri-device
*** Test Cases ***
Start_Testtool
[Documentation] Deploy and start test tool, then wait for all its devices to become online.
# Start test tool
SSHLibrary.Switch_Connection ${testtool}
- NetconfKeywords.Install_And_Start_Testtool device-count=1 schemas=${CURDIR}/../../../variables/netconf/CRUD/schemas mdsal=false debug=false
+ Run Keyword If '${IS_KARAF_APPL}' == 'True' NetconfKeywords.Install_And_Start_Testtool device-count=1 schemas=${CURDIR}/../../../variables/netconf/CRUD/schemas debug=false
+ ... ELSE NetconfKeywords.Start_Testtool ${NETCONF_FILENAME} device-count=1 schemas=${CURDIR}/../../../variables/netconf/CRUD/schemas debug=false
Configure_Device_On_Netconf
[Documentation] Configure the testtool device on Netconf connector.
- NetconfKeywords.Configure_Device_In_Netconf ${DEVICE_NAME}
+ NetconfKeywords.Configure_Device_In_Netconf ${DEVICE_NAME} device_type=${device_type}
Wait_For_Device_To_Become_Connected
[Documentation] Wait until the device becomes available through Netconf.
Create_Device_Data
[Documentation] Send some sample test data into the device and check that the request went OK.
- ${template_as_string}= BuiltIn.Set_Variable {'DEVICE_NAME': '${DEVICE_NAME}'}
- NetconfViaRestconf.Post_Xml_Template_Folder_Via_Restconf ${DIRECTORY_WITH_CRUD_TEMPLATES}${/}cars ${template_as_string}
+ ${template_as_string}= BuiltIn.Create_Dictionary DEVICE_NAME=${device_name}
+ TemplatedRequests.Post_As_Xml_Templated ${directory_with_crud_templates}${/}cars ${template_as_string}
-Deploy_And_Run_RestPerfClient
+Run_Restperfclient
[Documentation] Deploy and execute restperfclient, asking it to send the specified amount of requests to the netconf connector of the device.
- SSHLibrary.Switch_Connection ${restperfclient}
- SSHLibrary.Put_File ${CURDIR}/../../../variables/netconf/RestPerfClient/request1.json
- ${filename}= NexusKeywords.Deploy_Test_Tool netconf netconf-testtool rest-perf-client
- ${timeout}= BuiltIn.Evaluate ${REQUEST_COUNT}/100+10
- SSHLibrary.Set_Client_Configuration timeout=${timeout}
- ${options}= BuiltIn.Set_Variable --ip ${ODL_SYSTEM_IP} --port ${RESTCONFPORT} --edits ${REQUEST_COUNT}
- ${options}= BuiltIn.Set_Variable ${options} --destination /restconf/config/network-topology:network-topology/topology/topology-netconf/node/${DEVICE_NAME}/yang-ext:mount/car:cars
- ${options}= BuiltIn.Set_Variable ${options} --edit-content request1.json
- ${options}= BuiltIn.Set_Variable ${options} --auth ${ODL_RESTCONF_USER} ${ODL_RESTCONF_PASSWORD}
- ${command} BuiltIn.Set_Variable java -Xmx1G -XX:MaxPermSize=256M -jar ${filename} ${options}
- BuiltIn.Log Running restperfclient: ${command}
- ${restperfclientlog}= Utils.Get_Log_File_Name restperfclient
- BuiltIn.Set_Suite_Variable ${restperfclientlog} ${restperfclientlog}
- Execute_Command_Passes ${command} >${restperfclientlog} 2>&1
- SSHLibrary.Get_File ${restperfclientlog}
- ${result}= SSHLibrary.Execute_Command grep "FINISHED. Execution time:" ${restperfclientlog}
- BuiltIn.Should_Not_Be_Equal '${result}' ''
+ ${url}= BuiltIn.Set_Variable /rests/data/network-topology:network-topology/topology\=topology-netconf/node\=${DEVICE_NAME}/yang-ext:mount/car:cars
+ RestPerfClient.Invoke_Restperfclient ${TESTTOOL_DEVICE_TIMEOUT} ${url} async=false
Check_For_Failed_Requests
[Documentation] Make sure there are no failed requests in the restperfclient log.
... This is a separate test case to distinguish between restperfclient
... failure and failed requests. Failed requests are rejected because
... we don't want to test performance of ODL rejecting our requests.
- ${result}= SSHLibrary.Execute_Command grep "Request failed" ${restperfclientlog}
+ ${result}= RestPerfClient.Grep_Restperfclient_Log thread timed out
+ BuiltIn.Should_Be_Equal '${result}' ''
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Request failed
+ BuiltIn.Should_Be_Equal '${result}' ''
+ ${result}= RestPerfClient.Grep_Restperfclient_Log Status code
BuiltIn.Should_Be_Equal '${result}' ''
-Deconfigure_Device_From_Netconf
+Cleanup_And_Collect
[Documentation] Deconfigure the testtool device on Netconf connector.
[Setup] SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+ RestPerfClient.Collect_From_Restperfclient
NetconfKeywords.Remove_Device_From_Netconf ${DEVICE_NAME}
*** Keywords ***
Setup_Everything
[Documentation] Setup everything needed for the test cases.
+ # Calculate and set the value of the timeout
+ ${value}= BuiltIn.Evaluate ${REQUEST_COUNT}/10+20
+ Utils.Set_User_Configurable_Variable_Default TESTTOOL_DEVICE_TIMEOUT ${value} s
# Setup resources used by the suite.
SetupUtils.Setup_Utils_For_Setup_And_Teardown
- RequestsLibrary.Create_Session operational http://${ODL_SYSTEM_IP}:${RESTCONFPORT}${OPERATIONAL_API} auth=${AUTH}
NetconfKeywords.Setup_Netconf_Keywords
- # Connect to the tools system (rest-perf-client)
- ${restperfclient}= SSHKeywords.Open_Connection_To_Tools_System
- BuiltIn.Set_Suite_Variable ${restperfclient} ${restperfclient}
- # Initialize artifact deployment infrastructure.
- NexusKeywords.Initialize_Artifact_Deployment_And_Usage
- ${testtool}= SSHLibrary.Get Connection
- BuiltIn.Set_Suite_Variable ${testtool} ${testtool.index}
+ Run Keyword If '${IS_KARAF_APPL}' == 'False' RestPerfClient.Setup_Restperfclient build_version=${NETCONF_TESTTOOL_VERSION} build_location=org/opendaylight/netconf
+ ... ELSE RestPerfClient.Setup_Restperfclient
+ # Connect to the tools system (testtool)
+ ${testtool}= SSHKeywords.Open_Connection_To_Tools_System
+ BuiltIn.Set_Suite_Variable ${testtool} ${testtool}
+ ${device_type}= BuiltIn.Set_Variable_If ${USE_NETCONF_CONNECTOR}==${True} default ${device_type}
+ BuiltIn.Set_Suite_Variable ${device_type}
Teardown_Everything
[Documentation] Teardown the test infrastructure, perform cleanup and release all resources.
- Teardown_Netconf_Via_Restconf
RequestsLibrary.Delete_All_Sessions
+ RestPerfClient.Teardown_Restperfclient
SSHLibrary.Switch_Connection ${testtool}
BuiltIn.Run_Keyword_And_Ignore_Error NetconfKeywords.Stop_Testtool
-
-Wait_Until_Prompt
- [Documentation] Wait until prompt appears or timeout occurs. When timeout occurs, send Ctrl-C and then wait for the prompt again.
- ... This is necessary because the restperfclient can crash and hang, requiring that Ctrl-C character to get rid of it for good.
- ${status} ${result}= BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Read_Until_Prompt
- Return_From_Keyword_If '${status}' == 'PASS'
- Utils.Write_Bare_Ctrl_C
- SSHLibrary.Read_Until_Prompt
Code Review / integration / test.git / blobdiff