From: Jozef Behran Date: Mon, 14 Dec 2015 10:15:16 +0000 (+0100) Subject: Put testtool startup into its own testcase X-Git-Tag: release/beryllium-sr1~234 X-Git-Url: https://git.opendaylight.org/gerrit/gitweb?p=integration%2Ftest.git;a=commitdiff_plain;h=9b627cf3dff6c022c6ae754a657c1a9b6103c801 Put testtool startup into its own testcase The Install_And_Start_Testtool contains an attempt to wait for testtool to become usable. This is done by waiting for the first test tool device to come online. This was done with assumption that further testtool devices will come online very quickly after the first one. However it turns out the test tool startup may take considerable amount of time even after this first device comes online. Thus the "the remaining testtool devices come online quickly" assumption is not valid and overlapping this time consuming operation with the device configuration test case in the scaling test suite causes the test case to be blocked waiting for testtool to supply the next device, destroying the validity of its results. To fix this amend the "wait for testtool to boot" action to make sure that all testtool devices are online and then move the testtool startup into the first test case in the suite as it now can fail. Change-Id: Ie883d375e95e51d78c8343133182c644bde6e1df Signed-off-by: Jozef Behran --- diff --git a/csit/libraries/NetconfKeywords.robot b/csit/libraries/NetconfKeywords.robot index 74f6b8cda9..5c1237084f 100644 --- a/csit/libraries/NetconfKeywords.robot +++ b/csit/libraries/NetconfKeywords.robot @@ -24,6 +24,7 @@ ${DIRECTORY_WITH_DEVICE_TEMPLATES} ${CURDIR}/../variables/netconf/device ${FIRST_TESTTOOL_PORT} 17830 ${BASE_NETCONF_DEVICE_PORT} 17830 ${DEVICE_NAME_BASE} netconf-scaling-device +${TESTTOOL_DEVICE_TIMEOUT} 60s *** Keywords *** Setup_NetconfKeywords @@ -125,6 +126,11 @@ NetconfKeywords__Check_Device_Is_Up ${count}= SSHKeywords.Count_Port_Occurences ${last-port} LISTEN java BuiltIn.Should_Be_Equal_As_Integers ${count} 1 +NetconfKeywords__Wait_Device_Is_Up_And_Running + [Arguments] ${device_name} + ${number}= BuiltIn.Evaluate '${device_name}'.split('-').pop() + BuiltIn.Wait_Until_Keyword_Succeeds ${TESTTOOL_DEVICE_TIMEOUT} 1s Check_Device_Up_And_Running ${number} + Install_And_Start_Testtool [Arguments] ${device-count}=10 ${debug}=true ${schemas}=none ${tool_options}=${EMPTY} ${java_options}=${TESTTOOL_DEFAULT_JAVA_OPTIONS} [Documentation] Install and run testtool. Also arrange to collect its output into a log file. @@ -139,11 +145,10 @@ Install_And_Start_Testtool ${command} BuiltIn.Set_Variable java ${java_options} -jar ${filename} ${tool_options} --device-count ${device-count} --debug ${debug} ${schemas_option} BuiltIn.Log Running testtool: ${command} SSHLibrary.Write ${command} >testtool.log 2>&1 - # Wait for the testtool to boot up. - ${timeout}= BuiltIn.Evaluate (${device-count}/3)+5 - BuiltIn.Wait_Until_Keyword_Succeeds ${timeout}s 1s NetconfKeywords__Check_Device_Is_Up ${FIRST_TESTTOOL_PORT} # Store information needed by other keywords. BuiltIn.Set_Suite_Variable ${NetconfKeywords__testtool_device_count} ${device-count} + # Wait for the testtool to boot up. + Perform_Operation_On_Each_Device NetconfKeywords__Wait_Device_Is_Up_And_Running Check_Device_Up_And_Running [Arguments] ${device-number} @@ -167,7 +172,6 @@ NetconfKeywords__Perform_Operation_With_Checking_On_Next_Device ${ellapsed_seconds}= DateTime.Subtract_Date_From_Date ${deadline_Date} ${current_Date} BuiltIn.Run_Keyword_If ${ellapsed_seconds}<0 Fail The global time out period expired ${number}= BuiltIn.Evaluate ${current_port}-${BASE_NETCONF_DEVICE_PORT}+1 - BuiltIn.Wait_Until_Keyword_Succeeds 10s 1s NetconfKeywords.Check_Device_Up_And_Running ${number} BuiltIn.Run_Keyword ${operation} ${DEVICE_NAME_BASE}-${number} ${next}= BuiltIn.Evaluate ${current_port}+1 BuiltIn.Set_Suite_Variable ${current_port} ${next} diff --git a/csit/suites/netconf/CRUD/CRUD.robot b/csit/suites/netconf/CRUD/CRUD.robot index ed14c3a6bc..b0140ce696 100644 --- a/csit/suites/netconf/CRUD/CRUD.robot +++ b/csit/suites/netconf/CRUD/CRUD.robot @@ -32,6 +32,10 @@ ${DIRECTORY_WITH_TEMPLATE_FOLDERS} ${CURDIR}/../../../variables/netconf/CRUD ${device_name} netconf-test-device *** Test Cases *** +Start_Testtool + [Documentation] Deploy and start test tool, then wait for all its devices to become online. + NetconfKeywords.Install_And_Start_Testtool device-count=1 schemas=${CURDIR}/../../../variables/netconf/CRUD/schemas + Check_Device_Is_Not_Configured_At_Beginning [Documentation] Sanity check making sure our device is not there. Fail if found. [Tags] critical @@ -111,7 +115,6 @@ Setup_Everything # Connect to the Mininet machine SSHLibrary.Open_Connection ${TOOLS_SYSTEM_IP} prompt=${TOOLS_SYSTEM_PROMPT} Utils.Flexible_Mininet_Login - NetconfKeywords.Install_And_Start_Testtool device-count=10 schemas=${CURDIR}/../../../variables/netconf/CRUD/schemas Teardown_Everything [Documentation] Teardown the test infrastructure, perform cleanup and release all resources. diff --git a/csit/suites/netconf/scale/getmulti.robot b/csit/suites/netconf/scale/getmulti.robot index 74d4042ad7..ef63e4262a 100644 --- a/csit/suites/netconf/scale/getmulti.robot +++ b/csit/suites/netconf/scale/getmulti.robot @@ -32,6 +32,10 @@ ${device_name_base} netconf-scaling-device ${base_port} 17830 *** Test Cases *** +Start_Test_Tool + [Documentation] Deploy and start test tool, then wait for all its devices to become online. + NetconfKeywords.Install_And_Start_Testtool device-count=${DEVICE_COUNT} + Configure_Devices_On_Netconf [Documentation] Make requests to configure the testtool devices. ${timeout}= BuiltIn.Evaluate ${DEVICE_COUNT}*10 @@ -76,8 +80,7 @@ Setup_Everything # Connect to the tools machine SSHLibrary.Open_Connection ${TOOLS_SYSTEM_IP} Utils.Flexible_Mininet_Login - # Deploy testtool on it - NetconfKeywords.Install_And_Start_Testtool device-count=${DEVICE_COUNT} + # Deploy testing tools on it. SSHLibrary.Put_File ${CURDIR}/../../../../tools/netconf_tools/getter.py SSHLibrary.Put_File ${CURDIR}/../../../libraries/AuthStandalone.py diff --git a/csit/suites/netconf/scale/getsingle.robot b/csit/suites/netconf/scale/getsingle.robot index 9ef0e3de77..8931c9bc2f 100644 --- a/csit/suites/netconf/scale/getsingle.robot +++ b/csit/suites/netconf/scale/getsingle.robot @@ -26,6 +26,10 @@ Variables ${CURDIR}/../../../variables/Variables.py ${DEVICE_COUNT} 500 *** Test Cases *** +Start_Test_Tool + [Documentation] Deploy and start test tool, then wait for all its devices to become online. + NetconfKeywords.Install_And_Start_Testtool device-count=${DEVICE_COUNT} + Configure_Devices_Onto_Netconf [Documentation] Make requests to configure the testtool devices. [Tags] critical