From: Igor Foltin Date: Tue, 26 Sep 2017 15:15:59 +0000 (+0200) Subject: Update yang-model-validator test suite X-Git-Tag: release/carbon-sr2~23 X-Git-Url: https://git.opendaylight.org/gerrit/gitweb?p=integration%2Ftest.git;a=commitdiff_plain;h=28662568a7b982218f14604b977f83271238d05d Update yang-model-validator test suite Update this suite to reflect the changes that have been made in the YANG Tools artifact yang-model-validator (previously known as yang-system-test) since the test suite was first created. The -p option previously denoted the directories with YANG files that should be parsed as one big collection. However, the artifact is meant to be used as a single YANG file validator similarly to pyang. That is why the -p option is now used to specify directories with YANG dependencies, i.e. YANG files which might be needed to successfully validate the single YANG file specified as argument. Therefore, adjust the suite to iterate through the set of YANG files one by one while using the whole set of YANG models as dependencies within the -p option. Change-Id: I6225242bddf5dbf612a5e73153fe41c3ffa3f990 Signed-off-by: Igor Foltin --- diff --git a/csit/libraries/RemoteBash.robot b/csit/libraries/RemoteBash.robot index 9703bb3d57..e6c022c015 100644 --- a/csit/libraries/RemoteBash.robot +++ b/csit/libraries/RemoteBash.robot @@ -38,9 +38,11 @@ Write_Bare_Ctrl_D SSHLibrary.Write_Bare ${ctrl_d} Flush_Read + [Arguments] ${delay}=1 [Documentation] Attempt to read excess data (probably just multiple prompts), ignoring failure. ... Log the data or error message. Return None. - ${status} ${message} = BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Read + ... \${delay} parameter tunes how long a period of inactivity has to be to consider all excess data to be read. + ${status} ${message} = BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Read delay=${delay} BuiltIn.Log ${message} Abort_Execution diff --git a/csit/suites/yangtools/odl-yang-validator/odl-yang-validator.robot b/csit/suites/yangtools/yang-model-validator/yang-model-validator.robot similarity index 75% rename from csit/suites/yangtools/odl-yang-validator/odl-yang-validator.robot rename to csit/suites/yangtools/yang-model-validator/yang-model-validator.robot index 6be5916522..e15165988f 100644 --- a/csit/suites/yangtools/odl-yang-validator/odl-yang-validator.robot +++ b/csit/suites/yangtools/yang-model-validator/yang-model-validator.robot @@ -1,5 +1,5 @@ *** Settings *** -Documentation Suite for testing performance of odl-yang-validator utility. +Documentation Suite for testing performance of yang-model-validator utility. ... ... Copyright (c) 2016,2017 Cisco Systems, Inc. and others. All rights reserved. ... @@ -9,14 +9,14 @@ Documentation Suite for testing performance of odl-yang-validator utility. ... ... ... This suite measures time (only as a test case duration) needed -... for odl-yang-validator to execute on a set of yang model. +... for yang-model-validator to execute on a set of yang model. ... ... The set of Yang modules is large and fixed (no changes in future). ... It is the same set of models as in mdsal binding-parent suite. Suite Setup Setup_Suite Test Setup SetupUtils.Setup_Test_With_Logging_And_Fast_Failing # TODO: Suite Teardown to close SSH connections and other cleanup? Test Teardown Teardown_Test -Default Tags 1node odl-yang-validator critical +Default Tags 1node yang-model-validator critical Library RequestsLibrary Library SSHLibrary Library String @@ -40,24 +40,23 @@ Prepare_Yang_Files_To_Test YangCollection.Static_Set_As_Src ${dirs_to_process} = Get_Recursive_Dirs root=src/main/yang ${p_option_value} = BuiltIn.Catenate SEPARATOR=: @{dirs_to_process} + ${yang_files_to_validate} = Get_Yang_Files_From_Dirs ${dirs_to_process} + BuiltIn.Set_Suite_Variable ${yang_files_to_validate} BuiltIn.Set_Suite_Variable \${p_option_value} -Deploy_And_Start_Yang_System_Test_Utility - [Documentation] Download appropriate version of odl-yang-validator artifact and start it against the prepared set. +Deploy_And_Start_Odl_Yang_Validator_Utility + [Documentation] Download appropriate version of yang-model-validator artifact + ... and run it for each single yang file in the prepared set. ... The version is either given by ${EXPLICIT_YANG_SYSTEM_TEST_URL}, ... or constructed from Jenkins-shaped ${BUNDLE_URL}, or downloaded from Nexus based on ODL version. ${status} ${multipatch_url} = BuiltIn.Run_Keyword_And_Ignore_Error Construct_Multipatch_Url ${url} = Builtin.Set_Variable_If "${status}" == "PASS" ${multipatch_url} ${EXPLICIT_YANG_SYSTEM_TEST_URL} - ${logfile} = NexusKeywords.Install_And_Start_Java_Artifact component=yangtools artifact=odl-yang-validator suffix=jar-with-dependencies tool_options=-p ${p_option_value} explicit_url=${url} - BuiltIn.Set_Suite_Variable \${logfile} - -Wait_Until_Utility_Finishes - [Documentation] Repeatedly send endline to keep session alive; pass on prompt, fail on timeout. - RemoteBash.Wait_Without_Idle 60m - -Check_Return_Code - [Documentation] Get return code of previous command (the utility), pass if it is zero. - RemoteBash.Check_Return_Code + : FOR ${yang_file} IN @{yang_files_to_validate} + \ ${logfile} = NexusKeywords.Install_And_Start_Java_Artifact component=yangtools artifact=yang-model-validator suffix=jar-with-dependencies tool_options=-p ${p_option_value} ${yang_file} + \ ... explicit_url=${url} + \ BuiltIn.Set_Suite_Variable \${logfile} + \ Wait_Until_Utility_Finishes + \ Check_Return_Code Collect_Files_To_Archive [Documentation] Download created files so Releng scripts would archive it. @@ -90,10 +89,19 @@ Get_Recursive_Dirs ${flat_list} = Collections.Combine_Lists ${depth_1} @{subtrees} [Return] ${flat_list} +Get_Yang_Files_From_Dirs + [Arguments] ${dirs_to_process} + [Documentation] Return list of yang files from provided directories + ${collected_yang_files} = BuiltIn.Create_List + : FOR ${dir} IN @{dirs_to_process} + \ ${yang_files_in_dir} = SSHLibrary.List_Files_In_Directory path=${dir} pattern=*.yang absolute=True + \ ${collected_yang_files} = Collections.Combine_Lists ${collected_yang_files} ${yang_files_in_dir} + [Return] ${collected_yang_files} + Construct_Multipatch_Url [Documentation] If ${EXPLICIT_YANG_SYSTEM_TEST_URL} is non-empty, return it. Otherwise: ... Check whether ${BUNDLE_URL} is from multipatch build (or similar maven style job), - ... Check whether odl-yang-validator was built there as well, + ... Check whether yang-model-validator was built there as well, ... and return URL with proper version, or fail. BuiltIn.Return_From_Keyword_If """${EXPLICIT_YANG_SYSTEM_TEST_URL}""" != "" ${EXPLICIT_YANG_SYSTEM_TEST_URL} ${marker} = BuiltIn.Set_Variable /org.opendaylight.integration$distribution-karaf @@ -101,10 +109,18 @@ Construct_Multipatch_Url BuiltIn.Should_Be_True ${is_multipatch} ${yst_base_url} = String.Fetch_From_Left ${BUNDLE_URL} ${marker} RequestsLibrary.Create_Session alias=cmu url=${yst_base_url} - ${yst_general_uri} = BuiltIn.Set_Variable org.opendaylight.yangtools$odl-yang-validator/artifact/org.opendaylight.yangtools/odl-yang-validator + ${yst_general_uri} = BuiltIn.Set_Variable org.opendaylight.yangtools$yang-model-validator/artifact/org.opendaylight.yangtools/yang-model-validator ${yst_html} = TemplatedRequests.Get_From_Uri ${yst_general_uri} session=cmu # The following two lines are very specific to a particular Jenkins html layout. ${yst_almost_version} = String.Fetch_From_Right ${yst_html} - ${url} = BuiltIn.Set_Variable ${yst_base_url}/${yst_general_uri}/${yst_version}/odl-yang-validator-${yst_version}-jar-with-dependencies.jar + ${url} = BuiltIn.Set_Variable ${yst_base_url}/${yst_general_uri}/${yst_version}/yang-model-validator-${yst_version}-jar-with-dependencies.jar BuiltIn.Return_From_Keyword ${url} + +Wait_Until_Utility_Finishes + [Documentation] Repeatedly send endline to keep session alive; pass on prompt, fail on timeout. + RemoteBash.Wait_Without_Idle 60m + +Check_Return_Code + [Documentation] Get return code of previous command (the utility), pass if it is zero. + RemoteBash.Check_Return_Code diff --git a/csit/testplans/yangtools-system.txt b/csit/testplans/yangtools-system.txt index cc48d58d11..e81d4f50af 100644 --- a/csit/testplans/yangtools-system.txt +++ b/csit/testplans/yangtools-system.txt @@ -1,2 +1,2 @@ # Place the suites in run order: -integration/test/csit/suites/yangtools/odl-yang-validator/odl-yang-validator.robot +integration/test/csit/suites/yangtools/yang-model-validator/yang-model-validator.robot