2 Documentation Suite for testing performance of yang-model-validator utility.
4 ... Copyright (c) 2016,2017 Cisco Systems, Inc. and others. All rights reserved.
6 ... This program and the accompanying materials are made available under the
7 ... terms of the Eclipse Public License v1.0 which accompanies this distribution,
8 ... and is available at http://www.eclipse.org/legal/epl-v10.html
11 ... This suite executes the yang-model-validator tool and will turn up any major
12 ... breakages in that tool. Since yangtools is now a release integrated project
13 ... and the version of the tool is static and unchanging per release, this suite
14 ... does not need to run very often.
16 ... Two main things to check for this suite and the yang-model-validator tool:
18 ... 1) Does it work against the updated yang model repos (see YangCollection.robot)
19 ... and report valid issues in those models. When the models are updated, does
20 ... the tool still work as expected.
22 ... 2) What does the runtime of the tool look like as new versions of the tool are
23 ... released? Does validation take significanltly shorter (an improvement) or
24 ... longer (a regression)?
26 ... The set of Yang modules is large and fixed to specific commits from their relevant
27 ... repos. That fixed point can be updated periodically in the YangCollection.robot
28 ... library. Just be sure there is an apples to apples comparision (same exact repo
29 ... state) between yangtools releases, so #2 above is known.
32 Library RequestsLibrary
35 Resource ${CURDIR}/../../../libraries/CompareStream.robot
36 Resource ${CURDIR}/../../../libraries/NexusKeywords.robot
37 Resource ${CURDIR}/../../../libraries/RemoteBash.robot
38 Resource ${CURDIR}/../../../libraries/SetupUtils.robot
39 Resource ${CURDIR}/../../../libraries/SSHKeywords.robot
40 Resource ${CURDIR}/../../../libraries/TemplatedRequests.robot
41 Resource ${CURDIR}/../../../libraries/YangCollection.robot
43 Suite Setup Setup_Suite
44 Test Setup SetupUtils.Setup_Test_With_Logging_And_Fast_Failing
45 Test Teardown Teardown_Test
47 Default Tags 1node yang-model-validator critical
51 ${TEST_TOOL_NAME} yang-model-validator
52 ${EXPLICIT_YANG_SYSTEM_TEST_URL} ${EMPTY}
57 [Documentation] The ODL instance consumes resources, kill it.
58 ClusterManagement.Kill_Members_From_List_Or_All
60 Prepare_Yang_Files_To_Test
61 [Documentation] Set up collection of Yang files to test with, manually deleting any files/paths
62 ... that have known breakages that are issues with the models (not the validator tool).
63 YangCollection.Static_Set_As_Src
64 YangCollection.Delete_Static_Paths
66 Deploy_And_Start_Odl_Yang_Validator_Utility
67 [Documentation] Download appropriate version of ${TEST_TOOL_NAME} artifact
68 ... and run it for each single yang file in the prepared set.
69 ... The version is either given by ${EXPLICIT_YANG_SYSTEM_TEST_URL},
70 ... or constructed from Jenkins-shaped ${BUNDLE_URL}, or downloaded from Nexus based on ODL version.
71 ${dirs_to_process} = Get_Recursive_Dirs root=src/main/yang
72 ${yang_files_to_validate} = Get_Yang_Files_From_Dirs ${dirs_to_process}
73 ${yang_path_option} = Get_Yang_Model_Validator_Path_Option ${YANG_MODEL_PATHS}
74 FOR ${yang_file} IN @{yang_files_to_validate}
75 Log To Console working on: ${yang_file}
76 ${logfile} = NexusKeywords.Install_And_Start_Java_Artifact
77 ... component=yangtools
78 ... artifact=${TEST_TOOL_NAME}
79 ... suffix=jar-with-dependencies
80 ... tool_options=${yang_path_option} -- ${yang_file}
81 ... explicit_url=${EXPLICIT_YANG_SYSTEM_TEST_URL}
82 Wait_Until_Utility_Finishes
85 [Teardown] BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Get_File ${logfile}
90 [Documentation] Activate dependency Resources, create SSH connection.
91 SetupUtils.Setup_Utils_For_Setup_And_Teardown
92 NexusKeywords.Initialize_Artifact_Deployment_And_Usage tools_system_connect=False
93 SSHKeywords.Open_Connection_To_ODL_System
96 [Documentation] Make sure CWD is set back to dot, then proceed with SetupUtils stuff.
98 SetupUtils.Teardown_Test_Show_Bugs_And_Start_Fast_Failing_If_Test_Failed
101 [Documentation] Return list of sub-directories discovered recursively under ${root} relative to
102 ... the current working directory for a new shell spawned over the active SSH session.
103 ... This implementation returns absolute paths as that is easier.
104 [Arguments] ${root}=.
105 ${depth_1} = SSHLibrary.List_Directories_In_Directory path=${root} absolute=True
106 ${subtrees} = BuiltIn.Create_List
107 FOR ${subdir} IN @{depth_1}
108 ${tree} = Get_Recursive_Dirs root=${subdir}
109 # Relative paths would require prepending ${subdir}${/} to each @{tree} element.
110 Collections.Append_To_List ${subtrees} ${tree}
112 ${flat_list} = Collections.Combine_Lists ${depth_1} @{subtrees}
115 Get_Yang_Files_From_Dirs
116 [Documentation] Return list of yang files from provided directories
117 [Arguments] ${dirs_to_process}
118 ${collected_yang_files} = BuiltIn.Create_List
119 FOR ${dir} IN @{dirs_to_process}
120 ${yang_files_in_dir} = SSHLibrary.List_Files_In_Directory path=${dir} pattern=*.yang absolute=True
121 ${collected_yang_files} = Collections.Combine_Lists ${collected_yang_files} ${yang_files_in_dir}
123 RETURN ${collected_yang_files}
125 Get_Yang_Model_Validator_Path_Option
126 [Documentation] Return the path option for yang-model-validator from the provided list of YANG paths.
127 [Arguments] ${yang_paths}
128 ${separator} = CompareStream.Set_Variable_If_At_Most_Sulfur : ${SPACE}
129 ${path_option} = Evaluate "${separator}".join(${yang_paths})
130 ${path_option} = Catenate SEPARATOR=${SPACE} --path ${path_option}
131 RETURN ${path_option}
133 Wait_Until_Utility_Finishes
134 [Documentation] Repeatedly send endline to keep session alive; pass on prompt, fail on timeout.
135 RemoteBash.Wait_Without_Idle 60m
138 [Documentation] Get return code of previous command (the utility), pass if it is zero.
139 RemoteBash.Check_Return_Code