2 Documentation Suite for testing performance of yang-model-validator utility.
4 ... Copyright (c) 2016,2017 Cisco Systems, Inc. and others. All rights reserved.
6 ... This program and the accompanying materials are made available under the
7 ... terms of the Eclipse Public License v1.0 which accompanies this distribution,
8 ... and is available at http://www.eclipse.org/legal/epl-v10.html
11 ... This suite executes the yang-model-validator tool and will turn up any major
12 ... breakages in that tool. Since yangtools is now a release integrated project
13 ... and the version of the tool is static and unchanging per release, this suite
14 ... does not need to run very often.
16 ... Two main things to check for this suite and the yang-model-validator tool:
18 ... 1) Does it work against the updated yang model repos (see YangCollection.robot)
19 ... and report valid issues in those models. When the models are updated, does
20 ... the tool still work as expected.
22 ... 2) What does the runtime of the tool look like as new versions of the tool are
23 ... released? Does validation take significanltly shorter (an improvement) or
24 ... longer (a regression)?
26 ... The set of Yang modules is large and fixed to specific commits from their relevant
27 ... repos. That fixed point can be updated periodically in the YangCollection.robot
28 ... library. Just be sure there is an apples to apples comparision (same exact repo
29 ... state) between yangtools releases, so #2 above is known.
31 Suite Setup Setup_Suite
32 Test Setup SetupUtils.Setup_Test_With_Logging_And_Fast_Failing
33 Test Teardown Teardown_Test
34 Default Tags 1node yang-model-validator critical
35 Library RequestsLibrary
38 Resource ${CURDIR}/../../../libraries/CompareStream.robot
39 Resource ${CURDIR}/../../../libraries/NexusKeywords.robot
40 Resource ${CURDIR}/../../../libraries/RemoteBash.robot
41 Resource ${CURDIR}/../../../libraries/SetupUtils.robot
42 Resource ${CURDIR}/../../../libraries/SSHKeywords.robot
43 Resource ${CURDIR}/../../../libraries/TemplatedRequests.robot
44 Resource ${CURDIR}/../../../libraries/YangCollection.robot
47 ${TEST_TOOL_NAME} yang-model-validator
48 ${EXPLICIT_YANG_SYSTEM_TEST_URL} ${EMPTY}
52 [Documentation] The ODL instance consumes resources, kill it.
53 ClusterManagement.Kill_Members_From_List_Or_All
55 Prepare_Yang_Files_To_Test
56 [Documentation] Set up collection of Yang files to test with, manually deleting any files/paths
57 ... that have known breakages that are issues with the models (not the validator tool).
58 YangCollection.Static_Set_As_Src
59 YangCollection.Delete_Static_Paths
61 Deploy_And_Start_Odl_Yang_Validator_Utility
62 [Documentation] Download appropriate version of ${TEST_TOOL_NAME} artifact
63 ... and run it for each single yang file in the prepared set.
64 ... The version is either given by ${EXPLICIT_YANG_SYSTEM_TEST_URL},
65 ... or constructed from Jenkins-shaped ${BUNDLE_URL}, or downloaded from Nexus based on ODL version.
66 ${dirs_to_process} = Get_Recursive_Dirs root=src/main/yang
67 ${yang_files_to_validate} = Get_Yang_Files_From_Dirs ${dirs_to_process}
68 ${yang_path_option} = Get_Yang_Model_Validator_Path_Option ${YANG_MODEL_PATHS}
69 FOR ${yang_file} IN @{yang_files_to_validate}
70 Log To Console working on: ${yang_file}
71 ${logfile} = NexusKeywords.Install_And_Start_Java_Artifact component=yangtools artifact=${TEST_TOOL_NAME}
72 ... suffix=jar-with-dependencies tool_options=${yang_path_option} -- ${yang_file} explicit_url=${EXPLICIT_YANG_SYSTEM_TEST_URL}
73 Wait_Until_Utility_Finishes
76 [Teardown] BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Get_File ${logfile}
80 [Documentation] Activate dependency Resources, create SSH connection.
81 SetupUtils.Setup_Utils_For_Setup_And_Teardown
82 NexusKeywords.Initialize_Artifact_Deployment_And_Usage tools_system_connect=False
83 SSHKeywords.Open_Connection_To_ODL_System
86 [Documentation] Make sure CWD is set back to dot, then proceed with SetupUtils stuff.
88 SetupUtils.Teardown_Test_Show_Bugs_And_Start_Fast_Failing_If_Test_Failed
92 [Documentation] Return list of sub-directories discovered recursively under ${root} relative to
93 ... the current working directory for a new shell spawned over the active SSH session.
94 ... This implementation returns absolute paths as that is easier.
95 ${depth_1} = SSHLibrary.List_Directories_In_Directory path=${root} absolute=True
96 ${subtrees} = BuiltIn.Create_List
97 FOR ${subdir} IN @{depth_1}
98 ${tree} = Get_Recursive_Dirs root=${subdir}
99 # Relative paths would require prepending ${subdir}${/} to each @{tree} element.
100 Collections.Append_To_List ${subtrees} ${tree}
102 ${flat_list} = Collections.Combine_Lists ${depth_1} @{subtrees}
103 [Return] ${flat_list}
105 Get_Yang_Files_From_Dirs
106 [Arguments] ${dirs_to_process}
107 [Documentation] Return list of yang files from provided directories
108 ${collected_yang_files} = BuiltIn.Create_List
109 FOR ${dir} IN @{dirs_to_process}
110 ${yang_files_in_dir} = SSHLibrary.List_Files_In_Directory path=${dir} pattern=*.yang absolute=True
111 ${collected_yang_files} = Collections.Combine_Lists ${collected_yang_files} ${yang_files_in_dir}
113 [Return] ${collected_yang_files}
115 Get_Yang_Model_Validator_Path_Option
116 [Arguments] ${yang_paths}
117 [Documentation] Return the path option for yang-model-validator from the provided list of YANG paths.
118 ${separator} = CompareStream.Set_Variable_If_At_Most_Sulfur : ${SPACE}
119 ${path_option} = Evaluate "${separator}".join(${yang_paths})
120 ${path_option} = Catenate SEPARATOR=${SPACE} --path ${path_option}
121 [Return] ${path_option}
123 Wait_Until_Utility_Finishes
124 [Documentation] Repeatedly send endline to keep session alive; pass on prompt, fail on timeout.
125 RemoteBash.Wait_Without_Idle 60m
128 [Documentation] Get return code of previous command (the utility), pass if it is zero.
129 RemoteBash.Check_Return_Code