2 Documentation Nexus repository access keywords, and supporting Java and Maven handling.
4 ... Copyright (c) 2015,2016 Cisco Systems, Inc. and others. All rights reserved.
6 ... This program and the accompanying materials are made available under the
7 ... terms of the Eclipse Public License v1.0 which accompanies this distribution,
8 ... and is available at http://www.eclipse.org/legal/epl-v10.html
11 ... This library encapsulates a bunch of somewhat complex and commonly used
12 ... Nexus operations into reusable keywords to make writing test suites easier.
14 ... Currently, Java version detection is incorporated so that Java tools can be run reliably.
15 ... Also, suport for installing and running Maven is added, as that needs the Java detection.
16 ... TODO: Move Java detection and Maven to a separate Resource, or rename this Resource.
17 Library OperatingSystem
20 Resource ${CURDIR}/SSHKeywords.robot
24 ${JAVA_7_HOME_CENTOS} /usr/lib/jvm/java-1.7.0
25 ${JAVA_7_HOME_UBUNTU} /usr/lib/jvm/java-7-openjdk-amd64
26 ${JAVA_8_HOME_CENTOS} /usr/lib/jvm/java-1.8.0
27 ${JAVA_8_HOME_UBUNTU} /usr/lib/jvm/java-8-openjdk-amd64
28 ${JAVA_OPTIONS} -Xmx2560m # Note that '-Xmx=3g' is wrong syntax. Also 3GB heap may not fit in 4GB RAM.
29 ${JAVA_7_OPTIONS} -Xmx2048m -XX:MaxPermSize=512m
30 ${MAVEN_DEFAULT_OUTPUT_FILENAME} default_maven.log
31 ${MAVEN_OPTIONS} -Pq -Djenkins
32 ${MAVEN_REPOSITORY_PATH} /tmp/r
33 ${MAVEN_SETTINGS_URL} https://raw.githubusercontent.com/opendaylight/odlparent/master/settings.xml
34 ${MAVEN_VERSION} 3.3.9
35 ${NEXUS_FALLBACK_URL} ${NEXUSURL_PREFIX}/content/repositories/opendaylight.snapshot
38 Initialize_Artifact_Deployment_And_Usage
39 [Arguments] ${tools_system_connect}=True
40 [Documentation] Places search utility to ODL system, which will be needed for version detection.
41 ... By default also initialize a SSH connection to Tools system,
42 ... as following Keywords assume a working connection towards target system.
43 # Connect to the ODL machine.
44 ${odl} = SSHKeywords.Open_Connection_To_ODL_System
45 # Deploy the search tool.
46 SSHLibrary.Put_File ${CURDIR}/../../tools/deployment/search.sh
47 SSHLibrary.Close_Connection
48 # Optionally connect to the Tools System machine.
49 BuiltIn.Return_From_Keyword_If not (${tools_system_connect}) # the argument may be a convoluted Python expression
50 SSHKeywords.Open_Connection_To_Tools_System
52 NexusKeywords__Get_Items_To_Look_At
53 [Arguments] ${component}
54 [Documentation] Get a list of items that might contain the version number that we are looking for.
55 BuiltIn.Return_From_Keyword_If '${component}' == 'bgpcep' pcep-impl
56 [Return] ${component}-impl
58 NexusKeywords__Detect_Version_To_Pull
59 [Arguments] ${component}
60 [Documentation] Determine the exact Nexus directory to be used as a source for a particular test tool
61 ... Figure out what version of the tool needs to be pulled out of the
62 ... Nexus by looking at the version directory of the subsystem from
63 ... which the tool is being pulled. This code is REALLY UGLY but there
64 ... is no way around it until the bug
65 ... https://bugs.opendaylight.org/show_bug.cgi?id=5206 gets fixed.
66 ... I also don't want to depend on maven-metadata-local.xml and other
67 ... bits and pieces of ODL distribution which are not required for ODL
68 ... to function properly.
69 ${itemlist} = NexusKeywords__Get_Items_To_Look_At ${component}
70 ${current_ssh_connection} = SSHLibrary.Get Connection
71 SSHKeywords.Open_Connection_To_ODL_System
72 ${version} ${result} = SSHLibrary.Execute_Command sh search.sh ${WORKSPACE}/${BUNDLEFOLDER}/system ${itemlist} return_rc=True
73 SSHLibrary.Close_Connection
74 SSHKeywords.Restore Current SSH Connection From Index ${current_ssh_connection.index}
75 BuiltIn.Log ${version}
76 BuiltIn.Run_Keyword_If ${result}!=0 BuiltIn.Fail Component "${component}" not found, cannot locate test tool
77 ${version} ${location} = String.Split_String ${version} max_split=1
78 [Return] ${version} ${location}
81 [Arguments] ${component} ${artifact} ${name_prefix} ${name_suffix}=-executable.jar ${fallback_url}=${NEXUS_FALLBACK_URL}
82 [Documentation] Deploy the specified artifact from Nexus to the cwd of the machine to which the active SSHLibrary connection points.
83 ... Must have ${BUNDLE_URL} variable set to the URL from which the
84 ... tested ODL distribution was downloaded and this place must be
85 ... inside a repository created by a standard distribution
86 ... construction job. If this is detected to ne be the case, fallback URL is used.
87 ${urlbase} = String.Fetch_From_Left ${BUNDLE_URL} /org/opendaylight
88 # If the BUNDLE_URL points somewhere else (perhaps *patch-test* job in Jenkins),
89 # ${urlbase} is the whole ${BUNDLE_URL}, in which case we use the ${fallback_url}
90 ${urlbase} = BuiltIn.Set_Variable_If '${urlbase}' != '${BUNDLE_URL}' ${urlbase} ${fallback_url}
91 ${version} ${location} = NexusKeywords__Detect_Version_To_Pull ${component}
92 # TODO: Use RequestsLibrary and String instead of curl and bash utilities?
93 ${url} = BuiltIn.Set_Variable ${urlbase}/${location}/${artifact}/${version}
94 ${metadata}= SSHKeywords.Execute_Command_Should_Pass curl -L ${url}/maven-metadata.xml
95 ${namepart}= SSHKeywords.Execute_Command_Should_Pass echo "${metadata}" | grep value | head -n 1 | cut -d '>' -f 2 | cut -d '<' -f 1 stderr_must_be_empty=${True}
96 ${length} = BuiltIn.Get_Length ${namepart}
97 ${namepart} = BuiltIn.Set_Variable_If ${length} == 0 ${version} ${namepart}
98 ${filename} = BuiltIn.Set_Variable ${name_prefix}${namepart}${name_suffix}
99 BuiltIn.Log ${filename}
100 ${url} = BuiltIn.Set_Variable ${url}/${filename}
101 ${response} ${result} = SSHLibrary.Execute_Command wget -q -N ${url} 2>&1 return_rc=True
102 BuiltIn.Log ${response}
103 BuiltIn.Run_Keyword_If ${result} != 0 BuiltIn.Fail Artifact "${artifact}" in component "${component}" could not be downloaded from ${url}
107 [Arguments] ${component} ${artifact} ${suffix}=executable
108 [Documentation] Deploy a test tool.
109 ... The test tools have naming convention of the form
110 ... "<repository_url>/some/dir/somewhere/<tool-name>/<tool-name>-<version-tag>-${suffix}.jar"
111 ... where "<tool-name>" is the name of the tool and "<version-tag>" is
112 ... the version tag that is digged out of the maven metadata. This
113 ... keyword calculates ${name_prefix} and ${name_suffix} for
114 ... "Deploy_Artifact" and then calls "Deploy_Artifact" to do the real
115 ... work of deploying the artifact.
116 ${name_prefix} = BuiltIn.Set_Variable ${artifact}-
117 ${name_suffix} = BuiltIn.Set_Variable -${suffix}.jar
118 ${filename} = Deploy_Artifact ${component} ${artifact} ${name_prefix} ${name_suffix}
121 Compose_Dilemma_Filepath
122 [Arguments] ${default_path} ${specific_path}
123 [Documentation] Query active SSH connection, return specific path if it exists else default path.
124 ${out} ${rc} = SSHLibrary.Execute_Command ls -lA ${specific_path} 2>&1 return_rc=True
125 BuiltIn.Return_From_Keyword_If ${rc} == 0 ${specific_path}
126 BuiltIn.Return_From_Keyword ${default_path}
128 Compose_Base_Java_Command
129 [Arguments] ${openjdk}=${JDKVERSION}
130 [Documentation] Return string suitable for launching Java programs over SSHLibrary, depending on JRE version needed.
131 ... This requires that the SSH connection on which the command is going to be used is active as it is needed for querying files.
132 ... Commands composed for one SSH connection shall not be reused on other SSH connections as the two connections may have different Java setups.
133 ... Not directly related to Nexus, but versioned Java tools may need this.
134 # Check whether the user set the override and return it if yes.
135 BuiltIn.Run_Keyword_And_Return_If """${openjdk}""" == "openjdk8" Compose_Dilemma_Filepath ${JAVA_8_HOME_CENTOS}/bin/java ${JAVA_8_HOME_UBUNTU}/bin/java
136 BuiltIn.Run_Keyword_And_Return_If """${openjdk}""" == "openjdk7" Compose_Dilemma_Filepath ${JAVA_7_HOME_CENTOS}/bin/java ${JAVA_7_HOME_UBUNTU}/bin/java
137 # Attempt to call plain "java" command directly. If it works, return it.
138 ${out} ${rc} = SSHLibrary.Execute_Command java -version 2>&1 return_rc=True
139 BuiltIn.Return_From_Keyword_If ${rc} == 0 java
140 # Query the virtual machine for the JAVA_HOME environment variable and
141 # use it to assemble a (hopefully) working command. If that worked out,
143 ${java} = SSHLibrary.Execute_Command echo \$JAVA_HOME/bin/java 2>&1
144 ${out} ${rc} = SSHLibrary.Execute_Command ${java} -version 2>&1 return_rc=True
145 BuiltIn.Return_From_Keyword_If ${rc} == 0 ${java}
146 # There are bizzare test environment setups where the (correct) JAVA_HOME
147 # is set in the VM where Robot is running but not in the VM where the
148 # tools are supposed to run (usually because these two are really one
149 # and the same system and idiosyncracies of BASH prevent easy injection
150 # of the JAVA_HOME environment variable into a place where connections
151 # made by SSHLibrary would pick it up). So try to use that value to
152 # create a java command and check that it works.
153 ${JAVA_HOME} = OperatingSystem.Get_Environment_Variable JAVA_HOME ${EMPTY}
154 ${java} = BuiltIn.Set_Variable_If """${JAVA_HOME}"""!="" ${JAVA_HOME}/bin/java false
155 ${out} ${rc} = SSHLibrary.Execute_Command ${java} -version 2>&1 return_rc=True
156 BuiltIn.Return_From_Keyword_If ${rc} == 0 ${java}
157 # Nothing works, most likely java is not installed at all on the target
158 # machine or it is hopelesly lost. Bail out with a helpful message
159 # telling the user how to make it accessible for the script.
160 BuiltIn.Fail Unable to find Java; specify \${JDKVERSION}, put it to your PATH or set JAVA_HOME environment variable.
162 Compose_Full_Java_Command
163 [Arguments] ${options} ${openjdk}=${JDKVERSION}
164 [Documentation] Return full Bash command to run Java with given options.
165 ... This requires that the SSH connection on which the command is going to be used is active as it is needed for querying files.
166 ... The options may include JVM options, application command line arguments, Bash redirects and other constructs.
167 ${base_command} = Compose_Base_Java_Command openjdk=${openjdk}
168 ${full_command} = BuiltIn.Set_Variable ${base_command} ${options}
169 BuiltIn.Log ${full_command}
170 [Return] ${full_command}
173 [Arguments] ${openjdk}=${JDKVERSION}
174 [Documentation] Compose base java command and strip trailing "/bin/java".
175 ${java_command} = Compose_Base_Java_Command
176 ${java_home} ${bin} ${java} = String.Split_String_From_Right ${java_command} separator=/ max_split=2
177 [Return] ${java_home}
180 [Arguments] ${maven_version}=3.3.9 ${openjdk}=${JDKVERSION}
181 [Documentation] Download and unpack Maven, prepare launch command with proper Java version and download settings file.
182 ... This Keyword requires an active SSH connection to target machine.
183 ... This Keyword sets global variables, so that suites can reuse existing installation.
184 ... This Keyword can only place Maven (and settings) to remote current working directory.
185 ... This Keyword does not perform any initial or final cleanup.
186 # Avoid multiple initialization by several downstream libraries.
187 ${installed_version} = BuiltIn.Get_Variable_Value \${Maven__installed_version} None
188 BuiltIn.Return_From_Keyword_If """${installed_version}""" == """${maven_version}"""
189 BuiltIn.Set_Global_Variable \${Maven__installed_version} ${maven_version}
190 BuiltIn.Set_Global_Variable \${maven_directory} apache-maven-${maven_version}
191 SSHKeywords.Execute_Command_At_Cwd_Should_Pass rm -rf '${maven_directory}'
192 ${maven_archive_filename} = BuiltIn.Set_Variable ${maven_directory}-bin.tar.gz
193 ${maven_download_url} = BuiltIn.Set_Variable http://www-us.apache.org/dist/maven/maven-3/${maven_version}/binaries/${maven_archive_filename}
194 SSHKeywords.Execute_Command_At_Cwd_Should_Pass wget -N '${maven_download_url}' stderr_must_be_empty=False
195 SSHKeywords.Execute_Command_At_Cwd_Should_Pass tar xvf '${maven_archive_filename}'
196 ${java_home} = NexusKeywords.Compose_Java_Home openjdk=${openjdk}
197 ${java_actual_options} = BuiltIn.Set_Variable_If """${openjdk}""" == "openjdk7" ${JAVA_7_OPTIONS} ${JAVA_OPTIONS}
198 BuiltIn.Set_Global_Variable \${maven_bash_command} export JAVA_HOME='${java_home}' && export MAVEN_OPTS='${java_actual_options}' && ./${maven_directory}/bin/mvn
199 # TODO: Get settings files from Jenkins settings provider, somehow.
200 SSHKeywords.Execute_Command_At_Cwd_Should_Pass wget '${MAVEN_SETTINGS_URL}' -O settings.xml stderr_must_be_empty=False
203 [Arguments] ${maven_version}=3.3.9 ${openjdk}=${JDKVERSION} ${branch}=${EMPTY} ${patches}=${EMPTY}
204 [Documentation] Install Maven.
205 ... Depending on arguments, perform a multipatch build to populate local Maven repository with patched artifacts.
206 Install_Maven_Bare maven_version=${maven_version} openjdk=${openjdk}
207 BuiltIn.Return_From_Keyword_If """${patches}""" == "" No post-install build requested.
208 BuiltIn.Run_Keyword_If """${branch}""" == "" BuiltIn.Fail BRANCH needs to be specified for multipatch builds.
209 ${script_name} = BuiltIn.Set_Variable include-raw-integration-multipatch-distribution-test.sh
210 ${script_url} = BuiltIn.Set_Variable https://raw.githubusercontent.com/opendaylight/releng-builder/master/jjb/integration/${script_name}
211 SSHKeywords.Execute_Command_At_Cwd_Should_Pass wget -N '${script_url}' stderr_must_be_empty=False
212 SSHKeywords.Execute_Command_At_Cwd_Should_Pass export WORKSPACE='${WORKSPACE}' && export BRANCH='${branch}' && export PATCHES_TO_BUILD='${patches}' && bash '${script_name}' stderr_must_be_empty=False
213 Run_Maven pom_file=${WORKSPACE}/patch_tester/pom.xml
216 [Arguments] ${pom_file}=pom.xml ${log_file}=${MAVEN_DEFAULT_OUTPUT_FILENAME}
217 [Documentation] Determine arguments to use and call mvn command against given pom file.
218 SSHKeywords.Execute_Command_At_Cwd_Should_Pass mkdir -p '${MAVEN_REPOSITORY_PATH}'
219 ${maven_repository_options} = BuiltIn.Set_Variable -Dmaven.repo.local=${MAVEN_REPOSITORY_PATH} -Dorg.ops4j.pax.url.mvn.localRepository=${MAVEN_REPOSITORY_PATH}
220 SSHKeywords.Execute_Command_At_Cwd_Should_Pass ${maven_bash_command} clean install dependency:tree -V -B -DoutputFile=dependency_tree.log -s './settings.xml' -f '${pom_file}' ${MAVEN_OPTIONS} ${maven_repository_options} > '${log_file}'