2 Documentation Karaf library. This library is useful to deal with controller Karaf console for ssh sessions in cluster.
3 ... Running Setup_Karaf_Keywords is necessary. If SetupUtils initialization is called, this gets initialized as well.
4 ... If this gets initialized, ClusterManagement gets initialized as well.
6 Library OperatingSystem
7 Resource ${CURDIR}/ClusterManagement.robot
8 Resource ${CURDIR}/SSHKeywords.robot
9 Variables ${CURDIR}/../variables/Variables.py
13 ${connection_index_dict} &{EMPTY}
17 [Arguments] ${http_timeout}=1
18 [Documentation] Initialize ClusterManagement. Open ssh karaf connections to each ODL.
19 ClusterManagement.ClusterManagement_Setup http_timeout=${http_timeout}
20 BuiltIn.Comment First connections to Karaf console may fail, so WUKS is used. TODO: Track as a Bug.
21 : FOR ${index} IN @{ClusterManagement__member_index_list}
22 \ BuiltIn.Run_Keyword_And_Ignore_Error BuiltIn.Wait_Until_Keyword_Succeeds 3s 1s Open_Controller_Karaf_Console_On_Background member_index=${index}
24 Verify_Feature_Is_Installed
25 [Arguments] ${feature_name} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT}
26 [Documentation] Will Succeed if the given ${feature_name} is found in the output of "feature:list -i"
27 ${output} = Issue_Command_On_Karaf_Console feature:list -i | grep ${feature_name} ${controller} ${karaf_port}
28 BuiltIn.Should_Contain ${output} ${feature_name}
31 Issue_Command_On_Karaf_Console
32 [Arguments] ${cmd} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT} ${timeout}=10 ${loglevel}=INFO
33 [Documentation] Will execute the given ${cmd} by ssh'ing to the karaf console running on ${controller}
34 ... Note that this keyword will open&close new SSH connection, without switching back to previously current session.
35 SSHLibrary.Open_Connection ${controller} port=${karaf_port} prompt=${KARAF_PROMPT_LOGIN} timeout=${timeout}
36 SSHLibrary.Login ${KARAF_USER} ${KARAF_PASSWORD} loglevel=${loglevel}
37 SSHLibrary.Write ${cmd}
38 ${output} SSHLibrary.Read_Until_Regexp ${KARAF_PROMPT}
39 SSHLibrary.Close_Connection
43 Safe_Issue_Command_On_Karaf_Console
44 [Arguments] ${cmd} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT} ${timeout}=10 ${loglevel}=INFO
45 [Documentation] Run Issue_Command_On_Karaf_Console but restore previous connection afterwards.
46 BuiltIn.Run_Keyword_And_Return SSHKeywords.Run_Keyword_Preserve_Connection Issue_Command_On_Karaf_Console ${cmd} ${controller} ${karaf_port} ${timeout}
49 Check For Elements On Karaf Command Output Message
50 [Arguments] ${cmd} ${elements} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT} ${timeout}=5
51 [Documentation] Will execute the command using Issue Command On Karaf Console then check for the given elements
52 ... in the command output message
53 ${output} Issue_Command_On_Karaf_Console ${cmd} ${controller} ${karaf_port} ${timeout}
54 : FOR ${i} IN @{elements}
55 \ BuiltIn.Should_Contain ${output} ${i}
57 Verify_Bundle_Is_Installed
58 [Arguments] ${bundle_name} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT}
59 [Documentation] Will succeed if the given ${bundle name} is present in the output of "bundle:list -s "
60 ${output} = Issue_Command_On_Karaf_Console bundle:list -s | grep ${bundle_name} ${controller} ${karaf_port}
61 BuiltIn.Should_Contain ${output} ${bundle_name}
64 Verify_Bundle_Is_Not_Installed
65 [Arguments] ${bundle_name} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT}
66 [Documentation] Will succeed if the given ${bundle_name} is NOT found in the output of "bundle:list -s"
67 ${output} = Issue_Command_On_Karaf_Console bundle:list -i | grep ${bundle_name} ${controller} ${karaf_port}
68 BuiltIn.Should_Not_Contain ${output} ${bundle_name}
71 Check_Karaf_Log_Has_Messages
72 [Arguments] ${filter_string} @{message_list}
73 [Documentation] Will succeed if the @{messages} are found in \ the output of "log:display"
74 ${output} = Issue_Command_On_Karaf_Console log:display | grep ${filter_string}
75 : FOR ${message} IN @{message_list}
76 \ BuiltIn.Should_Contain ${output} ${message}
80 [Arguments] ${feature_name} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT} ${timeout}=180
81 [Documentation] Will Install the given ${feature_name}
82 BuiltIn.Log ${timeout}
83 ${output} = Issue_Command_On_Karaf_Console feature:install ${feature_name} ${controller} ${karaf_port} ${timeout}
87 Install_A_Feature_Using_Active_Connection
88 [Arguments] ${feature_name}
89 [Documentation] Will Install the given ${feature_name} using active connection
90 ${cmd} = BuiltIn.Set_Variable feature:install ${feature_name}
91 SSHLibrary.Write ${cmd}
92 ${output} SSHLibrary.Read_Until_Regexp ${KARAF_PROMPT}
97 [Arguments] ${feature_name} ${controller}=${ODL_SYSTEM_IP} ${karaf_port}=${KARAF_SHELL_PORT} ${timeout}=180
98 [Documentation] Will UnInstall the given ${feature_name}
99 ${output} = Issue_Command_On_Karaf_Console feature:uninstall ${feature_name} ${controller} ${karaf_port} ${timeout}
100 BuiltIn.Log ${output}
103 Open_Controller_Karaf_Console_On_Background
104 [Arguments] ${member_index}=${1} ${timeout}=10 ${loglevel}=INFO
105 [Documentation] If there is a stored ssh connection index of connection to the controller's karaf console for ${member_index},
106 ... close the previous connection. In any case create a new connection
107 ... to karaf console for ${member_index}, set correct prompt set and login to karaf console.
108 ... Store connection index for ${member_index} and restore the previous active connection.
109 ${current_ssh_connection_object}= SSHLibrary.Get_Connection
110 BuiltIn.Log ${connection_index_dict}
111 BuiltIn.Log ${member_index}
112 ${status} ${old_connection_index} = BuiltIn.Run_Keyword_And_Ignore_Error Get From Dictionary ${connection_index_dict} ${member_index}
113 BuiltIn.Run_Keyword_If '${status}'=='PASS' BuiltIn.Run_Keywords SSHLibrary.Switch_Connection ${old_connection_index}
114 ... AND SSHLibrary.Close_Connection
115 ${odl_ip} = ClusterManagement.Resolve_IP_Address_For_Member ${member_index}
116 SSHLibrary.Open_Connection ${odl_ip} port=${KARAF_SHELL_PORT} prompt=${KARAF_PROMPT_LOGIN} timeout=${timeout}
117 ${karaf_connection_object} = SSHLibrary.Get_Connection
118 Collections.Set_To_Dictionary ${connection_index_dict} ${member_index} ${karaf_connection_object.index}
119 SSHLibrary.Login ${KARAF_USER} ${KARAF_PASSWORD} loglevel=${loglevel}
120 [Teardown] SSHKeywords.Restore_Current_Ssh_Connection_From_Index ${current_ssh_connection_object.index}
122 Open_Controller_Karaf_Console_With_Timeout
123 [Arguments] ${member_index}=${1} ${timeout}=3s
124 [Documentation] Open new connection to karaf console for member index with specified timeout.
125 BuiltIn.Log ${member_index}
126 ${odl_ip} = ClusterManagement.Resolve_IP_Address_For_Member ${member_index}
127 SSHLibrary.Open_Connection ${odl_ip} port=${KARAF_SHELL_PORT} prompt=${KARAF_PROMPT_LOGIN} timeout=${timeout}
128 SSHLibrary.Login ${KARAF_USER} ${KARAF_PASSWORD}
130 Configure_Timeout_For_Karaf_Console
131 [Arguments] ${timeout} ${member_index_list}=${EMPTY}
132 [Documentation] Configure a different timeout for each Karaf console.
133 ${index_list} = ClusterManagement.List_Indices_Or_All given_list=${member_index_list}
134 ${current_connection_object} = SSHLibrary.Get_Connection
135 : FOR ${member_index} IN @{index_list} # usually: 1, 2, 3
136 \ ${karaf_connection_index} = Collections.Get_From_Dictionary ${connection_index_dict} ${member_index}
137 \ SSHLibrary.Switch_Connection ${karaf_connection_index}
138 \ SSHLibrary.Set_Client_Configuration timeout=${timeout}
139 [Teardown] SSHKeywords.Restore_Current_Ssh_Connection_From_Index ${current_connection_object.index}
141 Execute_Controller_Karaf_Command_On_Background
142 [Arguments] ${command} ${member_index}=${1}
143 [Documentation] Send command to karaf without affecting current SSH connection. Read, log and return response.
144 ${karaf_connection_index} = Collections.Get_From_Dictionary ${connection_index_dict} ${member_index}
145 ${current_connection_index} = SSHLibrary.Switch_Connection ${karaf_connection_index}
146 ${status_write} ${message_write} = BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Write ${command}
147 ${status_wait} ${message_wait} = BuiltIn.Run_Keyword_And_Ignore_Error SSHLibrary.Read_Until_Regexp ${KARAF_PROMPT}
148 BuiltIn.Run Keyword If '${status_write}' != 'PASS' BuiltIn.Fail Failed to send the command: ${command}
149 BuiltIn.Log ${message_wait}
150 BuiltIn.Run_Keyword_If '${status_wait}' != 'PASS' BuiltIn.Fail Failed to see prompt after sending the command: ${command}
151 [Teardown] SSHKeywords.Restore_Current_Ssh_Connection_From_Index ${current_connection_index}
152 [Return] ${message_wait}
154 Execute_Controller_Karaf_Command_With_Retry_On_Background
155 [Arguments] ${command} ${member_index}=${1}
156 [Documentation] Attemp to send command to karaf for ${member_index}, if fail then open connection and try again.
157 ${status} ${message} = BuiltIn.Run_Keyword_And_Ignore_Error Execute_Controller_Karaf_Command_On_Background ${command} ${member_index}
158 BuiltIn.Return_From_Keyword_If '${status}' == 'PASS' ${message}
159 # TODO: Verify this does not leak connections indices.
160 Open_Controller_Karaf_Console_On_Background ${member_index}
161 ${message} = Execute_Controller_Karaf_Command_On_Background ${command} ${member_index}
164 Log_Message_To_Controller_Karaf
165 [Arguments] ${message} ${member_index_list}=${EMPTY} ${tolerate_failure}=True
166 [Documentation] Make sure this resource is initialized. Send a message into the controller's karaf log file on every node listed (or all).
167 ... By default, failure while processing a node is silently ignored, unless ${tolerate_failure} is False.
168 ${index_list} = ClusterManagement.List_Indices_Or_All given_list=${member_index_list}
169 : FOR ${index} IN @{index_list} # usually: 1, 2, 3.
170 \ ${status} ${output} = BuiltIn.Run_Keyword_And_Ignore_Error Execute_Controller_Karaf_Command_With_Retry_On_Background log:log "ROBOT MESSAGE: ${message}" member_index=${index}
171 \ BuiltIn.Run_Keyword_Unless ${tolerate_failure} or "${status}" == "PASS" BuiltIn.Fail ${output}
173 Log_Test_Suite_Start_To_Controller_Karaf
174 [Arguments] ${member_index_list}=${EMPTY}
175 [Documentation] Log suite name to karaf log, useful in suite setup.
176 Log_Message_To_Controller_Karaf Starting suite ${SUITE_SOURCE} ${member_index_list}
178 Log_Testcase_Start_To_Controller_Karaf
179 [Arguments] ${member_index_list}=${EMPTY}
180 [Documentation] Log test case name to karaf log, useful in test case setup.
181 Log_Message_To_Controller_Karaf Starting test ${TEST_NAME} ${member_index_list}
183 Set_Bgpcep_Log_Levels
184 [Arguments] ${bgpcep_level}=${DEFAULT_BGPCEP_LOG_LEVEL} ${protocol_level}=${DEFAULT_PROTOCOL_LOG_LEVEL} ${member_index_list}=${EMPTY}
185 [Documentation] Assuming OCKCOB was used, set logging level on bgpcep and protocol loggers without affecting current SSH session.
186 # FIXME: Move to appropriate Resource
187 ${index_list} = ClusterManagement.List_Indices_Or_All given_list=${member_index_list}
188 : FOR ${index} IN @{index_list} # usually: 1, 2, 3.
189 \ Execute_Controller_Karaf_Command_On_Background log:set ${bgpcep_level} org.opendaylight.bgpcep member_index=${index}
190 \ Execute_Controller_Karaf_Command_On_Background log:set ${protocol_level} org.opendaylight.protocol member_index=${index}
193 [Arguments] ${message} ${timeout}=60 ${member_index}=${1}
194 [Documentation] Read karaf logs until message appear
195 # TODO: refactor this keyword to use the new workflow to account for multiple controllers. Initial work was done
196 # in this patch https://git.opendaylight.org/gerrit/#/c/45596/
197 # however, the consumers of this keyword were breaking after that change. Initial theory is that a previous
198 # keyword used before this "Wait For Karaf Log" keyword was closing the karaf console connection, so the
199 # "Flexible SSH Login" keyword from the patch above (45596) was failing.
200 BuiltIn.Log Waiting for '${message}' in karaf log
201 SSHLibrary.Open_Connection ${ODL_SYSTEM_IP} port=${KARAF_SHELL_PORT} prompt=${KARAF_PROMPT_LOGIN} timeout=${timeout}
202 SSHLibrary.Login ${KARAF_USER} ${KARAF_PASSWORD} loglevel=${loglevel}
203 SSHLibrary.Write log:tail
204 SSHLibrary.Read_Until ${message}
205 SSHLibrary.Close_Connection
208 [Arguments] ${bundle_id}
209 [Documentation] Restarts bundle passed as argument. Note this operation is only for testing and not production environments
210 # TODO: prepare this for cluster environment and multiple controllers
211 Execute_Controller_Karaf_Command_With_Retry_On_Background bundle:restart -f $(bundle:id '${bundle_id}')
214 [Documentation] Restarts Karaf and polls log to detect when Karaf is up and running again
215 # TODO: prepare this for cluster environment and multiple controllers
216 Execute_Controller_Karaf_Command_With_Retry_On_Background log:clear
217 Execute_Controller_Karaf_Command_With_Retry_On_Background shutdown -r -f
218 BuiltIn.Run_Keyword_And_Return_Status BuiltIn.Wait_Until_Keyword_Succeeds 240s 60s Wait_For_Karaf_Log Karaf started in
221 [Documentation] Restarts jetty bundle (to reload certificates or key/truststore information)
222 Execute_Controller_Karaf_Command_With_Retry_On_Background log:clear
223 Restart_Bundle OPS4J Pax Web - Jetty
224 Wait_For_Karaf_Log Instantiated the Application class org.opendaylight.restconf.RestconfApplication
225 Wait_For_Karaf_Log Instantiated the Application class org.opendaylight.netconf.sal.rest.impl.RestconfApplication
226 Wait_For_Karaf_Log Instantiated the Application class org.opendaylight.aaa.idm.IdmLightApplication