*** Settings *** Library SSHLibrary Library String Library DateTime Library ./UtilLibrary.py Resource KarafKeywords.txt *** Variables *** ${start} sudo mn --controller=remote,ip=${CONTROLLER} --topo tree,1 --switch ovsk,protocols=OpenFlow13 *** Keywords *** Start Suite [Documentation] Basic setup/cleanup work that can be done safely before any system ... is run. Log Start the test on the base edition Clean Mininet System ${mininet_conn_id}= Open Connection ${MININET} prompt=${LINUX_PROMPT} timeout=30s Set Suite Variable ${mininet_conn_id} Login With Public Key ${MININET_USER} ${USER_HOME}/.ssh/id_rsa any Execute Command sudo ovs-vsctl set-manager ptcp:6644 Write ${start} Read Until mininet> Stop Suite [Documentation] Cleanup/Shutdown work that should be done at the completion of all ... tests Log Stop the test on the base edition Switch Connection ${mininet_conn_id} Read Write exit Read Until ${LINUX_PROMPT} Close Connection Ensure All Nodes Are In Response [Arguments] ${URI} ${node_list} [Documentation] A GET is made to the supplied ${URI} and every item in the ${node_list} ... is verified to exist in the repsonse. This keyword currently implies that it's node ... specific but any list of strings can be given in ${node_list}. Refactoring of this ... to make it more generic should be done. (see keyword "Check For Elements At URI") : FOR ${node} IN @{node_list} \ ${resp} RequestsLibrary.Get session ${URI} \ Should Be Equal As Strings ${resp.status_code} 200 \ Should Contain ${resp.content} ${node} Check Nodes Stats [Arguments] ${node} [Documentation] A GET on the /node/${node} API is made and specific flow stat ... strings are checked for existence. ${resp} RequestsLibrary.Get session ${OPERATIONAL_NODES_API}/node/${node} Should Be Equal As Strings ${resp.status_code} 200 Should Contain ${resp.content} flow-capable-node-connector-statistics Should Contain ${resp.content} flow-table-statistics Check That Port Count Is Ok [Arguments] ${node} ${count} [Documentation] A GET on the /port API is made and the specified port ${count} is ... verified. A more generic Keyword "Check For Specific Number Of Elements At URI" ... also does this work and further consolidation should be done. ${resp} RequestsLibrary.Get session ${REST_CONTEXT}/${CONTAINER}/port Log ${resp.content} Should Be Equal As Strings ${resp.status_code} 200 Should Contain X Times ${resp.content} ${node} ${count} Check For Specific Number Of Elements At URI [Arguments] ${uri} ${element} ${expected_count} [Documentation] A GET is made to the specified ${URI} and the specific count of a ... given element is done (as supplied by ${element} and ${expected_count}) ${resp} RequestsLibrary.Get session ${uri} Log ${resp.content} Should Be Equal As Strings ${resp.status_code} 200 Should Contain X Times ${resp.content} ${element} ${expected_count} Check For Elements At URI [Arguments] ${uri} ${elements} [Documentation] A GET is made at the supplied ${URI} and every item in the list of ... ${elements} is verified to exist in the response ${resp} RequestsLibrary.Get session ${uri} Log ${resp.content} Should Be Equal As Strings ${resp.status_code} 200 : FOR ${i} IN @{elements} \ Should Contain ${resp.content} ${i} Check For Elements Not At URI [Arguments] ${uri} ${elements} [Documentation] A GET is made at the supplied ${URI} and every item in the list of ... ${elements} is verified to NOT exist in the response ${resp} RequestsLibrary.Get session ${uri} Log ${resp.content} Should Be Equal As Strings ${resp.status_code} 200 : FOR ${i} IN @{elements} \ Should Not Contain ${resp.content} ${i} Clean Mininet System [Arguments] ${mininet_system}=${MININET} Run Command On Remote System ${mininet_system} sudo mn -c Run Command On Remote System ${mininet_system} sudo ps -elf | egrep 'usr/local/bin/mn' | egrep python | awk '{print "sudo kill -9",$4}' | sh Clean Up Ovs [Arguments] ${mininet_system}=${MININET} [Documentation] Cleans up the OVS instance and remove any existing common known bridges. ${output}= Run Command On Remote System ${mininet_system} sudo ovs-vsctl list-br Log ${output} : FOR ${i} IN ${output} \ Run Command On Remote System ${mininet_system} sudo ovs-vsctl --if-exists del-br ${i} Run Command On Remote System ${mininet_system} sudo ovs-vsctl del-manager Extract Value From Content [Arguments] ${content} ${index} ${strip}=nostrip [Documentation] Will take the given response content and return the value at the given index as a string ${value}= Get Json Value ${content} ${index} ${value}= Convert To String ${value} ${value}= Run Keyword If '${strip}' == 'strip' Strip Quotes ${value} [Return] ${value} Get Process ID Based On Regex On Remote System [Arguments] ${remote_system} ${regex_string_to_match_on} [Documentation] Uses ps to find a process that matches the supplied regex. Returns the PID of that process ... The ${regex_string_to_match_on} should produce a unique process otherwise the PID returned may not be ... the expected PID # doing the extra -v grep in this command to exclude the grep process itself from the output ${output}= Run Command On Remote System ${remote_system} ps -elf | grep -v grep | grep ${regex_string_to_match_on} | awk '{print $4}' # ${output} contains the system prompt and all we want is the value of the number ${pid}= Fetch From Left ${output} \r [Return] ${pid} Get Process Thread Count On Remote System [Arguments] ${remote_system} ${pid} [Documentation] Executes the ps command to retrieve the lightweight process (aka thread) count. ${output}= Run Command On Remote System ${remote_system} ps --no-headers -o nlwp ${pid} # ${output} contains the system prompt and all we want is the value of the number ${thread_count}= Fetch From Left ${output} \r [Return] ${thread_count} Strip Quotes [Arguments] ${string_to_strip} [Documentation] Will strip ALL quotes from given string and return the new string ${string_to_return}= Replace String ${string_to_strip} " \ count=-1 [Return] ${string_to_return} Run Command On Remote System [Arguments] ${remote_system} ${cmd} ${user}=${MININET_USER} ${prompt}=${LINUX_PROMPT} ${prompt_timeout}=30s [Documentation] Reduces the common work of running a command on a remote system to a single higher level robot keyword, ... taking care to log in with a public key and. The command given is written and the output returned. No test conditions ... are checked. Log Attempting to execute ${cmd} on ${remote_system} by ${user} with ${keyfile_pass} and ${prompt} ${conn_id}= SSHLibrary.Open Connection ${remote_system} prompt=${prompt} timeout=${prompt_timeout} Login With Public Key ${user} ${USER_HOME}/.ssh/id_rsa ${KEYFILE_PASS} SSHLibrary.Write ${cmd} ${output}= SSHLibrary.Read Until ${prompt} SSHLibrary.Close Connection Log ${output} [Return] ${output} Verify File Exists On Remote System [Arguments] ${remote_system} ${file} ${user}=${MININET_USER} ${prompt}=${LINUX_PROMPT} ${prompt_timeout}=5s [Documentation] Will create connection with public key and will PASS if the given ${file} exists, otherwise will FAIL ${conn_id}= Open Connection ${remote_system} prompt=${prompt} timeout=${prompt_timeout} Login With Public Key ${user} ${USER_HOME}/.ssh/id_rsa any SSHLibrary.File Should Exist ${file} Close Connection Verify Controller Is Not Dead [Arguments] ${controller_ip}=${CONTROLLER} [Documentation] Will execute any tests to verify the controller is not dead. Some checks are ... Out Of Memory Execptions. Check Karaf Log File Does Not Have Messages ${controller_ip} java.lang.OutOfMemoryError Verify Controller Has No Null Pointer Exceptions [Arguments] ${controller_ip}=${CONTROLLER} [Documentation] Will execute any tests to verify the controller is not having any null pointer eceptions. Check Karaf Log File Does Not Have Messages ${controller_ip} java.lang.NullPointerException Get Epoch Time [Arguments] ${time} [Documentation] Get the Epoc time from MM/DD/YYYY HH:MM:SS ${epoch_time}= Convert Date ${time} epoch exclude_milles=True date_format=%m/%d/%Y %H:%M:%S ${epoch_time}= Convert To Integer ${epoch_time} [Return] ${epoch_time} Remove Space on String [Arguments] ${str} ${count}=-1 [Documentation] Remove the empty space from given string.count is optional,if its given ... that many occurence of space will be removed from left ${x}= Convert To String ${str} ${x}= Replace String ${str} ${SPACE} ${EMPTY} count=${count} [Return] ${x} Split Value from String [Arguments] ${str} ${splitter} [Documentation] Split the String based on given splitter and return as list @{x}= Split String ${str} ${splitter} [Return] @{x} Concatenate the String [Arguments] ${str1} ${str2} [Documentation] Catenate the two non-string objects and return as String ${str1}= Convert to String ${str1} ${str2}= Convert to String ${str2} ${output}= Catenate ${str1} ${str2} [Return] ${output} Remove All Elements At URI [Arguments] ${uri} ${resp} RequestsLibrary.Delete session ${uri} Should Be Equal As Strings ${resp.status_code} 200 Add Elements To URI From File [Arguments] ${dest_uri} ${data_file} ${body} OperatingSystem.Get File ${data_file} ${resp} RequestsLibrary.Put session ${dest_uri} data=${body} headers=${headers} Should Be Equal As Strings ${resp.status_code} 200 Post Elements To URI From File [Arguments] ${dest_uri} ${data_file} ${body} OperatingSystem.Get File ${data_file} ${resp} RequestsLibrary.Post session ${dest_uri} data=${body} headers=${headers} Should Be Equal As Strings ${resp.status_code} 200 Stop One Or More Controllers [Arguments] @{controllers} [Documentation] Give this keyword a scalar or list of controllers to be stopped. ${cmd} = Set Variable ${KARAF_HOME}/bin/stop : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} : FOR ${ip} IN @{controllers} \ Wait Until Keyword Succeeds 60 s 3 s Controller Down Check ${ip} Start One Or More Controllers [Arguments] @{controllers} [Documentation] Give this keyword a scalar or list of controllers to be started. ${cmd} = Set Variable ${KARAF_HOME}/bin/start : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} # TODO: This should throw an error if controller never comes up. : FOR ${ip} IN @{controllers} \ UtilLibrary.Wait For Controller Up ${ip} ${RESTCONFPORT} Kill One Or More Controllers [Arguments] @{controllers} [Documentation] Give this keyword a scalar or list of controllers to be stopped. ${cmd} = Set Variable ps axf | grep karaf | grep -v grep | awk '{print \"kill -9 \" $1}' | sh log ${cmd} ${controller_pid}= Get Process ID Based On Regex On Remote System ${CONTROLLER} karaf : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} : FOR ${ip} IN @{controllers} \ Wait Until Keyword Succeeds 60 s 3 s Controller Down Check ${ip} Controller Down Check [Arguments] ${ip} [Documentation] Checks to see if a controller is down by verifying that the karaf process isn't present. ${cmd} = Set Variable ps axf | grep karaf | grep -v grep | wc -l ${response} Run Command On Remote System ${ip} ${cmd} Log Number of controller instances running: ${response} Should Start With ${response} 0 Controller process found or there may be extra instances of ... karaf running on the host machine. Clean One Or More Journals [Arguments] @{controllers} [Documentation] Give this keyword a scalar or list of controllers on which to clean journals. ${del_cmd} = Set Variable rm -rf ${KARAF_HOME}/journal : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${del_cmd} Show Cluster Configuation Files [Arguments] @{controllers} [Documentation] Prints out the cluster configuration files for one or more controllers. Log controllers: @{controllers} ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/akka.conf : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/modules.conf : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/module-shards.conf : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/jolokia.xml : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} ${cmd} = Set Variable cat ${KARAF_HOME}/etc/initial/org.apache.karaf.management.cfg : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd} ${cmd} = Set Variable cat ${KARAF_HOME}/etc/org.apache.karaf.features.cfg : FOR ${ip} IN @{controllers} \ Run Command On Remote System ${ip} ${cmd}