- # From Ocata and moving forward, we can replace 'ostestr' with 'tempest run'
- # Note: If black-regex cancels out the entire regex (white-regex), all tests are run
- # --black-regex ${tempest_exclusion_regex} is removed for now since it only seems to work from newton
- ${results}= Write Commands Until Prompt ostestr --regex ${tempest_regex} timeout=${timeout}
- Log ${results}
- # Save stdout to file
- Create File tempest_output_${tempest_regex}.log data=${results}
- # output tempest generated log file which may have different debug levels than what stdout would show
- # FIXME: having the INFO level tempest logs is helpful as it gives details like the UUIDs of nouns used in the
- # the tests which can sometimes be tracked in ODL and Openstack logs when debugging. However, this "cat" step
- # does not even complete for the tempest.api.network tests in under 2min. We need a faster way to get this
- # info. Probably pulling the log file and storing it on the log server is best. Hopefully someone can get
- # to this. For now, commenting out this next debug step.
- # ${output}= Write Commands Until Prompt cat ${tempest_directory}/tempest.log timeout=120s
- # Log ${output}
- Should Contain ${results} Failed: 0
- # TODO: also need to verify some non-zero pass count as well as other results are ok (e.g. skipped, etc)
+ SSHLibrary.Read
+ Tempest Conf Modify Pause On Test Teardown True
+ SSHLibrary.Set Client Configuration timeout=${timeout}
+ SSHLibrary.Write python -m testtools.run ${tempest_regex}
+ ${output}= SSHLibrary.Read Until Regexp ${DEFAULT_LINUX_PROMPT_STRICT}|pdb.set_trace()
+ Log ${output}
+ Show Debugs
+ Get Test Teardown Debugs
+ SSHLibrary.Switch Connection ${tempest_conn_id}
+ SSHLibrary.Write continue
+ ${output}= SSHLibrary.Read Until Regexp ${DEFAULT_LINUX_PROMPT_STRICT}|pdb.set_trace()
+ Log ${output}
+ SSHLibrary.Write continue
+ ${output}= SSHLibrary.Read Until Prompt
+ Log ${output}
+ SSHLibrary.Close Connection
+ Should Contain ${output} OK
+ Should Not Contain ${output} FAILED
+
+Suite Setup
+ OpenStackOperations.OpenStack Suite Setup
+ Log In To Tempest Executor And Setup Test Environment
+
+Log In To Tempest Executor And Setup Test Environment
+ [Documentation] Initialize SetupUtils, open SSH connection to a devstack system and source the openstack
+ ... credentials needed to run the tempest tests. The (sometimes empty) tempest blacklist file will be created
+ ... and pushed to the tempest executor.
+ Create Blacklist File
+ # Tempest tests need an existing external network in order to create routers.
+ Create Network ${external_net_name} --external --default --provider-network-type flat --provider-physical-network ${PUBLIC_PHYSICAL_NETWORK}
+ Create Subnet ${external_net_name} ${external_subnet_name} ${external_subnet} --gateway ${external_gateway} --allocation-pool ${external_subnet_allocation_pool}
+ List Networks
+ ${control_node_conn_id}= SSHLibrary.Open Connection ${OS_CONTROL_NODE_IP} prompt=${DEFAULT_LINUX_PROMPT_STRICT}
+ SSHKeywords.Flexible SSH Login ${OS_USER}
+ Write Commands Until Prompt source ${DEVSTACK_DEPLOY_PATH}/openrc admin admin
+ Write Commands Until Prompt sudo rm -rf /opt/stack/tempest/.testrepository
+ ${net_id}= Get Net Id ${external_net_name}
+ Tempest Conf Add External Network And Floating Network Name ${net_id}
+
+Tempest Conf Add External Network And Floating Network Name
+ [Arguments] ${external_network_id}
+ [Documentation] Tempest will be run with a config file - this function will add the
+ ... given external network ID to the configuration file.
+ Modify Config In File On Existing SSH Connection ${tempest_config_file} set network public_network_id ${external_network_id}
+ Modify Config In File On Existing SSH Connection ${tempest_config_file} set DEFAULT debug False
+ Modify Config In File On Existing SSH Connection ${tempest_config_file} set DEFAULT log_level INFO
+ Modify Config In File On Existing SSH Connection ${tempest_config_file} set network floating_network_name ${external_net_name}
+ Write Commands Until Prompt sudo cat ${tempest_config_file}
+ Write Commands Until Prompt sudo chmod 777 ${tempest_config_file}
+
+Tempest Conf Modify Pause On Test Teardown
+ [Arguments] ${pause_flag}
+ [Documentation] Sets the DEFAULT section flag for pausing the test teardown. If True the tempest test case
+ ... being executed will break to a pdb() debug shell when it hits it's teardown() function.
+ Modify Config In File On Existing SSH Connection ${tempest_config_file} set DEFAULT pause_teardown ${pause_flag}
+
+Modify Config In File On Existing SSH Connection
+ [Arguments] ${config_file} ${modifier} ${config_section} ${config_key} ${config_value}=${EMPTY}
+ [Documentation] uses crudini to populate oslo cofg file.
+ # this keyword is only one line so seems like extra overhead, but this may be a good candidate to move
+ # to a library at some point, when/if other suites need to use it, so wanted to make it generic.
+ Write Commands Until Prompt sudo -E crudini --${modifier} ${config_file} ${config_section} ${config_key} ${config_value}
+
+Create Blacklist File
+ [Documentation] For each exclusion regex in the required @{${OPENSTACK_BRANCH}_exclusion_regexes} list a new
+ ... line will be created in the required ${blacklist_file} location. This file is pushed to the OS_CONTROL_NODE
+ ... which is assumed to be the tempest executor.
+ OperatingSystem.Create File ${blacklist_file}
+ : FOR ${exclusion} IN @{${OPENSTACK_BRANCH}_exclusion_regexes}
+ \ OperatingSystem.Append To File ${blacklist_file} ${exclusion}\n
+ Log File ${blacklist_file}
+ SSHKeywords.Copy File To Remote System ${OS_CONTROL_NODE_IP} ${blacklist_file} ${blacklist_file}
+
+Open Connection
+ [Arguments] ${name} ${ip}
+ ${conn_id} = SSHLibrary.Open Connection ${ip} prompt=${DEFAULT_LINUX_PROMPT}
+ SSHKeywords.Flexible SSH Login ${OS_USER} ${DEVSTACK_SYSTEM_PASSWORD}
+ BuiltIn.Set Suite Variable \${${name}} ${conn_id}
+ [Return] ${conn_id}