2 Documentation General Utils library. This library has broad scope, it can be used by any robot system tests.
4 Resource DevstackUtils.robot
5 Resource OpenStackOperations.robot
6 Resource SSHKeywords.robot
7 Resource ../variables/Variables.robot
10 @{stable/ocata_EXCLUSION_REGEXES} ${EMPTY}
11 @{stable/pike_EXCLUSION_REGEXES} ${EMPTY}
12 @{stable/queens_EXCLUSION_REGEXES} ${EMPTY}
13 @{master_EXCLUSION_REGEXES} ${EMPTY}
14 ${BLACKLIST_FILE} /tmp/blacklist.txt
15 ${TEMPEST_DIR} /opt/stack/tempest
16 ${TEMPEST_CONFIG_FILE} ${TEMPEST_DIR}/etc/tempest.conf
17 # Parameter values below are based on releng/builder - changing them requires updates in releng/builder as well
18 ${TEMPEST_TIMEOUT} 420s
22 OpenStackOperations.OpenStack Suite Setup
23 Tempest.Log In To Tempest Executor And Setup Test Environment
26 [Arguments] ${tempest_regex} ${timeout}=${TEMPEST_TIMEOUT} ${debug}=False
27 BuiltIn.Run Keyword If "${debug}" == "False" Tempest.Run Tempest Tests Without Debug ${tempest_regex} timeout=${timeout}
28 BuiltIn.Run Keyword If "${debug}" == "True" Tempest.Run Tempest Tests With Debug ${tempest_regex} timeout=${timeout}
29 BuiltIn.Run Keyword If "${debug}" != "True" and "${debug}" != "False" Fail debug argument must be True or False
31 Run Tempest Tests Without Debug
32 [Arguments] ${tempest_regex} ${TEMPEST_DIRectory}=${TEMPEST_DIR} ${timeout}=${TEMPEST_TIMEOUT}
33 [Documentation] Using ostestr will allow us to (by default) run tests in paralllel.
34 ... Because of the parallel fashion, we must ensure there is no pause on teardown so that flag in tempest.conf is
35 ... explicitly set to False.
36 BuiltIn.Return From Keyword If "skip_if_${OPENSTACK_BRANCH}" in @{TEST_TAGS}
37 BuiltIn.Return From Keyword If "skip_if_${SECURITY_GROUP_MODE}" in @{TEST_TAGS}
38 BuiltIn.Return From Keyword If "skip_if_${ODL_SNAT_MODE}" in @{TEST_TAGS}
39 ${tempest_conn_id} = SSHLibrary.Open Connection ${OS_CNTL_IP} prompt=${DEFAULT_LINUX_PROMPT_STRICT}
40 SSHKeywords.Flexible SSH Login ${OS_USER} ${DEVSTACK_SYSTEM_PASSWORD}
41 DevstackUtils.Write Commands Until Prompt source ${DEVSTACK_DEPLOY_PATH}/openrc admin admin
42 DevstackUtils.Write Commands Until Prompt cd ${TEMPEST_DIRectory}
44 Tempest.Tempest Conf Modify Pause On Test Teardown False
45 SSHLibrary.Set Client Configuration timeout=${timeout}
46 # There are tons of deprecation error messages when we use ostestr in our CSIT environment (openstack via devstack)
47 # The robot log files are very large and one culprit is all these deprecation warnings. If we redirect stderr to
48 # /dev/null we should be able to ignore them. We will miss any other errors, however.
49 ${output} = DevstackUtils.Write Commands Until Prompt And Log ostestr --regex ${tempest_regex} timeout=${timeout}
50 SSHLibrary.Close Connection
51 BuiltIn.Should Contain ${output} Failed: 0
53 Run Tempest Tests With Debug
54 [Arguments] ${tempest_regex} ${TEMPEST_DIRectory}=${TEMPEST_DIR} ${timeout}=${TEMPEST_TIMEOUT}
55 [Documentation] After setting pause_teardown=True in tempest.conf, use the python -m testtools.run module to execute
56 ... a single tempest test case. We need to run only one tempest test case at a time as there will
57 ... be potentional for an unkown number of debug pdb() prompts to catch and continue if we are running multiple
58 ... test cases with a single command. Essentially, this keyword only handles one breakpoint at a single teardown.
59 BuiltIn.Return From Keyword If "skip_if_${OPENSTACK_BRANCH}" in @{TEST_TAGS}
60 BuiltIn.Return From Keyword If "skip_if_${SECURITY_GROUP_MODE}" in @{TEST_TAGS}
61 BuiltIn.Return From Keyword If "skip_if_${ODL_SNAT_MODE}" in @{TEST_TAGS}
62 ${tempest_conn_id} = SSHLibrary.Open Connection ${OS_CNTL_IP} prompt=${DEFAULT_LINUX_PROMPT_STRICT}
63 SSHKeywords.Flexible SSH Login ${OS_USER} ${DEVSTACK_SYSTEM_PASSWORD}
64 DevstackUtils.Write Commands Until Prompt source ${DEVSTACK_DEPLOY_PATH}/openrc admin admin
65 DevstackUtils.Write Commands Until Prompt cd ${TEMPEST_DIRectory}
67 Tempest Conf Modify Pause On Test Teardown True
68 SSHLibrary.Set Client Configuration timeout=${timeout}
69 SSHLibrary.Write python -m testtools.run ${tempest_regex}
70 ${output} = SSHLibrary.Read Until Regexp ${DEFAULT_LINUX_PROMPT_STRICT}|pdb.set_trace()
72 OpenStackOperations.Show Debugs
73 OpenStackOperations.Get Test Teardown Debugs
74 SSHLibrary.Switch Connection ${tempest_conn_id}
75 SSHLibrary.Write continue
76 ${output} = SSHLibrary.Read Until Regexp ${DEFAULT_LINUX_PROMPT_STRICT}|pdb.set_trace()
78 SSHLibrary.Write continue
79 ${output} = SSHLibrary.Read Until Prompt
81 SSHLibrary.Close Connection
82 BuiltIn.Should Contain ${output} OK
83 BuiltIn.Should Not Contain ${output} FAILED
85 Log In To Tempest Executor And Setup Test Environment
86 [Documentation] Initialize SetupUtils, open SSH connection to a devstack system and source the openstack
87 ... credentials needed to run the tempest tests. The (sometimes empty) tempest blacklist file will be created
88 ... and pushed to the tempest executor.
89 Tempest.Create Blacklist File
90 # Tempest tests need an existing external network in order to create routers.
91 OpenStackOperations.Create Network ${EXTERNAL_NET_NAME} --external --default --provider-network-type flat --provider-physical-network ${PUBLIC_PHYSICAL_NETWORK}
92 OpenStackOperations.Create Subnet ${EXTERNAL_NET_NAME} ${EXTERNAL_SUBNET_NAME} ${EXTERNAL_SUBNET} --gateway ${EXTERNAL_GATEWAY} --allocation-pool ${EXTERNAL_SUBNET_ALLOCATION_POOL}
93 OpenStackOperations.List Networks
94 ${control_node_conn_id} = SSHLibrary.Open Connection ${OS_CNTL_IP} prompt=${DEFAULT_LINUX_PROMPT_STRICT}
95 SSHKeywords.Flexible SSH Login ${OS_USER}
96 DevstackUtils.Write Commands Until Prompt And Log sudo pip install -U --verbose pip timeout=120s
97 DevstackUtils.Write Commands Until Prompt And Log sudo pip install -U --verbose os-testr>=1.0.0 timeout=120s
98 DevstackUtils.Write Commands Until Prompt And Log ostestr --version
99 DevstackUtils.Write Commands Until Prompt And Log testr init
100 DevstackUtils.Write Commands Until Prompt source ${DEVSTACK_DEPLOY_PATH}/openrc admin admin
101 DevstackUtils.Write Commands Until Prompt sudo rm -rf /opt/stack/tempest/.testrepository
102 ${net_id} = OpenStackOperations.Get Net Id ${EXTERNAL_NET_NAME}
103 Tempest.Tempest Conf Add External Network And Floating Network Name ${net_id}
105 Tempest Conf Add External Network And Floating Network Name
106 [Arguments] ${external_network_id}
107 [Documentation] Tempest will be run with a config file - this function will add the
108 ... given external network ID to the configuration file.
109 Tempest.Modify Config In File On Existing SSH Connection ${TEMPEST_CONFIG_FILE} set network public_network_id ${external_network_id}
110 Tempest.Modify Config In File On Existing SSH Connection ${TEMPEST_CONFIG_FILE} set DEFAULT debug False
111 Tempest.Modify Config In File On Existing SSH Connection ${TEMPEST_CONFIG_FILE} set DEFAULT log_level INFO
112 Tempest.Modify Config In File On Existing SSH Connection ${TEMPEST_CONFIG_FILE} set network floating_network_name ${EXTERNAL_NET_NAME}
113 DevstackUtils.Write Commands Until Prompt sudo cat ${TEMPEST_CONFIG_FILE}
114 DevstackUtils.Write Commands Until Prompt sudo chmod 777 ${TEMPEST_CONFIG_FILE}
116 Tempest Conf Modify Pause On Test Teardown
117 [Arguments] ${pause_flag}
118 [Documentation] Sets the DEFAULT section flag for pausing the test teardown. If True the tempest test case
119 ... being executed will break to a pdb() debug shell when it hits it's teardown() function.
120 Tempest.Modify Config In File On Existing SSH Connection ${TEMPEST_CONFIG_FILE} set DEFAULT pause_teardown ${pause_flag}
122 Modify Config In File On Existing SSH Connection
123 [Arguments] ${config_file} ${modifier} ${config_section} ${config_key} ${config_value}=${EMPTY}
124 [Documentation] uses crudini to populate oslo cofg file.
125 # this keyword is only one line so seems like extra overhead, but this may be a good candidate to move
126 # to a library at some point, when/if other suites need to use it, so wanted to make it generic.
127 DevstackUtils.Write Commands Until Prompt sudo -E crudini --${modifier} ${config_file} ${config_section} ${config_key} ${config_value}
129 Create Blacklist File
130 [Documentation] For each exclusion regex in the required @{${OPENSTACK_BRANCH}_EXCLUSION_REGEXES} list a new
131 ... line will be created in the required ${BLACKLIST_FILE} location. This file is pushed to the OS_CONTROL_NODE
132 ... which is assumed to be the tempest executor.
133 OperatingSystem.Create File ${BLACKLIST_FILE}
134 : FOR ${exclusion} IN @{${OPENSTACK_BRANCH}_EXCLUSION_REGEXES}
135 \ OperatingSystem.Append To File ${BLACKLIST_FILE} ${exclusion}\n
136 OperatingSystem.Log File ${BLACKLIST_FILE}
137 SSHKeywords.Copy File To Remote System ${OS_CNTL_IP} ${BLACKLIST_FILE} ${BLACKLIST_FILE}