2 Documentation Test suite for running tempest tests. It is assumed that the test environment
3 ... is already deployed and ready.
4 Suite Setup Log In To Tempest Executor And Setup Test Environment
5 Suite Teardown Clean Up After Running Tempest
6 Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
7 Test Teardown Run Keywords Get Test Teardown Debugs
8 Test Template DevstackUtils.Run Tempest Tests
9 Library OperatingSystem
11 Resource ../../../libraries/DevstackUtils.robot
12 Resource ../../../libraries/OpenStackOperations.robot
13 Resource ../../../libraries/SetupUtils.robot
14 Resource ../../../libraries/SSHKeywords.robot
15 Variables ../../../variables/Variables.py
16 Resource ../../../variables/netvirt/Variables.robot
19 ${blacklist_file} /tmp/blacklist.txt
20 @{stable/newton_exclusion_regexes} ${EMPTY}
21 @{stable/ocata_exclusion_regexes} ${EMPTY}
22 @{stable/pike_exclusion_regexes} ${EMPTY}
23 @{master_exclusion_regexes} ${EMPTY}
24 ${tempest_config_file} /opt/stack/tempest/etc/tempest.conf
25 ${external_physical_network} physnet1
26 ${external_net_name} external-net
27 ${external_subnet_name} external-subnet
28 # Parameter values below are based on releng/builder - changing them requires updates in releng/builder as well
29 ${external_gateway} 10.10.10.250
30 ${external_subnet_allocation_pool} start=10.10.10.2,end=10.10.10.249
31 ${external_subnet} 10.10.10.0/24
35 ${TEST_NAME} ${blacklist_file} ${tempest_config_file} timeout=900s
37 tempest.scenario.test_aggregates_basic_ops.TestAggregatesBasicOps.test_aggregate_basic_ops
38 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
40 tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_pause_unpause
41 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
43 tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot
44 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
46 tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_rebuild
47 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
49 tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start
50 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
52 tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_suspend_resume
53 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
55 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_connectivity_between_vms_on_different_networks
56 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
58 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic
59 [Tags] skip_if_stable/mitaka
60 # Failing due to default security rules behavior missing in Mitaka
61 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
63 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_mtu_sized_frames
64 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
66 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
67 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
69 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port
70 [Tags] skip_if_transparent skip_if_stable/mitaka
71 # Failing due to default security rules behavior missing in Mitaka, and also in all transparent runs
72 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
74 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_preserve_preexisting_port
75 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
77 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_router_rescheduling
78 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
80 tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details
81 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
83 tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os
84 [Tags] skip_if_transparent skip_if_learn
85 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
87 tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
88 [Tags] skip_if_transparent skip_if_learn
89 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
91 tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os
92 [Tags] skip_if_transparent skip_if_learn
93 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
95 tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os
96 [Tags] skip_if_transparent skip_if_learn
97 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
99 tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps
100 [Tags] skip_if_transparent skip_if_learn
101 ${TEST_NAME} ${blacklist_file} ${tempest_config_file} timeout=900s
103 tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
104 ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
107 Log In To Tempest Executor And Setup Test Environment
108 [Documentation] Initialize SetupUtils, open SSH connection to a devstack system and source the openstack
109 ... credentials needed to run the tempest tests. The (sometimes empty) tempest blacklist file will be created
110 ... and pushed to the tempest executor.
111 Create Blacklist File
112 SetupUtils.Setup_Utils_For_Setup_And_Teardown
113 # source_pwd is expected to exist in the below Create Network, Create Subnet keywords. Might be a bug.
114 ${source_pwd} Set Variable yes
115 Set Suite Variable ${source_pwd}
116 # Tempest tests need an existing external network in order to create routers.
117 Create Network ${external_net_name} --external --default --provider-network-type flat --provider-physical-network ${PUBLIC_PHYSICAL_NETWORK}
118 Create Subnet ${external_net_name} ${external_subnet_name} ${external_subnet} --gateway ${external_gateway} --allocation-pool ${external_subnet_allocation_pool}
120 ${control_node_conn_id}= SSHLibrary.Open Connection ${OS_CONTROL_NODE_IP} prompt=${DEFAULT_LINUX_PROMPT_STRICT}
121 SSHKeywords.Flexible SSH Login ${OS_USER}
122 Write Commands Until Prompt source ${DEVSTACK_DEPLOY_PATH}/openrc admin admin
123 Write Commands Until Prompt sudo rm -rf /opt/stack/tempest/.testrepository
124 ${net_id}= Get Net Id ${external_net_name} ${control_node_conn_id}
125 Tempest Conf Add External Network ${net_id}
127 Tempest Conf Add External Network
128 [Arguments] ${external_network_id}
129 [Documentation] Tempest will be run with a config file - this function will add the
130 ... given external network ID to the configuration file.
131 Modify Config In File On Existing SSH Connection ${tempest_config_file} set network public_network_id ${external_network_id}
132 Modify Config In File On Existing SSH Connection ${tempest_config_file} set DEFAULT debug False
133 Modify Config In File On Existing SSH Connection ${tempest_config_file} set DEFAULT log_level INFO
134 Write Commands Until Prompt sudo cat ${tempest_config_file}
135 Write Commands Until Prompt sudo chmod 777 ${tempest_config_file}
137 Modify Config In File On Existing SSH Connection
138 [Arguments] ${config_file} ${modifier} ${config_section} ${config_key} ${config_value}=${EMPTY}
139 [Documentation] uses crudini to populate oslo cofg file.
140 # this keyword is only one line so seems like extra overhead, but this may be a good candidate to move
141 # to a library at some point, when/if other suites need to use it, so wanted to make it generic.
142 Write Commands Until Prompt sudo -E crudini --${modifier} ${config_file} ${config_section} ${config_key} ${config_value}
144 Clean Up After Running Tempest
145 [Documentation] Clean up any extra leftovers that were created to allow tempest tests to run.
146 Delete Network ${external_net_name}
148 Close All Connections
150 Create Blacklist File
151 [Documentation] For each exclusion regex in the required @{${OPENSTACK_BRANCH}_exclusion_regexes} list a new
152 ... line will be created in the required ${blacklist_file} location. This file is pushed to the OS_CONTROL_NODE
153 ... which is assumed to be the tempest executor.
154 OperatingSystem.Create File ${blacklist_file}
155 : FOR ${exclusion} IN @{${OPENSTACK_BRANCH}_exclusion_regexes}
156 \ OperatingSystem.Append To File ${blacklist_file} ${exclusion}\n
157 Log File ${blacklist_file}
158 SSHKeywords.Copy File To Remote System ${OS_CONTROL_NODE_IP} ${blacklist_file} ${blacklist_file}