*** Settings ***
Documentation Test suite for running tempest tests. It is assumed that the test environment
... is already deployed and ready.
-Suite Setup Log In To Tempest Executor And Setup Test Environment
-Suite Teardown Clean Up After Running Tempest
+Suite Setup DevstackUtils.Log In To Tempest Executor And Setup Test Environment
+Suite Teardown DevstackUtils.Clean Up After Running Tempest
Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
Test Teardown Run Keywords Get Test Teardown Debugs
Test Template DevstackUtils.Run Tempest Tests
+Library OperatingSystem
Library SSHLibrary
Resource ../../../libraries/DevstackUtils.robot
Resource ../../../libraries/OpenStackOperations.robot
Resource ../../../libraries/SetupUtils.robot
-Resource ../../../libraries/Utils.robot
+Resource ../../../libraries/SSHKeywords.robot
Variables ../../../variables/Variables.py
-
-*** Variables ***
-${exclusion_regex} 'test_routers_negative.RoutersNegativeIpV6Test.test_router_set_gateway_used_ip_returns_409|test_routers_negative.RoutersNegativeTest.test_router_set_gateway_used_ip_returns_409'
-${tempest_config_file} /opt/stack/tempest/etc/tempest.conf
-${external_net_name} external-net
-${external_subnet_name} external-subnet
-${external_gateway} 10.10.10.250
-${external_subnet} 10.10.10.0/24
-${network_vlan_id} 167
+Resource ../../../variables/netvirt/Variables.robot
*** Test Cases ***
tempest.api.network
- # mitaka release and earlier would fail on the RoutersNegative* tests, so skipping here and running
- # explicitly in their own test cases which will skip_if_stable/mitaka is the openstack branch
- ${TEST_NAME} ${exclusion_regex}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file} timeout=900s
-tempest.api.network.test_routers_negative.RoutersNegativeIpV6Test
- [Tags] skip_if_stable/mitaka
- # OpenStack patch to fix this was merged in newton+ and will not be back-ported to mitaka
- # https://review.openstack.org/#/c/219215
- ${TEST_NAME}
+tempest.scenario.test_aggregates_basic_ops.TestAggregatesBasicOps.test_aggregate_basic_ops
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
-tempest.api.network.test_routers_negative.RoutersNegativeTest
- [Tags] skip_if_stable/mitaka
- # OpenStack patch to fix this was merged in newton+ and will not be back-ported to mitaka
- # https://review.openstack.org/#/c/219215
- ${TEST_NAME}
+tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_pause_unpause
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
+
+tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
+
+tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_rebuild
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
+
+tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
+
+tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_suspend_resume
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_connectivity_between_vms_on_different_networks
- ${TEST_NAME}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic
[Tags] skip_if_stable/mitaka
# Failing due to default security rules behavior missing in Mitaka
- ${TEST_NAME}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_mtu_sized_frames
- ${TEST_NAME}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
- ${TEST_NAME}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
+
+tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port
+ [Tags] skip_if_transparent skip_if_stable/mitaka
+ # Failing due to default security rules behavior missing in Mitaka, and also in all transparent runs
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_preserve_preexisting_port
- ${TEST_NAME}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_router_rescheduling
- ${TEST_NAME}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details
- ${TEST_NAME}
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
-tempest.scenario.test_aggregates_basic_ops.TestAggregatesBasicOps.test_aggregate_basic_ops
- ${TEST_NAME}
+tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os
+ [Tags] skip_if_transparent skip_if_learn
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
-tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_pause_unpause
- ${TEST_NAME}
-
-tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start
- ${TEST_NAME}
+tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
+ [Tags] skip_if_transparent skip_if_learn
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
-tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot
- ${TEST_NAME}
+tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os
+ [Tags] skip_if_transparent skip_if_learn
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
-tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_rebuild
- ${TEST_NAME}
+tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os
+ [Tags] skip_if_transparent skip_if_learn
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}
-tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_suspend_resume
- ${TEST_NAME}
+tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps
+ [Tags] skip_if_transparent skip_if_learn
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file} timeout=900s
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
- ${TEST_NAME}
-
-tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port
- [Tags] skip_if_transparent skip_if_stable/mitaka
- # Failing due to default security rules behavior missing in Mitaka, and also in all transparent runs
- ${TEST_NAME}
-
-*** Keywords ***
-Log In To Tempest Executor And Setup Test Environment
- [Documentation] Initialize SetupUtils, open SSH connection to a devstack system and source the openstack
- ... credentials needed to run the tempest tests
- SetupUtils.Setup_Utils_For_Setup_And_Teardown
- # source_pwd is expected to exist in the below Create Network, Create Subnet keywords. Might be a bug.
- ${source_pwd} Set Variable yes
- Set Suite Variable ${source_pwd}
- # Tempest tests need an existing external network in order to create routers.
- Create Network ${external_net_name} --router:external --provider:network_type=vlan --provider:physical_network=${PUBLIC_PHYSICAL_NETWORK} --provider:segmentation_id=${network_vlan_id}
- Create Subnet ${external_net_name} ${external_subnet_name} ${external_subnet} --gateway ${external_gateway}
- List Networks
- ${control_node_conn_id}= SSHLibrary.Open Connection ${OS_CONTROL_NODE_IP} prompt=${DEFAULT_LINUX_PROMPT_STRICT}
- Utils.Flexible SSH Login ${OS_USER}
- Write Commands Until Prompt source ${DEVSTACK_DEPLOY_PATH}/openrc admin admin
- Write Commands Until Prompt sudo rm -rf /opt/stack/tempest/.testrepository
- ${net_id}= Get Net Id ${external_net_name} ${control_node_conn_id}
- Tempest Conf Add External Network ${net_id}
-
-Tempest Conf Add External Network
- [Arguments] ${external_network_id}
- [Documentation] Tempest will be run with a config file - this function will add the
- ... given external network ID to the configuration file.
- Modify Config In File On Existing SSH Connection set network public_network_id ${external_network_id}
- Modify Config In File On Existing SSH Connection set DEFAULT debug False
- Modify Config In File On Existing SSH Connection set DEFAULT log_level INFO
- Write Commands Until Prompt sudo cat ${tempest_config_file}
- Write Commands Until Prompt sudo chmod 777 ${tempest_config_file}
-
-Modify Config In File On Existing SSH Connection
- [Arguments] ${config_file} ${modifier} ${config_section} ${config_key} ${config_value}=${EMPTY}
- [Documentation] uses crudini to populate oslo cofg file.
- # this keyword is only one line so seems like extra overhead, but this may be a good candidate to move
- # to a library at some point, when/if other suites need to use it, so wanted to make it generic.
- Write Commands Until Prompt sudo -E crudini --${modifier} ${config_file} ${config_section} ${config_key} ${config_value}
-
-Clean Up After Running Tempest
- [Documentation] Clean up any extra leftovers that were created to allow tempest tests to run.
- Delete Network ${external_net_name}
- List Networks
- Close All Connections
+ ${TEST_NAME} ${blacklist_file} ${tempest_config_file}