*** Settings ***
-Documentation Test suite to verify packet flows between vm instances.
-Suite Setup Devstack Suite Setup Tests source_pwd=yes
-Suite Teardown Close All Connections
-Library SSHLibrary
-Library OperatingSystem
-Library RequestsLibrary
-Library Collections
-Resource ../../../libraries/Utils.robot
-Resource ../../../libraries/OpenStackOperations.robot
-Resource ../../../libraries/DevstackUtils.robot
-Resource ../../../libraries/OVSDB.robot
-Library ../../../libraries/Common.py
-Variables ../../../variables/Variables.py
-Resource ../../../libraries/ClusterKeywords.robot
+Documentation Test suite to verify packet flows between vm instances.
+Suite Setup Devstack Suite Setup source_pwd=yes
+Suite Teardown Close All Connections
+Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+Test Teardown Get Test Teardown Debugs
+Library SSHLibrary
+Library OperatingSystem
+Library RequestsLibrary
+Library Collections
+Resource ../../../libraries/Utils.robot
+Resource ../../../libraries/OpenStackOperations.robot
+Resource ../../../libraries/DevstackUtils.robot
+Resource ../../../libraries/OVSDB.robot
+Resource ../../../libraries/ClusterOvsdb.robot
+Resource ../../../libraries/ClusterManagement.robot
+Resource ../../../libraries/SetupUtils.robot
+Variables ../../../variables/Variables.py
*** Variables ***
@{NETWORKS_NAME} l2_net_1 l2_net_2
@{SUBNETS_NAME} l2_sub_net_1 l2_sub_net_2
-@{NET_1_VM_INSTANCES} VmInstance1_l2_net_1 VmInstance2_net_1 VmInstance3_net_1
-@{NET_2_VM_INSTANCES} VmInstance1_l2_net_2 VmInstance2_net_2 VmInstance3_net_2
-@{NET_1_VM_IPS} 70.0.0.3 70.0.0.4 70.0.0.5
-@{NET_2_VM_IPS} 80.0.0.3 80.0.0.4 80.0.0.5
+@{NET_1_VM_INSTANCES} VmInstance1_l2_net_1 VmInstance2_net_1 VmInstance3_net_1
+@{NET_2_VM_INSTANCES} VmInstance1_l2_net_2 VmInstance2_net_2 VmInstance3_net_2
@{VM_IPS_NOT_DELETED} 70.0.0.4
-@{GATEWAY_IPS} 70.0.0.1 80.0.0.1
-@{DHCP_IPS} 70.0.0.2 80.0.0.2
-@{cluster_down_list} 1 2
+@{cluster_down_list} ${1} ${2}
@{SUBNETS_RANGE} 70.0.0.0/24 80.0.0.0/24
*** Test Cases ***
-Create Cluster List
- [Documentation] Create original cluster list.
- ${original_cluster_list} ClusterKeywords.Create Controller Index List
- Set Suite Variable ${original_cluster_list}
- Log ${original_cluster_list}
-
-Check OVS Manager Connection Status
- [Documentation] This will verify if the OVS manager is connected
- ${output}= Wait Until Keyword Succeeds 5s 1s Verify OVS Reports Connected ${OS_CONTROL_NODE_IP}
- Log ${output}
- Set Suite Variable ${status} is_connected: true
- ${dictionary}= Create Dictionary ${status}=9
- Utils.Check Item Occurrence ${output} ${dictionary}
+Create All Controller Sessions
+ [Documentation] Create sessions for all three contorllers.
+ ClusterManagement.ClusterManagement Setup
Create Networks
[Documentation] Create Network with neutron request.
: FOR ${NetworkElement} IN @{NETWORKS_NAME}
- \ Create Network ${NetworkElement}
+ \ OpenStackOperations.Create Network ${NetworkElement}
Create Subnets For l2_net_1
[Documentation] Create Sub Nets for the Networks with neutron request.
- Create SubNet l2_net_1 l2_sub_net_1 @{SUBNETS_RANGE}[0]
+ OpenStackOperations.Create SubNet l2_net_1 l2_sub_net_1 @{SUBNETS_RANGE}[0]
Create Subnets For l2_net_2
[Documentation] Create Sub Nets for the Networks with neutron request.
- Create SubNet l2_net_2 l2_sub_net_2 @{SUBNETS_RANGE}[1]
+ OpenStackOperations.Create SubNet l2_net_2 l2_sub_net_2 @{SUBNETS_RANGE}[1]
+
+Create Bridge Manually and Verify Before Fail
+ [Documentation] Create bridge with OVS command and verify it gets applied from all instances.
+ ClusterOvsdb.Create Sample Bridge Manually And Verify ${OS_CONTROL_NODE_IP}
+
+Add Tap Device Manually and Verify Before Fail
+ [Documentation] Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
+ ClusterOvsdb.Add Sample Tap Device To The Manual Bridge And Verify ${OS_CONTROL_NODE_IP}
+
+Delete the Bridge Manually and Verify Before Fail
+ [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances.
+ ClusterOvsdb.Delete Sample Bridge Manually And Verify ${OS_CONTROL_NODE_IP}
Take Down ODL1
- [Documentation] Kill the karaf in First Controller
- ClusterKeywords.Kill Multiple Controllers 1
- ${new_cluster_list} ClusterKeywords.Create Controller Index List
- Remove Values From List ${new_cluster_list} 1
+ [Documentation] Kill the karaf in First Controller
+ ${new_cluster_list} = ClusterManagement.Kill Single Member 1
Set Suite Variable ${new_cluster_list}
-Create Vm Instances For l2_net_1
- [Documentation] Create Vm instances using flavor and image names for a network.
- Log ${devstack_conn_id}
- Create Vm Instances l2_net_1 ${NET_1_VM_INSTANCES}
- [Teardown] Show Debugs ${NET_1_VM_INSTANCES}
+Create Bridge Manually and Verify After Fail
+ [Documentation] Create bridge with OVS command and verify it gets applied from all instances.
+ ClusterOvsdb.Create Sample Bridge Manually And Verify ${OS_CONTROL_NODE_IP} ${new_cluster_list}
+
+Add Tap Device Manually and Verify After Fail
+ [Documentation] Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
+ ClusterOvsdb.Add Sample Tap Device To The Manual Bridge And Verify ${OS_CONTROL_NODE_IP} ${new_cluster_list}
+
+Delete the Bridge Manually and Verify After Fail
+ [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances.
+ ClusterOvsdb.Delete Sample Bridge Manually And Verify ${OS_CONTROL_NODE_IP} ${new_cluster_list}
Bring Up ODL1
- [Documentation] Bring up ODL1 again
- ClusterKeywords.Start Multiple Controllers 300s 1
- ${new_cluster_list} ClusterKeywords.Create Controller Index List
- Set Suite Variable ${new_cluster_list}
+ [Documentation] Bring up ODL1 again
+ ClusterManagement.Start Single Member 1
+
+Create Bridge Manually and Verify After Recover
+ [Documentation] Create bridge with OVS command and verify it gets applied from all instances.
+ ClusterOvsdb.Create Sample Bridge Manually And Verify ${OS_CONTROL_NODE_IP}
+
+Add Tap Device Manually and Verify After Recover
+ [Documentation] Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
+ ClusterOvsdb.Add Sample Tap Device To The Manual Bridge And Verify ${OS_CONTROL_NODE_IP}
+
+Delete the Bridge Manually and Verify After Recover
+ [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances.
+ ClusterOvsdb.Delete Sample Bridge Manually And Verify ${OS_CONTROL_NODE_IP}
Take Down ODL2
- [Documentation] Kill the karaf in Second Controller
- ClusterKeywords.Kill Multiple Controllers 2
- ${new_cluster_list} ClusterKeywords.Create Controller Index List
- Remove Values From List ${new_cluster_list} 2
- Set Suite Variable ${new_cluster_list}
+ [Documentation] Kill the karaf in Second Controller
+ ClusterManagement.Kill Single Member 2
Create Vm Instances For l2_net_2
[Documentation] Create Vm instances using flavor and image names for a network.
- Create Vm Instances l2_net_2 ${NET_2_VM_INSTANCES}
- [Teardown] Show Debugs ${NET_2_VM_INSTANCES}
+ OpenStackOperations.Create Vm Instances l2_net_2 ${NET_2_VM_INSTANCES} sg=csit
+
+Create Vm Instances For l2_net_1
+ [Documentation] Create Vm instances using flavor and image names for a network.
+ Log ${devstack_conn_id}
+ OpenStackOperations.Create Vm Instances l2_net_1 ${NET_1_VM_INSTANCES} sg=csit
+
+Check Vm Instances Have Ip Address
+ [Documentation] Test case to verify that all created VMs are ready and have received their ip addresses.
+ ... We are polling first and longest on the last VM created assuming that if it's received it's address
+ ... already the other instances should have theirs already or at least shortly thereafter.
+ # first, ensure all VMs are in ACTIVE state. if not, we can just fail the test case and not waste time polling
+ # for dhcp addresses
+ : FOR ${vm} IN @{NET_1_VM_INSTANCES} @{NET_2_VM_INSTANCES}
+ \ Wait Until Keyword Succeeds 15s 5s Verify VM Is ACTIVE ${vm}
+ ${NET1_VM_COUNT} Get Length ${NET_1_VM_INSTANCES}
+ ${NET2_VM_COUNT} Get Length ${NET_2_VM_INSTANCES}
+ ${LOOP_COUNT} Evaluate ${NET1_VM_COUNT}+${NET2_VM_COUNT}
+ : FOR ${index} IN RANGE 1 ${LOOP_COUNT}
+ \ ${NET1_VM_IPS} ${NET1_DHCP_IP} Verify VMs Received DHCP Lease @{NET_1_VM_INSTANCES}
+ \ ${NET2_VM_IPS} ${NET2_DHCP_IP} Verify VMs Received DHCP Lease @{NET_2_VM_INSTANCES}
+ \ ${NET1_VM_LIST_LENGTH}= Get Length ${NET1_VM_IPS}
+ \ ${NET2_VM_LIST_LENGTH}= Get Length ${NET2_VM_IPS}
+ \ Exit For Loop If ${NET1_VM_LIST_LENGTH}==${NET1_VM_COUNT} and ${NET2_VM_LIST_LENGTH}==${NET2_VM_COUNT}
+ Append To List ${NET1_VM_IPS} ${NET1_DHCP_IP}
+ Set Suite Variable ${NET1_VM_IPS}
+ Append To List ${NET2_VM_IPS} ${NET2_DHCP_IP}
+ Set Suite Variable ${NET2_VM_IPS}
+ [Teardown] Run Keywords Show Debugs @{NET_1_VM_INSTANCES} @{NET_2_VM_INSTANCES}
+ ... AND Get Test Teardown Debugs
Bring Up ODL2
- [Documentation] Bring up ODL2 again
- ClusterKeywords.Start Multiple Controllers 300s 2
- ${new_cluster_list} ClusterKeywords.Create Controller Index List
- Set Suite Variable ${new_cluster_list}
+ [Documentation] Bring up ODL2 again
+ ClusterManagement.Start Single Member 2
Ping Vm Instance1 In l2_net_1
[Documentation] Check reachability of vm instances by pinging to them.
- Ping Vm From DHCP Namespace l2_net_1 @{NET_1_VM_IPS}[0]
+ Get OvsDebugInfo
+ OpenStackOperations.Ping Vm From DHCP Namespace l2_net_1 @{NET_1_VM_IPS}[0]
Ping Vm Instance2 In l2_net_1
[Documentation] Check reachability of vm instances by pinging to them.
- Ping Vm From DHCP Namespace l2_net_1 @{NET_1_VM_IPS}[1]
+ Get OvsDebugInfo
+ OpenStackOperations.Ping Vm From DHCP Namespace l2_net_1 @{NET_1_VM_IPS}[1]
Ping Vm Instance3 In l2_net_1
[Documentation] Check reachability of vm instances by pinging to them.
- Ping Vm From DHCP Namespace l2_net_1 @{NET_1_VM_IPS}[2]
+ Get OvsDebugInfo
+ OpenStackOperations.Ping Vm From DHCP Namespace l2_net_1 @{NET_1_VM_IPS}[2]
Ping Vm Instance1 In l2_net_2
[Documentation] Check reachability of vm instances by pinging to them.
- Ping Vm From DHCP Namespace l2_net_2 @{NET_2_VM_IPS}[0]
+ Get OvsDebugInfo
+ OpenStackOperations.Ping Vm From DHCP Namespace l2_net_2 @{NET_2_VM_IPS}[0]
Ping Vm Instance2 In l2_net_2
[Documentation] Check reachability of vm instances by pinging to them.
- Ping Vm From DHCP Namespace l2_net_2 @{NET_2_VM_IPS}[1]
+ Get OvsDebugInfo
+ OpenStackOperations.Ping Vm From DHCP Namespace l2_net_2 @{NET_2_VM_IPS}[1]
Ping Vm Instance3 In l2_net_2
[Documentation] Check reachability of vm instances by pinging to them.
- Ping Vm From DHCP Namespace l2_net_2 @{NET_2_VM_IPS}[2]
+ Get OvsDebugInfo
+ OpenStackOperations.Ping Vm From DHCP Namespace l2_net_2 @{NET_2_VM_IPS}[2]
Take Down ODL3
- [Documentation] Kill the karaf in Third Controller
- ClusterKeywords.Kill Multiple Controllers 3
- ${new_cluster_list} ClusterKeywords.Create Controller Index List
- Remove Values From List ${new_cluster_list} 3
- Set Suite Variable ${new_cluster_list}
+ [Documentation] Kill the karaf in Third Controller
+ ClusterManagement.Kill Single Member 3
Connectivity Tests From Vm Instance1 In l2_net_1
[Documentation] Logging to the vm instance using generated key pair.
- ${dst_ip_list}= Create List @{NET_1_VM_IPS}[1] @{DHCP_IPS}[0] @{NET_1_VM_IPS}[2]
- Log ${dst_ip_list}
- Test Operations From Vm Instance l2_net_1 @{NET_1_VM_IPS}[0] ${dst_ip_list}
+ OpenStackOperations.Test Operations From Vm Instance l2_net_1 @{NET_1_VM_IPS}[0] ${NET_1_VM_IPS}
Connectivity Tests From Vm Instance2 In l2_net_1
[Documentation] Logging to the vm instance using generated key pair.
- ${dst_ip_list}= Create List @{NET_1_VM_IPS}[0] @{DHCP_IPS}[0] @{NET_1_VM_IPS}[2]
- Log ${dst_ip_list}
- Test Operations From Vm Instance l2_net_1 @{NET_1_VM_IPS}[1] ${dst_ip_list}
+ OpenStackOperations.Test Operations From Vm Instance l2_net_1 @{NET_1_VM_IPS}[1] ${NET_1_VM_IPS}
Connectivity Tests From Vm Instance3 In l2_net_1
[Documentation] Logging to the vm instance using generated key pair.
- ${dst_ip_list}= Create List @{NET_1_VM_IPS}[0] @{DHCP_IPS}[0] @{NET_1_VM_IPS}[1]
- Log ${dst_ip_list}
- Test Operations From Vm Instance l2_net_1 @{NET_1_VM_IPS}[2] ${dst_ip_list}
+ OpenStackOperations.Test Operations From Vm Instance l2_net_1 @{NET_1_VM_IPS}[2] ${NET_1_VM_IPS}
Bring Up ODL3
- [Documentation] Bring up ODL3 again
- ClusterKeywords.Start Multiple Controllers 300s 3
- ${new_cluster_list} ClusterKeywords.Create Controller Index List
- Set Suite Variable ${new_cluster_list}
+ [Documentation] Bring up ODL3 again
+ ClusterManagement.Start Single Member 3
Take Down ODL1 and ODL2
- [Documentation] Kill the karaf in First and Second Controller
- : FOR ${index} IN @{cluster_down_list}
- \ ClusterKeywords.Kill Multiple Controllers ${index}
+ [Documentation] Kill the karaf in First and Second Controller
+ ClusterManagement.Kill Members From List Or All ${cluster_down_list}
+ [Teardown] Get OvsDebugInfo
Connectivity Tests From Vm Instance1 In l2_net_2
[Documentation] Logging to the vm instance using generated key pair.
- ${dst_ip_list}= Create List @{NET_2_VM_IPS}[1] @{DHCP_IPS}[1] @{NET_2_VM_IPS}[2]
- Log ${dst_ip_list}
- Test Operations From Vm Instance l2_net_2 @{NET_2_VM_IPS}[0] ${dst_ip_list}
+ OpenStackOperations.Test Operations From Vm Instance l2_net_2 @{NET_2_VM_IPS}[0] ${NET_2_VM_IPS}
+ [Teardown] Get OvsDebugInfo
Connectivity Tests From Vm Instance2 In l2_net_2
[Documentation] Logging to the vm instance using generated key pair.
- ${dst_ip_list}= Create List @{NET_2_VM_IPS}[0] @{DHCP_IPS}[1] @{NET_2_VM_IPS}[2]
- Log ${dst_ip_list}
- Test Operations From Vm Instance l2_net_2 @{NET_2_VM_IPS}[1] ${dst_ip_list}
+ OpenStackOperations.Test Operations From Vm Instance l2_net_2 @{NET_2_VM_IPS}[1] ${NET_2_VM_IPS}
+ [Teardown] Get OvsDebugInfo
Connectivity Tests From Vm Instance3 In l2_net_2
[Documentation] Logging to the vm instance using generated key pair.
- ${dst_ip_list}= Create List @{NET_2_VM_IPS}[0] @{DHCP_IPS}[1] @{NET_2_VM_IPS}[1]
- Log ${dst_ip_list}
- Test Operations From Vm Instance l2_net_2 @{NET_2_VM_IPS}[2] ${dst_ip_list}
+ OpenStackOperations.Test Operations From Vm Instance l2_net_2 @{NET_2_VM_IPS}[2] ${NET_2_VM_IPS}
+ [Teardown] Get OvsDebugInfo
Bring Up ODL1 and ODL2
- [Documentation] Bring up ODL1 and ODL2 again.
- : FOR ${index} IN @{cluster_down_list}
- \ ClusterKeywords.Start Multiple Controllers 300s ${index}
+ [Documentation] Bring up ODL1 and ODL2 again.
+ ClusterManagement.Start Members From List Or All ${cluster_down_list}
Delete Vm Instance
[Documentation] Delete Vm instances using instance names.
- Delete Vm Instance VmInstance1_l2_net_1
-
-Ping All Vm Instances
- [Documentation] Check reachability of vm instances by pinging to them.
- ${net_id}= Get Net Id l2_net_1
- : FOR ${VmIpElement} IN @{VM_IPS_NOT_DELETED}
- \ ${output}= Ping Vm From DHCP Namespace ${net_id} ${VmIpElement}
- \ Should Contain ${output} 64 bytes
+ OpenStackOperations.Delete Vm Instance VmInstance1_l2_net_1
No Ping For Deleted Vm
[Documentation] Check non reachability of deleted vm instances by pinging to them.
- ${output}= Ping Vm From DHCP Namespace ${net_id} @{NET_1_VM_IPS}[0]
- Should Contain ${output} Destination Host Unreachable
+ ${output}= OpenStackOperations.Ping From DHCP Should Not Succeed l2_network_1 @{NET_1_VM_IPS}[0]
Delete Vm Instances In network_1
[Documentation] Delete Vm instances using instance names in network_1.
: FOR ${VmElement} IN @{NET_1_VM_INSTANCES}
- \ Delete Vm Instance ${VmElement}
+ \ OpenStackOperations.Delete Vm Instance ${VmElement}
Delete Vm Instances In network_2
[Documentation] Delete Vm instances using instance names in network_2.
: FOR ${VmElement} IN @{NET_2_VM_INSTANCES}
- \ Delete Vm Instance ${VmElement}
+ \ OpenStackOperations.Delete Vm Instance ${VmElement}
Delete Sub Networks In network_1
[Documentation] Delete Sub Nets for the Networks with neutron request.
- Delete SubNet l2_sub_net_1
+ OpenStackOperations.Delete SubNet l2_sub_net_1
Delete Sub Networks In network_2
[Documentation] Delete Sub Nets for the Networks with neutron request.
- Delete SubNet l2_sub_net_2
+ OpenStackOperations.Delete SubNet l2_sub_net_2
Delete Networks
[Documentation] Delete Networks with neutron request.
: FOR ${NetworkElement} IN @{NETWORKS_NAME}
- \ Delete Network ${NetworkElement}
+ \ OpenStackOperations.Delete Network ${NetworkElement}