2 Documentation Test suite to check connectivity in L3 using routers.
3 Suite Setup OpenStackOperations.OpenStack Suite Setup
4 Suite Teardown OpenStackOperations.OpenStack Suite Teardown
5 Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
6 Test Teardown OpenStackOperations.Get Test Teardown Debugs
8 Library OperatingSystem
9 Library RequestsLibrary
11 Resource ../../../libraries/Utils.robot
12 Resource ../../../libraries/OpenStackOperations.robot
13 Resource ../../../libraries/DevstackUtils.robot
14 Resource ../../../libraries/OVSDB.robot
15 Resource ../../../libraries/ClusterOvsdb.robot
16 Resource ../../../libraries/ClusterManagement.robot
17 Resource ../../../libraries/SetupUtils.robot
18 Resource ../../../variables/Variables.robot
19 Resource ../../../variables/netvirt/Variables.robot
22 ${SECURITY_GROUP} cl3_sg
23 @{NETWORKS} cl3_net_1 cl3_net_2
24 @{SUBNETS} cl3_sub_1 cl3_sub_2
25 @{ROUTERS} cl3_router_1 cl3_router_2 cl3_router_3
26 @{NET_1_VMS} cl3_net_1_vm_1 cl3_net_1_vm_2 cl3_net_1_vm_3
27 @{NET_2_VMS} cl3_net_2_vm_1 cl3_net_2_vm_2 cl3_net_2_vm_3
28 @{SUBNET_CIDRS} 36.0.0.0/24 37.0.0.0/24
29 @{GATEWAY_IPS} 36.0.0.1 37.0.0.1
30 @{ODL_1_AND_2_DOWN} ${1} ${2}
31 @{ODL_2_AND_3_DOWN} ${2} ${3}
35 Create All Controller Sessions
36 [Documentation] Create sessions for all three controllers.
37 ClusterManagement.ClusterManagement Setup
38 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
40 Take Down Leader Of Default Shard
41 [Documentation] Stop the karaf on ODL cluster leader
42 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
43 ${cluster_leader} ${followers} = ClusterManagement.Get Leader And Followers For Shard shard_type=config
44 BuiltIn.Set Suite Variable ${cluster_leader}
45 ${new_cluster_list} = ClusterManagement.Stop Single Member ${cluster_leader} msg=up: ODL1, ODL2, ODL3, down=none
46 BuiltIn.Set Suite Variable ${new_cluster_list}
47 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
50 [Documentation] Create Network with neutron request.
51 FOR ${NetworkElement} IN @{NETWORKS}
52 OpenStackOperations.Create Network ${NetworkElement}
55 Create Subnets For net_1
56 [Documentation] Create Sub Nets for the Networks with neutron request.
57 OpenStackOperations.Create SubNet @{NETWORKS}[0] @{SUBNETS}[0] @{SUBNET_CIDRS}[0]
59 Create Subnets For net_2
60 [Documentation] Create Sub Nets for the Networks with neutron request.
61 OpenStackOperations.Create SubNet @{NETWORKS}[1] @{SUBNETS}[1] @{SUBNET_CIDRS}[1]
63 Bring Up Leader Of Default Shard
64 [Documentation] Bring up on cluster leader
65 ClusterManagement.Start Single Member ${cluster_leader} msg=up: ${new_cluster_list}, down: ${cluster_leader}
66 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
68 Add Ssh Allow All Rule
69 [Documentation] Allow all TCP/UDP/ICMP packets for this suite
70 OpenStackOperations.Create Allow All SecurityGroup ${SECURITY_GROUP}
73 [Documentation] Stop the karaf in First Controller
74 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
75 ClusterManagement.Stop Single Member 1 msg=up: ODL1, ODL2, ODL3, down=none
76 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
78 Create Vm Instances For net_1
79 [Documentation] Create Vm instances using flavor and image names for a network.
80 OpenStackOperations.Create Vm Instance On Compute Node @{NETWORKS}[0] @{NET_1_VMS}[0] ${OS_CMP1_HOSTNAME} sg=${SECURITY_GROUP}
81 OpenStackOperations.Create Vm Instance On Compute Node @{NETWORKS}[0] @{NET_1_VMS}[1] ${OS_CMP1_HOSTNAME} sg=${SECURITY_GROUP}
82 OpenStackOperations.Create Vm Instance On Compute Node @{NETWORKS}[0] @{NET_1_VMS}[2] ${OS_CMP2_HOSTNAME} sg=${SECURITY_GROUP}
85 [Documentation] Bring up ODL1 again
86 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
87 ClusterManagement.Start Single Member 1 msg=up: ODL2, ODL3, down: ODL1
88 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
91 [Documentation] Stop the karaf in Second Controller
92 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
93 ClusterManagement.Stop Single Member 2 msg=up: ODL1, ODL2, ODL3, down=none
94 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
96 Create Vm Instances For net_2
97 [Documentation] Create Vm instances using flavor and image names for a network.
98 OpenStackOperations.Create Vm Instance On Compute Node @{NETWORKS}[1] @{NET_2_VMS}[0] ${OS_CMP1_HOSTNAME} sg=${SECURITY_GROUP}
99 OpenStackOperations.Create Vm Instance On Compute Node @{NETWORKS}[1] @{NET_2_VMS}[1] ${OS_CMP2_HOSTNAME} sg=${SECURITY_GROUP}
100 OpenStackOperations.Create Vm Instance On Compute Node @{NETWORKS}[1] @{NET_2_VMS}[2] ${OS_CMP2_HOSTNAME} sg=${SECURITY_GROUP}
102 Check Vm Instances Have Ip Address
103 @{NET_1_L3_VM_IPS} ${NET_1_DHCP_IP} = OpenStackOperations.Get VM IPs @{NET_1_VMS}
104 @{NET_2_L3_VM_IPS} ${NET_2_DHCP_IP} = OpenStackOperations.Get VM IPs @{NET_2_VMS}
105 BuiltIn.Set Suite Variable @{NET_1_L3_VM_IPS}
106 BuiltIn.Set Suite Variable @{NET_2_L3_VM_IPS}
107 BuiltIn.Should Not Contain ${NET_1_L3_VM_IPS} None
108 BuiltIn.Should Not Contain ${NET_2_L3_VM_IPS} None
109 BuiltIn.Should Not Contain ${NET_1_DHCP_IP} None
110 BuiltIn.Should Not Contain ${NET_2_DHCP_IP} None
111 [Teardown] BuiltIn.Run Keywords OpenStackOperations.Show Debugs @{NET_1_VMS} @{NET_2_VMS}
112 ... AND OpenStackOperations.Get Test Teardown Debugs
115 [Documentation] Bring up ODL2 again
116 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
117 ClusterManagement.Start Single Member 2 msg=up: ODL1, ODL3, down: ODL2
118 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
121 [Documentation] Stop the karaf in Third Controller
122 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
123 ClusterManagement.Stop Single Member 3 msg=up: ODL1, ODL2, ODL3, down=none
124 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
126 Create Router router_2
127 [Documentation] Create Router and Add Interface to the subnets.
128 OpenStackOperations.Create Router @{ROUTERS}[1]
129 [Teardown] Report_Failure_Due_To_Bug 6117
131 Create Router router_3
132 [Documentation] Create Router and Add Interface to the subnets.
133 OpenStackOperations.Create Router @{ROUTERS}[2]
135 Add Interfaces To Router
136 [Documentation] Add Interfaces
137 FOR ${interface} IN @{SUBNETS}
138 OpenStackOperations.Add Router Interface @{ROUTERS}[2] ${interface}
141 Verify Created Routers
142 [Documentation] Check created routers using northbound rest calls
143 ${data} Utils.Get Data From URI 1 ${NEUTRON_ROUTERS_API}
145 Should Contain ${data} @{ROUTERS}[2]
148 [Documentation] Bring up ODL3 again
149 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
150 ClusterManagement.Start Single Member 3 msg=up: ODL1, ODL2, down: ODL3
151 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
153 Ping Vm Instance1 In net_2 From net_1
154 [Documentation] Check reachability of vm instances by pinging to them after creating routers.
155 OpenStackOperations.Ping Vm From DHCP Namespace @{NETWORKS}[0] @{NET_2_L3_VM_IPS}[0]
157 Ping Vm Instance2 In net_2 From net_1
158 [Documentation] Check reachability of vm instances by pinging to them after creating routers.
159 OpenStackOperations.Ping Vm From DHCP Namespace @{NETWORKS}[0] @{NET_2_L3_VM_IPS}[1]
161 Ping Vm Instance3 In net_2 From net_1
162 [Documentation] Check reachability of vm instances by pinging to them after creating routers.
163 OpenStackOperations.Ping Vm From DHCP Namespace @{NETWORKS}[0] @{NET_2_L3_VM_IPS}[2]
165 Ping Vm Instance1 In net_1 From net_2
166 [Documentation] Check reachability of vm instances by pinging to them after creating routers.
167 OpenStackOperations.Ping Vm From DHCP Namespace @{NETWORKS}[1] @{NET_1_L3_VM_IPS}[0]
169 Ping Vm Instance2 In net_1 From net_2
170 [Documentation] Check reachability of vm instances by pinging to them after creating routers.
171 OpenStackOperations.Ping Vm From DHCP Namespace @{NETWORKS}[1] @{NET_1_L3_VM_IPS}[1]
173 Ping Vm Instance3 In net_1 From net_2
174 [Documentation] Check reachability of vm instances by pinging to them after creating routers.
175 OpenStackOperations.Ping Vm From DHCP Namespace @{NETWORKS}[1] @{NET_1_L3_VM_IPS}[2]
177 Connectivity Tests From Vm Instance1 In net_1 In Healthy Cluster
178 [Documentation] ssh to the VM instance and test operations.
179 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
180 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[0] @{NET_1_L3_VM_IPS}[0] ${dst_list}
181 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
183 Connectivity Tests From Vm Instance2 In net_1 In Healthy Cluster
184 [Documentation] ssh to the VM instance and test operations.
185 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
186 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[0] @{NET_1_L3_VM_IPS}[1] ${dst_list}
187 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
189 Connectivity Tests From Vm Instance3 In net_1 In Healthy Cluster
190 [Documentation] ssh to the VM instance and test operations.
191 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
192 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[0] @{NET_1_L3_VM_IPS}[2] ${dst_list}
193 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
195 Take Down ODL1 and ODL2
196 [Documentation] Stop the karaf in First and Second Controller
197 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
198 ClusterManagement.Stop Single Member 1 msg=up: ODL1, ODL2, ODL3, down=none
199 ClusterManagement.Stop Single Member 2 msg=up: ODL2, ODL3, down=ODL1
200 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
201 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
203 Connectivity Tests From Vm Instance1 In net_1 With Two ODLs Down
204 [Documentation] ssh to the VM instance and test operations.
205 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
206 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[0] @{NET_1_L3_VM_IPS}[0] ${dst_list}
207 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
209 Connectivity Tests From Vm Instance2 In net_1 With Two ODLs Down
210 [Documentation] ssh to the VM instance and test operations.
211 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
212 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[0] @{NET_1_L3_VM_IPS}[1] ${dst_list}
213 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
215 Connectivity Tests From Vm Instance3 In net_1 With Two ODLs Down
216 [Documentation] ssh to the VM instance and test operations.
217 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
218 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[0] @{NET_1_L3_VM_IPS}[2] ${dst_list}
219 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
221 Bring Up ODL1 and ODL2
222 [Documentation] Bring up ODL1 and ODL2 again. Do not check for cluster sync until all nodes are
223 ... up. akka will not let nodes join until they are all back up if two were down.
224 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
225 ClusterManagement.Start Single Member 1 msg=up: ODL3, down: ODL1, ODL2 wait_for_sync=False
226 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
227 ClusterManagement.Start Single Member 2 msg=up: ODL1, ODL3, down: ODL2
228 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
229 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
231 Take Down ODL2 and ODL3
232 [Documentation] Stop the karaf in First and Second Controller
233 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
234 ClusterManagement.Stop Single Member 2 msg=up: ODL1, ODL2, ODL3, down=none
235 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
236 ClusterManagement.Stop Single Member 3 msg=up: ODL1, ODL3, down=ODL2
237 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
238 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
240 Connectivity Tests From Vm Instance1 In net_2
241 [Documentation] ssh to the VM instance and test operations.
242 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
243 BuiltIn.Wait Until Keyword Succeeds 30s 10s OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[1] @{NET_2_L3_VM_IPS}[0] ${dst_list}
244 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
246 Connectivity Tests From Vm Instance2 In net_2
247 [Documentation] ssh to the VM instance and test operations.
248 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
249 BuiltIn.Wait Until Keyword Succeeds 30s 10s OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[1] @{NET_2_L3_VM_IPS}[1] ${dst_list}
250 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
252 Connectivity Tests From Vm Instance3 In net_2
253 [Documentation] ssh to the VM instance and test operations.
254 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
255 BuiltIn.Wait Until Keyword Succeeds 30s 10s OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[1] @{NET_2_L3_VM_IPS}[2] ${dst_list}
256 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
258 Bring Up ODL2 and ODL3
259 [Documentation] Bring up ODL2 and ODL3 again. Do not check for cluster sync until all nodes are
260 ... up. akka will not let nodes join until they are all back up if two were down.
261 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
262 ClusterManagement.Start Single Member 2 msg=up: ODL1, down: ODL2, ODL3 wait_for_sync=False
263 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
264 ClusterManagement.Start Single Member 3 msg=up: ODL1, ODL2, down: ODL3
265 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
266 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
268 Take Down All Instances
269 [Documentation] Stop karaf on all controllers
270 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
271 ClusterManagement.Stop_Members_From_List_Or_All
272 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
273 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
275 Bring Up All Instances
276 [Documentation] Bring up all controllers. Do not check for cluster sync until all nodes are
277 ... up. akka will not let nodes join until they are all back up if two were down.
278 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
279 ClusterManagement.Start Members From List Or All
280 BuiltIn.Run Keyword And Ignore Error ClusterManagement.Get Raft State Of Shard Of All Member Nodes shard_name=default shard_type=config
281 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
283 Connectivity Tests From Vm Instance2 In net_2 after recovering all nodes
284 [Documentation] ssh to the VM instance and test operations.
285 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
286 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[1] @{NET_2_L3_VM_IPS}[1] ${dst_list}
287 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
289 Connectivity Tests From Vm Instance3 In net_2 after recovering all nodes
290 [Documentation] ssh to the VM instance and test operations.
291 ${dst_list} = BuiltIn.Create List @{NET_2_L3_VM_IPS} @{NET_1_L3_VM_IPS}
292 OpenStackOperations.Test Operations From Vm Instance @{NETWORKS}[1] @{NET_2_L3_VM_IPS}[2] ${dst_list}
293 [Teardown] OpenStackOperations.Get Test Teardown Debugs fail=False
295 Delete Vm Instances In net_1
296 [Documentation] Delete Vm instances using instance names in net_1.
297 FOR ${vm} IN @{NET_1_VMS}
298 OpenStackOperations.Delete Vm Instance ${vm}
301 Delete Vm Instances In net_2
302 [Documentation] Delete Vm instances using instance names in net_2.
303 FOR ${vm} IN @{NET_2_VMS}
304 OpenStackOperations.Delete Vm Instance ${vm}
307 Delete Router Interfaces
308 [Documentation] Remove Interface to the subnets.
309 FOR ${interface} IN @{SUBNETS}
310 OpenStackOperations.Remove Interface @{ROUTERS}[2] ${interface}
314 [Documentation] Delete Router and Interface to the subnets.
315 OpenStackOperations.Delete Router @{ROUTERS}[1]
316 OpenStackOperations.Delete Router @{ROUTERS}[2]
318 Verify Deleted Routers
319 [Documentation] Check deleted routers using northbound rest calls
320 ${data} = Utils.Get Data From URI 1 ${NEUTRON_ROUTERS_API}
322 BuiltIn.Should Not Contain ${data} @{ROUTERS}[2]
324 Delete Sub Network In net_1
325 [Documentation] Delete Sub Nets for the Networks with neutron request.
326 OpenStackOperations.Delete SubNet @{SUBNETS}[0]
328 Delete Sub Network In net_2
329 [Documentation] Delete Sub Nets for the Networks with neutron request.
330 OpenStackOperations.Delete SubNet @{SUBNETS}[1]
333 [Documentation] Delete Networks with neutron request.
334 FOR ${network} IN @{NETWORKS}
335 OpenStackOperations.Delete Network ${network}
338 Delete Security Group
339 [Documentation] Delete security groups with neutron request
340 OpenStackOperations.Delete SecurityGroup ${SECURITY_GROUP}
343 [Documentation] Verify that flows have been cleaned up properly after removing all neutron configurations
344 DataModels.Verify Flows Are Cleaned Up On All OpenStack Nodes