2 Documentation Test suite for Cluster HA - Device Owner failover
3 Suite Setup Create Controller Sessions
4 Suite Teardown Delete All Sessions
5 Library RequestsLibrary
6 Resource ../../../libraries/ClusterOpenFlow.robot
7 Resource ../../../libraries/ClusterKeywords.robot
8 Resource ../../../libraries/MininetKeywords.robot
9 Variables ../../../variables/Variables.py
12 Create Original Cluster List
13 [Documentation] Create original cluster list.
14 ${original_cluster_list} Create Controller Index List
15 Set Suite Variable ${original_cluster_list}
17 Check Shards Status Before Fail
18 [Documentation] Check Status for all shards in OpenFlow application.
19 Check OpenFlow Shards Status ${original_cluster_list}
21 Start Mininet Multiple Connections
22 [Documentation] Start mininet with connection to all cluster instances.
23 ${mininet_conn_id}= Start Mininet Multiple Controllers ${TOOLS_SYSTEM_IP} ${original_cluster_list}
24 Set Suite Variable ${mininet_conn_id}
26 Check Entity Owner Status And Find Owner and Candidate Before Fail
27 [Documentation] Check Entity Owner Status and identify owner and candidate.
28 ${original_owner} ${original_candidates_list} Get OpenFlow Entity Owner Status For One Device ${original_cluster_list}
29 ${original_candidate}= Get From List ${original_candidates_list} 0
30 Set Suite Variable ${original_owner}
31 Set Suite Variable ${original_candidate}
33 Check Network Operational Information Before Fail
34 [Documentation] Check device is in operational inventory and topology in all cluster instances.
35 Check OpenFlow Network Operational Information For One Device ${original_cluster_list}
37 Add Configuration In Owner and Verify Before Fail
38 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
39 Add Sample Flow and Verify ${original_cluster_list} ${original_owner}
41 Modify Configuration In Owner and Verify Before Fail
42 [Documentation] Modify Flow in Owner and verify it gets applied from all instances.
43 Modify Sample Flow and Verify ${original_cluster_list} ${original_owner}
45 Delete Configuration In Owner and Verify Before Fail
46 [Documentation] Delete Flow in Owner and verify it gets applied from all instances.
47 Delete Sample Flow and Verify ${original_cluster_list} ${original_owner}
49 Add Configuration In Candidate and Verify Before Fail
50 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
51 Add Sample Flow and Verify ${original_cluster_list} ${original_candidate}
53 Modify Configuration In Candidate and Verify Before Fail
54 [Documentation] Modify Flow in Owner and verify it gets applied from all instances.
55 Modify Sample Flow and Verify ${original_cluster_list} ${original_candidate}
57 Delete Configuration In Candidate and Verify Before Fail
58 [Documentation] Delete Flow in Owner and verify it gets removed from all instances.
59 Delete Sample Flow and Verify ${original_cluster_list} ${original_candidate}
61 Send RPC Add to Owner and Verify Before Fail
62 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
63 Send RPC Add Sample Flow and Verify ${original_cluster_list} ${original_owner}
65 Send RPC Delete to Owner and Verify Before Fail
66 [Documentation] Delete Flow in Owner and verify it gets removed from all instances.
67 Send RPC Delete Sample Flow and Verify ${original_cluster_list} ${original_owner}
69 Send RPC Add to Candidate and Verify Before Fail
70 [Documentation] Add Flow in Candidate and verify it gets applied from all instances.
71 Send RPC Add Sample Flow and Verify ${original_cluster_list} ${original_candidate}
73 Send RPC Delete to Candidate and Verify Before Fail
74 [Documentation] Delete Flow in Candidate and verify it gets removed from all instances.
75 Send RPC Delete Sample Flow and Verify ${original_cluster_list} ${original_candidate}
77 Modify Network And Verify Before Fail
78 [Documentation] Take a link down and verify port status in all instances.
79 Take OpenFlow Device Link Down and Verify ${original_cluster_list}
81 Restore Network And Verify Before Fail
82 [Documentation] Take the link up and verify port status in all instances.
83 Take OpenFlow Device Link Up and Verify ${original_cluster_list}
86 [Documentation] Kill Owner Instance and verify it is dead
87 Kill Multiple Controllers ${original_owner}
88 ${new_cluster_list} Create Controller Index List
89 Remove Values From List ${new_cluster_list} ${original_owner}
90 Set Suite Variable ${new_cluster_list}
92 Check Shards Status After Fail
93 [Documentation] Create original cluster list and check Status for all shards in OpenFlow application.
94 Check OpenFlow Shards Status ${new_cluster_list}
96 Check Entity Owner Status And Find Owner and Candidate After Fail
97 [Documentation] Check Entity Owner Status and identify owner and candidate.
98 ${new_owner} ${new_candidates_list} Get OpenFlow Entity Owner Status For One Device ${new_cluster_list}
99 ${new_candidate}= Get From List ${new_candidates_list} 0
100 Set Suite Variable ${new_owner}
101 Set Suite Variable ${new_candidate}
103 Check Network Operational Information After Fail
104 [Documentation] Check device is in operational inventory and topology in all cluster instances.
105 Check OpenFlow Network Operational Information For One Device ${new_cluster_list}
107 Add Configuration In Owner and Verify After Fail
108 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
109 Add Sample Flow and Verify ${new_cluster_list} ${new_owner}
111 Modify Configuration In Owner and Verify After Fail
112 [Documentation] Modify Flow in Owner and verify it gets applied from all instances.
113 Modify Sample Flow and Verify ${new_cluster_list} ${new_owner}
115 Delete Configuration In Owner and Verify After Fail
116 [Documentation] Delete Flow in Owner and verify it gets applied from all instances.
117 Delete Sample Flow and Verify ${new_cluster_list} ${new_owner}
119 Add Configuration In Candidate and Verify After Fail
120 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
121 Add Sample Flow and Verify ${new_cluster_list} ${new_candidate}
123 Modify Configuration In Candidate and Verify After Fail
124 [Documentation] Modify Flow in Owner and verify it gets applied from all instances.
125 Modify Sample Flow and Verify ${new_cluster_list} ${new_candidate}
127 Delete Configuration In Candidate and Verify After Fail
128 [Documentation] Delete Flow in Owner and verify it gets applied from all instances.
129 Delete Sample Flow and Verify ${new_cluster_list} ${new_candidate}
131 Send RPC Add to Owner and Verify After Fail
132 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
133 Send RPC Add Sample Flow and Verify ${new_cluster_list} ${new_owner}
135 Send RPC Delete to Owner and Verify After Fail
136 [Documentation] Delete Flow in Owner and verify it gets removed from all instances.
137 Send RPC Delete Sample Flow and Verify ${new_cluster_list} ${new_owner}
139 Send RPC Add to Candidate and Verify After Fail
140 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
141 Send RPC Add Sample Flow and Verify ${new_cluster_list} ${new_candidate}
143 Send RPC Delete to Candidate and Verify After Fail
144 [Documentation] Delete Flow in Owner and verify it gets removed from all instances.
145 Send RPC Delete Sample Flow and Verify ${new_cluster_list} ${new_candidate}
147 Modify Network and Verify After Fail
148 [Documentation] Take a link down and verify port status in all instances.
149 Take OpenFlow Device Link Down and Verify ${new_cluster_list}
151 Restore Network and Verify After Fail
152 [Documentation] Take the link up and verify port status in all instances.
153 Take OpenFlow Device Link Up and Verify ${new_cluster_list}
155 Start Old Owner Instance
156 [Documentation] Kill Owner Instance and verify it is dead
157 Start Multiple Controllers 300s ${original_owner}
159 Check Shards Status After Recover
160 [Documentation] Create original cluster list and check Status for all shards in OpenFlow application.
161 Wait Until Keyword Succeeds 5s 1s Check OpenFlow Shards Status ${original_cluster_list}
163 Check Entity Owner Status After Recover
164 [Documentation] Check Entity Owner Status and identify owner and candidate.
165 ${new_owner} ${new_candidates_list} Wait Until Keyword Succeeds 5s 1s Get OpenFlow Entity Owner Status For One Device ${original_cluster_list}
166 Set Suite Variable ${new_owner}
168 Check Network Operational Information After Recover
169 [Documentation] Check device is in operational inventory and topology in all cluster instances.
170 Check OpenFlow Network Operational Information For One Device ${original_cluster_list}
172 Add Configuration In Owner and Verify After Recover
173 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
174 Add Sample Flow and Verify ${original_cluster_list} ${new_owner}
176 Modify Configuration In Owner and Verify After Recover
177 [Documentation] Modify Flow in Owner and verify it gets applied from all instances.
178 Modify Sample Flow and Verify ${original_cluster_list} ${new_owner}
180 Delete Configuration In Owner and Verify After Recover
181 [Documentation] Delete Flow in Owner and verify it gets applied from all instances.
182 Delete Sample Flow and Verify ${original_cluster_list} ${new_owner}
184 Add Configuration In Old Owner and Verify After Recover
185 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
186 Add Sample Flow and Verify ${originalcluster_list} ${original_owner}
188 Modify Configuration In Old Owner and Verify After Recover
189 [Documentation] Modify Flow in Owner and verify it gets applied from all instances.
190 Modify Sample Flow and Verify ${original_cluster_list} ${original_owner}
192 Delete Configuration In Old Owner and Verify After Recover
193 [Documentation] Delete Flow in Owner and verify it gets applied from all instances.
194 Delete Sample Flow and Verify ${original_cluster_list} ${original_owner}
196 Send RPC Add to Owner and Verify After Recover
197 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
198 Send RPC Add Sample Flow and Verify ${original_cluster_list} ${new_owner}
200 Send RPC Delete to Owner and Verify After Recover
201 [Documentation] Delete Flow in Owner and verify it gets removed from all instances.
202 Send RPC Delete Sample Flow and Verify ${original_cluster_list} ${new_owner}
204 Send RPC Add to Old Owner and Verify After Recover
205 [Documentation] Add Flow in Owner and verify it gets applied from all instances.
206 Send RPC Add Sample Flow and Verify ${original_cluster_list} ${original_owner}
208 Send RPC Delete to Old Owner and Verify After Recover
209 [Documentation] Delete Flow in Owner and verify it gets removed from all instances.
210 Send RPC Delete Sample Flow and Verify ${original_cluster_list} ${original_owner}
212 Modify Network and Verify After Recover
213 [Documentation] Take a link down and verify port status in all instances.
214 Take OpenFlow Device Link Down and Verify ${original_cluster_list}
216 Restore Network and Verify After Recover
217 [Documentation] Take the link up and verify port status in all instances.
218 Take OpenFlow Device Link Up and Verify ${original_cluster_list}
220 Stop Mininet and Exit
221 [Documentation] Stop mininet and exit connection.
222 Stop Mininet And Exit ${mininet_conn_id}
225 Check No Network Operational Information
226 [Documentation] Check device is not in operational inventory or topology in all cluster instances.
227 Check No OpenFlow Network Operational Information ${original_cluster_list}