2 Documentation Test suite for OF-18 Spurious Leader Election verification of term change while flows are added/deleted #
3 Suite Setup ClusterManagement Setup
4 Suite Teardown Delete All Sessions
5 Resource ../../../libraries/BulkomaticKeywords.robot
6 Resource ../../../libraries/MininetKeywords.robot
7 Resource ../../../libraries/ClusterManagement.robot
8 Resource ../../../libraries/ClusterOpenFlow.robot
9 Resource ../../../libraries/Utils.robot
10 Variables ../../../variables/Variables.py
13 ${operation_timeout} 400s
14 ${restart_timeout} 450s
15 ${flow_count_per_switch} 10000
16 ${flow_count_per_switch_ten_percent} 1000
18 ${flow_count_after_add} 10000
19 ${flow_count_after_del} 0
20 ${flow_count_after_del_ten_percent} 9000
21 ${orig_json_config_add} sal_add_bulk_flow_config.json
22 ${orig_json_config_get} sal_get_bulk_flow_config.json
23 ${orig_json_config_del} sal_del_bulk_flow_config.json
24 ${orig_json_config_table_add} add_table.json
25 ${shard_name} inventory
27 ${verify_restconf} False
30 Check Shard And Get Inventory
31 [Documentation] Check Status for all shards and Finding a leader and followers in the inventory config shard
32 Check Shards Status And Initialize Variables
33 Get Inventory Follower
35 Initial Current Term Verification
36 [Documentation] Verifying current term for Leader Node Before
37 ${current_term_value_before} Get_Current_Term_Of_Shard_At_Member
38 BuiltIn.Log to console Current Term is ${current_term_value_before}
39 BuiltIn.Set Suite Variable ${current_term_value_before}
41 Add Bulk Flow From Follower
42 [Documentation] ${flow_count_after_add} Flows added via Follower Node1 and verify it gets applied in all instances.
43 BulkomaticKeywords.Add Bulk Flow In Node ${temp_json_config_add} ${Follower_Node_1} ${operation_timeout}
45 Get Bulk Flows And Verify In Cluster
46 [Documentation] Initiate get operation and check flow count across cluster nodes
47 BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_add} ${Inventory_Leader_List}
49 Current Term Verification After Adding Bulk Flow
50 [Documentation] Verifying current term for Leader Node after pushing the flows
51 ${current_term_value_after} Get_Current_Term_Of_Shard_At_Member
52 BuiltIn.Set Suite Variable ${current_term_value_after}
54 Current Term Comparison Before And After Addition Of Flow
55 [Documentation] Comparison of Current Term Before and After Addition of Flows
56 BuiltIn.Log to console Current Term after pushing the flows is ${current_term_value_after}
57 Run Keyword If ${current_term_value_before} == ${current_term_value_after} Log SUCCESS
59 Should Be Equal ${current_term_value_before} ${current_term_value_after}
61 Delete and Add ten percent of the flows for 5 iterations
62 [Documentation] Performeing 5 iterations for Delete and Add ten Percentage of the flows
63 : FOR ${index} IN RANGE 1 6
65 \ BulkomaticKeywords.Delete Bulk Flow In Node ${temp_json_config_del_ten_percent} ${Follower_Node_1} ${operation_timeout}
66 \ BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_del_ten_percent} ${Inventory_Leader_List}
67 \ BulkomaticKeywords.Add Bulk Flow In Node ${temp_json_config_add_ten_percent} ${Follower_Node_1} ${operation_timeout}
68 \ BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_add} ${Inventory_Leader_List}
70 Current Term Verification After Continuous Deletion and Addition Of Flows for 5 iterations
71 [Documentation] Verifying current term for Leader Node after continuous deletion and addition of ten percent of the flows
72 ${current_term_value_after} Get_Current_Term_Of_Shard_At_Member
73 BuiltIn.Set Suite Variable ${current_term_value_after}
75 Current Term Comparison Before and After Continuous Deletion and Addition Of Flows for 5 iterations
76 [Documentation] Comparison Current Term Before and After Continuous Deletion and Addition Of Flows for 5 iterations
77 BuiltIn.Log to console Current Term after pushing the flows is ${current_term_value_after}
78 Run Keyword If ${current_term_value_before} == ${current_term_value_after} Log SUCCESS
80 Should Be Equal ${current_term_value_before} ${current_term_value_after}
82 Delete All Flows From Follower Node
83 [Documentation] ${flow_count_after_add} Flows deleted via Leader Node and verify it gets applied in all instances.
84 BulkomaticKeywords.Delete Bulk Flow In Node ${temp_json_config_del} ${Follower_Node_1} ${operation_timeout}
86 Verify No Flows In Cluster After Flow Deletion
87 [Documentation] Verifying No FLows in Cluster after Flows Deletion
88 BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_del} ${Inventory_Leader_List}
90 PreLeader Verification
91 [Documentation] Verifying LastIndex and LastApplied and compare both are equal
92 ${LastIndex} Get_Last_Index_Of_Shard_At_Member
93 ${LastApplied} Get_Last_Applied_Of_Shard_At_Member
94 Should Be Equal ${LastIndex} ${LastApplied}
95 BulkomaticKeywords.Add Table In Node ${temp_json_config_table} ${Follower_Node_1} ${operation_timeout}
96 ${LastIndex} Get_Last_Index_Of_Shard_At_Member
97 ${LastApplied} Get_Last_Applied_Of_Shard_At_Member
98 Should Be Equal ${LastIndex} ${LastApplied}
101 Check Shards Status And Initialize Variables
102 [Documentation] Check Status for all shards in OpenFlow application.
103 ClusterOpenFlow.Check OpenFlow Shards Status
104 ${temp_json_config_add} BulkomaticKeywords.Set DPN And Flow Count In Json Add ${orig_json_config_add} ${switch_count} ${flow_count_per_switch}
105 ${temp_json_config_get} BulkomaticKeywords.Set DPN And Flow Count In Json Get ${orig_json_config_get} ${switch_count} ${flow_count_after_add}
106 ${temp_json_config_del} BulkomaticKeywords.Set DPN And Flow Count In Json Del ${orig_json_config_del} ${switch_count} ${flow_count_per_switch}
107 ${temp_json_config_del_ten_percent} BulkomaticKeywords.Set DPN And Flow Count In Json Del ${orig_json_config_del} ${switch_count} ${flow_count_per_switch_ten_percent}
108 ${temp_json_config_add_ten_percent} BulkomaticKeywords.Set DPN And Flow Count In Json Add ${orig_json_config_add} ${switch_count} ${flow_count_per_switch_ten_percent}
109 ${temp_json_config_table} BulkomaticKeywords.Set DPN And Flow Count In Json Add ${orig_json_config_table_add} ${switch_count} ${flow_count_per_switch}
110 Set Suite Variable ${temp_json_config_add}
111 Set Suite Variable ${temp_json_config_get}
112 Set Suite Variable ${temp_json_config_del}
113 Set Suite Variable ${temp_json_config_del_ten_percent}
114 Set Suite Variable ${temp_json_config_add_ten_percent}
115 Set Suite Variable ${temp_json_config_table}
117 Get Inventory Follower
118 [Documentation] Find a leader and followers in the inventory config shard
119 ${inventory_leader} ${inventory_followers} ClusterOpenFlow.Get InventoryConfig Shard Status
120 ${Follower_Node_1}= Get From List ${Inventory_Followers} 0
121 ${Follower_Node_2}= Get From List ${Inventory_Followers} 1
122 ${Inventory_Leader_List}= Create List ${inventory_leader}
123 ${Inventory_Follower_Node1_List}= Create List ${Follower_Node_1}
124 BuiltIn.Log to console ${\n}
125 BuiltIn.Log to console The follower Node1 is ${Follower_Node_1}
126 BuiltIn.Log to console The follower Node2 is ${Follower_Node_2}
127 BuiltIn.Log to console The leader Node is ${Inventory_Leader}
128 BuiltIn.Set Suite Variable ${Follower_Node_1}
129 BuiltIn.Set Suite Variable ${Follower_Node_2}
130 BuiltIn.Set Suite Variable ${Inventory_Leader}
131 BuiltIn.Set Suite Variable ${Inventory_Leader_List}
132 BuiltIn.Set Suite Variable ${Inventory_Follower_Node1_List}
134 Get_Current_Term_Of_Shard_At_Member
135 [Documentation] Find a Raft Property Values From Shard Member
136 ${current_term_value} = ClusterManagement.Get_Raft_Property_From_Shard_Member CurrentTerm ${shard_name} ${shard_type} ${Inventory_Leader} ${verify_restconf}
137 [Return] ${current_term_value}
139 Get_Last_Index_Of_Shard_At_Member
140 [Documentation] Find a leader and followers in the inventory config shard
141 ${LastIndex} = ClusterManagement.Get_Raft_Property_From_Shard_Member LastIndex ${shard_name} ${shard_type} ${Inventory_Leader} ${verify_restconf}
142 [Return] ${LastIndex}
144 Get_Last_Applied_Of_Shard_At_Member
145 [Documentation] Find a leader and followers in the inventory config shard
146 ${LastApplied} = ClusterManagement.Get_Raft_Property_From_Shard_Member LastApplied ${shard_name} ${shard_type} ${Inventory_Leader} ${verify_restconf}
147 [Return] ${LastApplied}