2 Documentation Test suite for OF-18 Spurious Leader Election verification of term change while flows are added/deleted #
3 Suite Setup SetupUtils.Setup_Utils_For_Setup_And_Teardown
4 Suite Teardown Delete All Sessions
5 Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
6 Resource ../../../libraries/SetupUtils.robot
7 Resource ../../../libraries/BulkomaticKeywords.robot
8 Resource ../../../libraries/MininetKeywords.robot
9 Resource ../../../libraries/ClusterManagement.robot
10 Resource ../../../libraries/ClusterOpenFlow.robot
11 Resource ../../../libraries/Utils.robot
12 Variables ../../../variables/Variables.py
15 ${operation_timeout} 400s
16 ${restart_timeout} 450s
17 ${flow_count_per_switch} 10000
18 ${flow_count_per_switch_ten_percent} 1000
20 ${flow_count_after_add} 10000
21 ${flow_count_after_del} 0
22 ${flow_count_after_del_ten_percent} 9000
23 ${orig_json_config_add} sal_add_bulk_flow_config.json
24 ${orig_json_config_get} sal_get_bulk_flow_config.json
25 ${orig_json_config_del} sal_del_bulk_flow_config.json
26 ${orig_json_config_table_add} add_table.json
27 ${shard_name} inventory
29 ${verify_restconf} False
32 Check Shard And Get Inventory
33 [Documentation] Check Status for all shards and Finding a leader and followers in the inventory config shard
34 Check Shards Status And Initialize Variables
35 Get Inventory Follower
37 Initial Current Term Verification
38 [Documentation] Verifying current term for Leader Node Before
39 ${current_term_value_before} Get_Current_Term_Of_Shard_At_Member
40 BuiltIn.Log to console Current Term is ${current_term_value_before}
41 BuiltIn.Set Suite Variable ${current_term_value_before}
43 Add Bulk Flow From Follower
44 [Documentation] ${flow_count_after_add} Flows added via Follower Node1 and verify it gets applied in all instances.
45 BulkomaticKeywords.Add Bulk Flow In Node ${temp_json_config_add} ${Follower_Node_1} ${operation_timeout}
47 Get Bulk Flows And Verify In Cluster
48 [Documentation] Initiate get operation and check flow count across cluster nodes
49 BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_add} ${Inventory_Leader_List}
51 Current Term Verification After Adding Bulk Flow
52 [Documentation] Verifying current term for Leader Node after pushing the flows
53 ${current_term_value_after} Get_Current_Term_Of_Shard_At_Member
54 BuiltIn.Set Suite Variable ${current_term_value_after}
56 Current Term Comparison Before And After Addition Of Flow
57 [Documentation] Comparison of Current Term Before and After Addition of Flows
58 BuiltIn.Log to console Current Term after pushing the flows is ${current_term_value_after}
59 Run Keyword If ${current_term_value_before} == ${current_term_value_after} Log SUCCESS
61 Should Be Equal ${current_term_value_before} ${current_term_value_after}
63 Delete and Add ten percent of the flows for 5 iterations
64 [Documentation] Performeing 5 iterations for Delete and Add ten Percentage of the flows
65 : FOR ${index} IN RANGE 1 6
67 \ BulkomaticKeywords.Delete Bulk Flow In Node ${temp_json_config_del_ten_percent} ${Follower_Node_1} ${operation_timeout}
68 \ BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_del_ten_percent} ${Inventory_Leader_List}
69 \ BulkomaticKeywords.Add Bulk Flow In Node ${temp_json_config_add_ten_percent} ${Follower_Node_1} ${operation_timeout}
70 \ BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_add} ${Inventory_Leader_List}
72 Current Term Verification After Continuous Deletion and Addition Of Flows for 5 iterations
73 [Documentation] Verifying current term for Leader Node after continuous deletion and addition of ten percent of the flows
74 ${current_term_value_after} Get_Current_Term_Of_Shard_At_Member
75 BuiltIn.Set Suite Variable ${current_term_value_after}
77 Current Term Comparison Before and After Continuous Deletion and Addition Of Flows for 5 iterations
78 [Documentation] Comparison Current Term Before and After Continuous Deletion and Addition Of Flows for 5 iterations
79 BuiltIn.Log to console Current Term after pushing the flows is ${current_term_value_after}
80 Run Keyword If ${current_term_value_before} == ${current_term_value_after} Log SUCCESS
82 Should Be Equal ${current_term_value_before} ${current_term_value_after}
84 Delete All Flows From Follower Node
85 [Documentation] ${flow_count_after_add} Flows deleted via Leader Node and verify it gets applied in all instances.
86 BulkomaticKeywords.Delete Bulk Flow In Node ${temp_json_config_del} ${Follower_Node_1} ${operation_timeout}
88 Verify No Flows In Cluster After Flow Deletion
89 [Documentation] Verifying No FLows in Cluster after Flows Deletion
90 BulkomaticKeywords.Get Bulk Flow And Verify Count In Cluster ${temp_json_config_get} ${operation_timeout} ${flow_count_after_del} ${Inventory_Leader_List}
92 PreLeader Verification
93 [Documentation] Verifying LastIndex and LastApplied and compare both are equal
94 ${LastIndex} Get_Last_Index_Of_Shard_At_Member
95 ${LastApplied} Get_Last_Applied_Of_Shard_At_Member
96 Should Be Equal ${LastIndex} ${LastApplied}
97 BulkomaticKeywords.Add Table In Node ${temp_json_config_table} ${Follower_Node_1} ${operation_timeout}
98 ${LastIndex} Get_Last_Index_Of_Shard_At_Member
99 ${LastApplied} Get_Last_Applied_Of_Shard_At_Member
100 Should Be Equal ${LastIndex} ${LastApplied}
103 Check Shards Status And Initialize Variables
104 [Documentation] Check Status for all shards in OpenFlow application.
105 ClusterOpenFlow.Check OpenFlow Shards Status
106 ${temp_json_config_add} BulkomaticKeywords.Set DPN And Flow Count In Json Add ${orig_json_config_add} ${switch_count} ${flow_count_per_switch}
107 ${temp_json_config_get} BulkomaticKeywords.Set DPN And Flow Count In Json Get ${orig_json_config_get} ${switch_count} ${flow_count_after_add}
108 ${temp_json_config_del} BulkomaticKeywords.Set DPN And Flow Count In Json Del ${orig_json_config_del} ${switch_count} ${flow_count_per_switch}
109 ${temp_json_config_del_ten_percent} BulkomaticKeywords.Set DPN And Flow Count In Json Del ${orig_json_config_del} ${switch_count} ${flow_count_per_switch_ten_percent}
110 ${temp_json_config_add_ten_percent} BulkomaticKeywords.Set DPN And Flow Count In Json Add ${orig_json_config_add} ${switch_count} ${flow_count_per_switch_ten_percent}
111 ${temp_json_config_table} BulkomaticKeywords.Set DPN And Flow Count In Json Add ${orig_json_config_table_add} ${switch_count} ${flow_count_per_switch}
112 Set Suite Variable ${temp_json_config_add}
113 Set Suite Variable ${temp_json_config_get}
114 Set Suite Variable ${temp_json_config_del}
115 Set Suite Variable ${temp_json_config_del_ten_percent}
116 Set Suite Variable ${temp_json_config_add_ten_percent}
117 Set Suite Variable ${temp_json_config_table}
119 Get Inventory Follower
120 [Documentation] Find a leader and followers in the inventory config shard
121 ${inventory_leader} ${inventory_followers} ClusterOpenFlow.Get InventoryConfig Shard Status
122 ${Follower_Node_1}= Get From List ${Inventory_Followers} 0
123 ${Follower_Node_2}= Get From List ${Inventory_Followers} 1
124 ${Inventory_Leader_List}= Create List ${inventory_leader}
125 ${Inventory_Follower_Node1_List}= Create List ${Follower_Node_1}
126 BuiltIn.Log to console ${\n}
127 BuiltIn.Log to console The follower Node1 is ${Follower_Node_1}
128 BuiltIn.Log to console The follower Node2 is ${Follower_Node_2}
129 BuiltIn.Log to console The leader Node is ${Inventory_Leader}
130 BuiltIn.Set Suite Variable ${Follower_Node_1}
131 BuiltIn.Set Suite Variable ${Follower_Node_2}
132 BuiltIn.Set Suite Variable ${Inventory_Leader}
133 BuiltIn.Set Suite Variable ${Inventory_Leader_List}
134 BuiltIn.Set Suite Variable ${Inventory_Follower_Node1_List}
136 Get_Current_Term_Of_Shard_At_Member
137 [Documentation] Find a Raft Property Values From Shard Member
138 ${current_term_value} = ClusterManagement.Get_Raft_Property_From_Shard_Member CurrentTerm ${shard_name} ${shard_type} ${Inventory_Leader} ${verify_restconf}
139 [Return] ${current_term_value}
141 Get_Last_Index_Of_Shard_At_Member
142 [Documentation] Find a leader and followers in the inventory config shard
143 ${LastIndex} = ClusterManagement.Get_Raft_Property_From_Shard_Member LastIndex ${shard_name} ${shard_type} ${Inventory_Leader} ${verify_restconf}
144 [Return] ${LastIndex}
146 Get_Last_Applied_Of_Shard_At_Member
147 [Documentation] Find a leader and followers in the inventory config shard
148 ${LastApplied} = ClusterManagement.Get_Raft_Property_From_Shard_Member LastApplied ${shard_name} ${shard_type} ${Inventory_Leader} ${verify_restconf}
149 [Return] ${LastApplied}