*** Settings ***
Documentation Test suite for Cluster HA - Device Owner failover
-Suite Setup ClusterManagement Setup
+Suite Setup SetupUtils.Setup_Utils_For_Setup_And_Teardown
Suite Teardown Delete All Sessions
+Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
Library RequestsLibrary
+Resource ../../../libraries/SetupUtils.robot
Resource ../../../libraries/ClusterOpenFlow.robot
Resource ../../../libraries/MininetKeywords.robot
Resource ../../../libraries/ClusterManagement.robot
Start Mininet Multiple Connections
[Documentation] Start mininet tree,2 with connection to all cluster instances.
- ${mininet_conn_id}= MininetKeywords.Start Mininet Multiple Controllers ${TOOLS_SYSTEM_IP} ${ClusterManagement__member_index_list} --topo tree,2 --switch ovsk,protocols=OpenFlow13
- Set Suite Variable ${mininet_conn_id}
-
-Check Entity Owner Status And Find Owner and Candidate Before Fail
- [Documentation] Check Entity Owner Status and identify owner and candidate for first switch s1.
- ${original_owner} ${original_candidate_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 1
- ${original_candidate}= Get From List ${original_candidate_list} 0
- Set Suite Variable ${original_owner}
- Set Suite Variable ${original_candidate_list}
- Set Suite Variable ${original_candidate}
-
-Reconnect Extra Switches To Candidate And Check Entity Owner
- [Documentation] Connect switches s2 and s3 to candidate instance.
- OVSDB.Set Controller In OVS Bridge ${TOOLS_SYSTEM_IP} s2 tcp:${ODL_SYSTEM_${original_candidate}_IP}:6633
- OVSDB.Set Controller In OVS Bridge ${TOOLS_SYSTEM_IP} s3 tcp:${ODL_SYSTEM_${original_candidate}_IP}:6633
- ${member_list} = BuiltIn.Run_Keyword_If '${ODL_STREAM}' != 'beryllium' and '${ODL_OF_PLUGIN}' == 'lithium' Create List @{ClusterManagement__member_index_list}
- ... ELSE Create List ${original_candidate}
- ${owner} ${candidate_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:2 1 ${member_list}
- Should Be Equal ${owner} ${original_candidate}
- ${owner} ${candidate_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:3 1 ${member_list}
- Should Be Equal ${owner} ${original_candidate}
+ ${mininet_conn_id}= MininetKeywords.Start Mininet Multiple Controllers ${TOOLS_SYSTEM_IP} ${ClusterManagement__member_index_list} --topo tree,2
+ BuiltIn.Set Suite Variable ${mininet_conn_id}
+ BuiltIn.Wait Until Keyword Succeeds 10s 1s OVSDB.Check OVS OpenFlow Connections ${TOOLS_SYSTEM_IP} 9
+
+Check Entity Owner Status And Find Owner and Successor Before Fail
+ [Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
+ ${original_owner} ${original_successor_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 1
+ ${original_successor}= Collections.Get From List ${original_successor_list} 0
+ BuiltIn.Set Suite Variable ${original_owner}
+ BuiltIn.Set Suite Variable ${original_successor_list}
+ BuiltIn.Set Suite Variable ${original_successor}
+
+Reconnect Extra Switches To Successors And Check OVS Connections
+ [Documentation] Connect switches s2 and s3 to successor instances.
+ ${controller_opt} = BuiltIn.Set Variable
+ : FOR ${index} IN @{original_successor_list}
+ \ ${controller_opt} = BuiltIn.Catenate ${controller_opt} ${SPACE}tcp:${ODL_SYSTEM_${index}_IP}:${ODL_OF_PORT}
+ \ Log ${controller_opt}
+ OVSDB.Set Controller In OVS Bridge ${TOOLS_SYSTEM_IP} s2 ${controller_opt}
+ OVSDB.Set Controller In OVS Bridge ${TOOLS_SYSTEM_IP} s3 ${controller_opt}
+ BuiltIn.Wait Until Keyword Succeeds 10s 1s OVSDB.Check OVS OpenFlow Connections ${TOOLS_SYSTEM_IP} 7
Check Network Operational Information Before Fail
[Documentation] Check devices in operational inventory and topology in all cluster instances.
[Documentation] Delete Flow in Owner and verify it gets applied from all instances.
ClusterOpenFlow.Delete Sample Flow and Verify ${original_owner}
-Add Configuration In Candidate and Verify Before Fail
- [Documentation] Add Flow in Candidate and verify it gets applied from all instances.
- ClusterOpenFlow.Add Sample Flow and Verify ${original_candidate}
+Add Configuration In Successor and Verify Before Fail
+ [Documentation] Add Flow in Successor and verify it gets applied from all instances.
+ ClusterOpenFlow.Add Sample Flow and Verify ${original_successor}
-Modify Configuration In Candidate and Verify Before Fail
- [Documentation] Modify Flow in candidate and verify it gets applied from all instances.
- ClusterOpenFlow.Modify Sample Flow and Verify ${original_candidate}
+Modify Configuration In Successor and Verify Before Fail
+ [Documentation] Modify Flow in successor and verify it gets applied from all instances.
+ ClusterOpenFlow.Modify Sample Flow and Verify ${original_successor}
-Delete Configuration In Candidate and Verify Before Fail
- [Documentation] Delete Flow in Candidate and verify it gets removed from all instances.
- ClusterOpenFlow.Delete Sample Flow and Verify ${original_candidate}
+Delete Configuration In Successor and Verify Before Fail
+ [Documentation] Delete Flow in Successor and verify it gets removed from all instances.
+ ClusterOpenFlow.Delete Sample Flow and Verify ${original_successor}
Send RPC Add to Owner and Verify Before Fail
[Documentation] Add Flow in Owner and verify it gets applied from all instances.
[Documentation] Delete Flow in Owner and verify it gets removed from all instances.
ClusterOpenFlow.Send RPC Delete Sample Flow and Verify ${original_owner}
-Send RPC Add to Candidate and Verify Before Fail
- [Documentation] Add Flow in Candidate and verify it gets applied from all instances.
- ClusterOpenFlow.Send RPC Add Sample Flow and Verify ${original_candidate}
+Send RPC Add to Successor and Verify Before Fail
+ [Documentation] Add Flow in Successor and verify it gets applied from all instances.
+ ClusterOpenFlow.Send RPC Add Sample Flow and Verify ${original_successor}
-Send RPC Delete to Candidate and Verify Before Fail
- [Documentation] Delete Flow in Candidate and verify it gets removed from all instances.
- ClusterOpenFlow.Send RPC Delete Sample Flow and Verify ${original_candidate}
+Send RPC Delete to Successor and Verify Before Fail
+ [Documentation] Delete Flow in Successor and verify it gets removed from all instances.
+ ClusterOpenFlow.Send RPC Delete Sample Flow and Verify ${original_successor}
Modify Network And Verify Before Fail
[Documentation] Take a link down and verify port status in all instances.
Kill Owner Instance
[Documentation] Kill Owner Instance and verify it is dead
ClusterManagement.Kill Single Member ${original_owner}
- Set Suite Variable ${new_cluster_list} ${original_candidate_list}
+ BuiltIn.Set Suite Variable ${new_cluster_list} ${original_successor_list}
Check Shards Status After Fail
[Documentation] Create original cluster list and check Status for all shards in OpenFlow application.
ClusterOpenFlow.Check OpenFlow Shards Status After Cluster Event ${new_cluster_list}
-Check Entity Owner Status And Find Owner and Candidate After Fail
- [Documentation] Check Entity Owner Status and identify owner and candidate.
- ${new_owner} ${new_candidate_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 ${original_candidate} ${new_cluster_list}
- ${new_candidate}= Get From List ${new_candidate_list} 0
- Set Suite Variable ${new_owner}
- Set Suite Variable ${new_candidate}
+Check Entity Owner Status And Find Owner and Successor After Fail
+ [Documentation] Check Entity Owner Status and identify owner and successor.
+ ${new_owner} ${new_successor_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 ${original_successor} ${new_cluster_list}
+ ${new_successor}= Collections.Get From List ${new_successor_list} 0
+ BuiltIn.Set Suite Variable ${new_owner}
+ BuiltIn.Set Suite Variable ${new_successor}
Check Network Operational Information After Fail
[Documentation] Check devices in operational inventory and topology in all cluster instances.
[Documentation] Delete Flow in Owner and verify it gets applied from all instances.
ClusterOpenFlow.Delete Sample Flow and Verify ${new_owner} ${new_cluster_list}
-Add Configuration In Candidate and Verify After Fail
- [Documentation] Add Flow in Candidate and verify it gets applied from all instances.
- ClusterOpenFlow.Add Sample Flow and Verify ${new_candidate} ${new_cluster_list}
+Add Configuration In Successor and Verify After Fail
+ [Documentation] Add Flow in Successor and verify it gets applied from all instances.
+ ClusterOpenFlow.Add Sample Flow and Verify ${new_successor} ${new_cluster_list}
-Modify Configuration In Candidate and Verify After Fail
- [Documentation] Modify Flow in Candidate and verify it gets applied from all instances.
- ClusterOpenFlow.Modify Sample Flow and Verify ${new_candidate} ${new_cluster_list}
+Modify Configuration In Successor and Verify After Fail
+ [Documentation] Modify Flow in Successor and verify it gets applied from all instances.
+ ClusterOpenFlow.Modify Sample Flow and Verify ${new_successor} ${new_cluster_list}
-Delete Configuration In Candidate and Verify After Fail
- [Documentation] Delete Flow in Candidate and verify it gets applied from all instances.
- ClusterOpenFlow.Delete Sample Flow and Verify ${new_candidate} ${new_cluster_list}
+Delete Configuration In Successor and Verify After Fail
+ [Documentation] Delete Flow in Successor and verify it gets applied from all instances.
+ ClusterOpenFlow.Delete Sample Flow and Verify ${new_successor} ${new_cluster_list}
Send RPC Add to Owner and Verify After Fail
[Documentation] Add Flow in Owner and verify it gets applied from all instances.
[Documentation] Delete Flow in Owner and verify it gets removed from all instances.
ClusterOpenFlow.Send RPC Delete Sample Flow and Verify ${new_owner} ${new_cluster_list}
-Send RPC Add to Candidate and Verify After Fail
- [Documentation] Add Flow in Candidate and verify it gets applied from all instances.
- ClusterOpenFlow.Send RPC Add Sample Flow and Verify ${new_candidate} ${new_cluster_list}
+Send RPC Add to Successor and Verify After Fail
+ [Documentation] Add Flow in Successor and verify it gets applied from all instances.
+ ClusterOpenFlow.Send RPC Add Sample Flow and Verify ${new_successor} ${new_cluster_list}
-Send RPC Delete to Candidate and Verify After Fail
- [Documentation] Delete Flow in Candidate and verify it gets removed from all instances.
- ClusterOpenFlow.Send RPC Delete Sample Flow and Verify ${new_candidate} ${new_cluster_list}
+Send RPC Delete to Successor and Verify After Fail
+ [Documentation] Delete Flow in Successor and verify it gets removed from all instances.
+ ClusterOpenFlow.Send RPC Delete Sample Flow and Verify ${new_successor} ${new_cluster_list}
Modify Network and Verify After Fail
[Documentation] Take a link down and verify port status in all instances.
ClusterOpenFlow.Check OpenFlow Shards Status After Cluster Event
Check Entity Owner Status After Recover
- [Documentation] Check Entity Owner Status and identify owner and candidate.
- ${new_owner} ${new_candidates_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 1
- Set Suite Variable ${new_owner}
+ [Documentation] Check Entity Owner Status and identify owner and successor.
+ ${new_owner} ${new_successors_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 1
+ BuiltIn.Set Suite Variable ${new_owner}
Check Network Operational Information After Recover
[Documentation] Check devices in operational inventory and topology in all cluster instances.
Code Review / integration / test.git / blobdiff