2 Documentation Test suite to test cluster connection and propagation switchover using virtual ip, this suite requires additional TOOLS_SYSTEM VM.
3 ... VM is used for its assigned ip-address that will be overlayed by virtual-ip used in test suites.
4 ... Resources of this VM are not required and after start of Test suite this node shutted down and to reduce routing conflicts.
5 Suite Setup Setup Custom SXP Cluster Session
6 Suite Teardown Clean Custom SXP Cluster Session
7 Test Teardown Clean SXP Cluster
8 Library ../../../libraries/Sxp.py
9 Resource ../../../libraries/ClusterManagement.robot
10 Resource ../../../libraries/SxpClusterLib.robot
13 Isolation of SXP service follower Test
14 [Documentation] Test SXP connection switchover only if Controller with SCS is isolated
15 SxpClusterLib.Check Shards Status
16 Setup Custom SXP Cluster ${VIRTUAL_IP} listener
17 ${controller_index} = SxpClusterLib.Get Active Controller
18 Isolate SXP Controller ${controller_index} listener
20 Isolation of SXP service follower Test Listener Part
21 [Documentation] Test SXP binding propagation only if Controller with SCS is isolated
22 SxpClusterLib.Check Shards Status
23 ${controller_index} = SxpClusterLib.Get Active Controller
24 Setup Custom SXP Cluster ${VIRTUAL_IP} listener
25 Setup SXP Cluster Bindings ${CLUSTER_NODE_ID} controller${controller_index}
26 ${controller_index} = SxpClusterLib.Get Active Controller
27 Isolate SXP Controller With Bindings ${controller_index} ${DEVICE_NODE_ID} listener ${DEVICE_SESSION}
29 Isolation of SXP service follower Test Speaker Part
30 [Documentation] Test SXP binding propagation only if Controller with SCS is isolated,
31 ... the same case as above but with initiator of connection between nodes in oposite mode
32 SxpClusterLib.Check Shards Status
33 Setup Custom SXP Cluster ${VIRTUAL_IP} speaker
34 Setup SXP Cluster Bindings ${DEVICE_NODE_ID} ${DEVICE_SESSION}
35 ${controller_index} = SxpClusterLib.Get Active Controller
36 Isolate SXP Controller With Bindings ${controller_index} ${CLUSTER_NODE_ID} speaker
39 Setup Custom SXP Cluster Session
40 [Documentation] Prepare topology for testing, creates sessions and generate Route definitions based on Cluster nodes ip
41 SxpClusterLib.Shutdown Tools Node
42 SxpClusterLib.Setup SXP Cluster Session
43 ${controller_index} = SxpClusterLib.Get Active Controller
44 ${mac_addresses} = SxpClusterLib.Map Followers To Mac Addresses
45 BuiltIn.Set Suite Variable ${MAC_ADDRESS_TABLE} ${mac_addresses}
46 ${route} = Sxp.Route Definition Xml ${VIRTUAL_IP} ${VIRTUAL_IP_MASK} ${VIRTUAL_INTERFACE}
47 ${routes} = Sxp.Route Definitions Xml ${route}
48 SxpLib.Put Routing Configuration To Controller ${routes} controller${controller_index}
50 Clean Custom SXP Cluster Session
51 [Documentation] Cleans up resources generated by test
52 ${controller_index} = SxpClusterLib.Get Active Controller
53 SxpLib.Clean Routing Configuration To Controller controller${controller_index}
54 SxpClusterLib.Clean SXP Cluster Session
56 Setup Custom SXP Cluster
57 [Arguments] ${peer_address} ${peer_mode}
58 [Documentation] Setup and connect SXP cluster topology
59 SxpLib.Add Node ${DEVICE_NODE_ID} ip=0.0.0.0 session=${DEVICE_SESSION}
60 BuiltIn.Wait Until Keyword Succeeds 20 1 SxpLib.Check Node Started ${DEVICE_NODE_ID} session=${DEVICE_SESSION} system=${TOOLS_SYSTEM_IP}
62 ${cluster_mode} = Sxp.Get Opposing Mode ${peer_mode}
63 SxpLib.Add Connection version4 ${peer_mode} ${peer_address} 64999 ${DEVICE_NODE_ID} session=${DEVICE_SESSION}
64 ${controller_id} = SxpClusterLib.Get Active Controller
65 SxpLib.Add Node ${CLUSTER_NODE_ID} ip=${peer_address} session=controller${controller_id}
66 BuiltIn.Wait Until Keyword Succeeds 20 1 SxpClusterLib.Check Cluster Node started ${CLUSTER_NODE_ID}
67 SxpLib.Add Connection version4 ${cluster_mode} ${TOOLS_SYSTEM_IP} 64999 ${CLUSTER_NODE_ID} session=controller${controller_id}
68 BuiltIn.Wait Until Keyword Succeeds 120 1 SxpClusterLib.Check Cluster is Connected ${CLUSTER_NODE_ID} mode=${cluster_mode} session=controller${controller_id}
70 Setup SXP Cluster Bindings
71 [Arguments] ${node} ${session}
72 [Documentation] Setup initial bindings to SXP device
73 : FOR ${i} IN RANGE 1 ${BINDINGS}
74 \ SxpLib.Add Bindings ${i}0 ${i}.${i}.${i}.${i}/32 node=${node} session=${session}
76 Isolate SXP Controller
77 [Arguments] ${controller_index} ${peer_mode}
78 [Documentation] Isolate one of cluster nodes and perform check that Device is still connected then revert isolation (and check connection again).
79 ${cluster_mode} = Sxp.Get Opposing Mode ${peer_mode}
80 ClusterManagement.Isolate_Member_From_List_Or_All ${controller_index}
81 BuiltIn.Wait Until Keyword Succeeds 240 1 ClusterManagement.Sync_Status_Should_Be_False ${controller_index}
82 BuiltIn.Wait Until Keyword Succeeds 240 1 SxpClusterLib.Ip Addres Should Not Be Routed To Follower ${MAC_ADDRESS_TABLE} ${VIRTUAL_IP} ${controller_index}
83 ${active_follower} = SxpClusterLib.Get Active Controller
84 BuiltIn.Wait Until Keyword Succeeds 240 1 SXpClusterLib.Ip Addres Should Be Routed To Follower ${MAC_ADDRESS_TABLE} ${VIRTUAL_IP} ${active_follower}
85 BuiltIn.Wait Until Keyword Succeeds 60 1 SxpClusterLib.Check Cluster is Connected ${CLUSTER_NODE_ID} mode=${cluster_mode} session=controller${active_follower}
86 BuiltIn.Wait Until Keyword Succeeds 60 1 Check Device is Connected ${DEVICE_NODE_ID} ${VIRTUAL_IP} ${peer_mode}
87 ... session=${DEVICE_SESSION}
88 ClusterManagement.Flush_Iptables_From_List_Or_All
89 BuiltIn.Wait Until Keyword Succeeds 240 1 ClusterManagement.Sync_Status_Should_Be_True ${controller_index}
90 BuiltIn.Wait Until Keyword Succeeds 60 1 SxpClusterLib.Check Cluster is Connected ${CLUSTER_NODE_ID} mode=${cluster_mode} session=controller${active_follower}
91 BuiltIn.Wait Until Keyword Succeeds 60 1 Check Device is Connected ${DEVICE_NODE_ID} ${VIRTUAL_IP} ${peer_mode}
92 ... session=${DEVICE_SESSION}
94 Isolate SXP Controller With Bindings
95 [Arguments] ${controller_index} ${node} ${peer_mode} ${session}=${EMPTY}
96 [Documentation] Isolate one of cluster nodes and perform check that bindings were propagated then revert isolation (and check connection again).
97 ${find_session} = BuiltIn.Set Variable If '${session}' == '${EMPTY}' ${True} ${False}
98 ${cluster_mode} = Sxp.Get Opposing Mode ${peer_mode}
99 ${session} = BuiltIn.Set Variable If ${find_session} controller${controller_index} ${session}
100 ClusterManagement.Isolate_Member_From_List_Or_All ${controller_index}
101 BuiltIn.Wait Until Keyword Succeeds 240 1 ClusterManagement.Sync_Status_Should_Be_False ${controller_index}
102 BuiltIn.Wait Until Keyword Succeeds 240 1 SxpClusterLib.Ip Addres Should Not Be Routed To Follower ${MAC_ADDRESS_TABLE} ${VIRTUAL_IP} ${controller_index}
103 ${active_follower} = SxpClusterLib.Get Active Controller
104 BuiltIn.Wait Until Keyword Succeeds 240 1 SXpClusterLib.Ip Addres Should Be Routed To Follower ${MAC_ADDRESS_TABLE} ${VIRTUAL_IP} ${active_follower}
105 BuiltIn.Wait Until Keyword Succeeds 60 1 SxpClusterLib.Check Cluster is Connected ${CLUSTER_NODE_ID} mode=${cluster_mode} session=controller${active_follower}
106 BuiltIn.Wait Until Keyword Succeeds 60 1 Check Device is Connected ${DEVICE_NODE_ID} ${VIRTUAL_IP} ${peer_mode}
107 ... session=${DEVICE_SESSION}
108 ${session} = BuiltIn.Set Variable If ${find_session} controller${active_follower} ${session}
109 BuiltIn.Wait Until Keyword Succeeds 30 1 Check Bindings ${node} ${session}
110 ClusterManagement.Flush_Iptables_From_List_Or_All
111 BuiltIn.Wait Until Keyword Succeeds 240 1 ClusterManagement.Sync_Status_Should_Be_True ${controller_index}
112 BuiltIn.Wait Until Keyword Succeeds 60 1 SxpClusterLib.Check Cluster is Connected ${CLUSTER_NODE_ID} mode=${cluster_mode} session=controller${active_follower}
113 BuiltIn.Wait Until Keyword Succeeds 60 1 Check Device is Connected ${DEVICE_NODE_ID} ${VIRTUAL_IP} ${peer_mode}
114 ... session=${DEVICE_SESSION}
115 BuiltIn.Wait Until Keyword Succeeds 30 1 Check Bindings ${node} ${session}
117 Check Device is Connected
118 [Arguments] ${node} ${remote_ip} ${mode}=any ${version}=version4 ${port}=64999 ${session}=session
119 [Documentation] Checks if SXP device is connected to at least one cluster node
120 ${resp} = SxpLib.Get Connections node=${node} session=${session}
121 SxpLib.Should Contain Connection ${resp} ${remote_ip} ${port} ${mode} ${version}
124 [Arguments] ${node} ${session}
125 [Documentation] Checks that bindings were propagated to Peer
126 ${resp} SxpLib.Get Bindings node=${node} session=${session}
127 : FOR ${i} IN RANGE 1 ${BINDINGS}
128 \ SxpLib.Should Contain Binding ${resp} ${i}0 ${i}.${i}.${i}.${i}/32