*** Settings ***
-Documentation Switch connections and cluster are restarted.
-Suite Setup Initialization Phase
-Suite Teardown Final Phase
-Library RequestsLibrary
-Resource ../../../libraries/ClusterManagement.robot
-Resource ../../../libraries/ClusterOpenFlow.robot
-Resource ../../../libraries/TemplatedRequests.robot
-Resource ../../../libraries/MininetKeywords.robot
-Resource ../../../libraries/Utils.robot
-Resource ../../../variables/Variables.robot
+Documentation Switch connections and cluster are restarted.
+
+Library RequestsLibrary
+Library Collections
+Resource ../../../libraries/SetupUtils.robot
+Resource ../../../libraries/ClusterManagement.robot
+Resource ../../../libraries/ClusterOpenFlow.robot
+Resource ../../../libraries/TemplatedRequests.robot
+Resource ../../../libraries/MininetKeywords.robot
+Resource ../../../libraries/Utils.robot
+Resource ../../../libraries/OvsManager.robot
+Resource ../../../variables/Variables.robot
+Resource ../../../variables/openflowplugin/Variables.robot
+
+Suite Setup Initialization Phase
+Suite Teardown Final Phase
+Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+
*** Variables ***
-${SWITCHES} 3
-${ITER} 100
-${VAR_DIR} ${CURDIR}/../../../variables/openflowplugin
+${SWITCHES} 3
+${ITER} 100
+${VAR_DIR} ${CURDIR}/../../../variables/openflowplugin
-*** Test Cases ***
-Enable Stale Flow Entry
- [Documentation] Enable stale flow entry feature.
- # Stale flows/groups feature is only available in Boron onwards.
- CompareStream.Run Keyword If At Least Boron TemplatedRequests.Put As Json Templated folder=${VAR_DIR}/frm-config mapping={"STALE":"true"} session=session
+*** Test Cases ***
Add Groups And Flows
[Documentation] Add ${ITER} groups 1&2 and flows in every switch.
- Add Groups And Flows ${ITER}
+ Add Groups And Flows On Member ${ITER}
Start Mininet Multiple Connections
[Documentation] Start mininet linear with connection to all cluster instances.
- ${mininet_conn_id}= MininetKeywords.Start Mininet Multiple Controllers ${TOOLS_SYSTEM_IP} ${ClusterManagement__member_index_list} --topo linear,${SWITCHES} --switch ovsk,protocols=OpenFlow13
+ ${cluster_index_list}= ClusterManagement.List All Indices
+ ${mininet_conn_id}= MininetKeywords.Start Mininet Multiple Controllers
+ ... ${TOOLS_SYSTEM_IP}
+ ... ${cluster_index_list}
+ ... --topo linear,${SWITCHES}
+ BuiltIn.Set Suite Variable ${cluster_index_list}
BuiltIn.Set Suite Variable ${mininet_conn_id}
- BuiltIn.Wait Until Keyword Succeeds 10s 1s OVSDB.Check OVS OpenFlow Connections ${TOOLS_SYSTEM_IP} ${SWITCHES*3}
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... OVSDB.Check OVS OpenFlow Connections
+ ... ${TOOLS_SYSTEM_IP}
+ ... ${SWITCHES*3}
Check Linear Topology
[Documentation] Check Linear Topology.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Linear Topology ${SWITCHES}
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check Linear Topology On Member ${SWITCHES}
+
+Check Stats Are Not Frozen
+ [Documentation] Check that duration flow stat is increasing.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Flow Stats Are Not Frozen
Check Flows In Operational DS
- [Documentation] Check Groups after mininet starts.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Number Of Flows ${all_flows}
+ [Documentation] Check Flows in operational DS.
+ BuiltIn.Wait Until Keyword Succeeds 10s 1s ClusterOpenFlow.Check Number Of Flows On Member ${all_flows}
Check Groups In Operational DS
- [Documentation] Check Flows after mininet starts.
- Check Number Of Groups ${all_groups}
+ [Documentation] Check Groups in operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${all_groups}
Check Flows In Switch
- [Documentation] Check Flows in switch after mininet starts.
+ [Documentation] Check Flows in switch.
MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${all_flows}
Check Entity Owner Status And Find Owner and Successor Before Fail
[Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
- ${original_owner} ${original_successor_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 1
+ ${original_owner} ${original_successor_list}= ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... 1
BuiltIn.Set Suite Variable ${original_owner}
+ BuiltIn.Set Suite Variable ${new_cluster_list} ${original_successor_list}
Disconnect Mininet From Owner
[Documentation] Disconnect mininet from the owner
- ${owner_list} BuiltIn.Create List ${original_owner}
- Disconnect Cluster Mininet break ${owner_list}
- BuiltIn.Set Suite Variable ${owner_list}
+ ${original_owner_list}= BuiltIn.Create List ${original_owner}
+ MininetKeywords.Disconnect Cluster Mininet break ${original_owner_list}
+ BuiltIn.Set Suite Variable ${original_owner_list}
Check Entity Owner Status And Find Owner and Successor After Fail
[Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
- ${new_owner} ${new_successor_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 1
+ ${new_owner} ${new_successor_list}= BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... 1
+ ... ${new_cluster_list}
+ ... after_stop=True
+ ${owner_list}= BuiltIn.Create List ${original_owner} ${new_owner}
+ BuiltIn.Set Suite Variable ${owner_list}
+ BuiltIn.Set Suite Variable ${new_owner}
+ BuiltIn.Set Suite Variable ${new_successor_list}
+
+Check Switch Moves To New Master
+ [Documentation] Check switch s1 is connected to new Master.
+ ${new_master}= BuiltIn.Set Variable ${ODL_SYSTEM_${new_owner}_IP}
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... OvsManager.Should Be Master
+ ... s1
+ ... ${new_master}
+ ... update_data=${True}
Check Linear Topology After Disconnect
- [Documentation] Check Linear Topology After Disconnecting mininet from owner.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Linear Topology ${SWITCHES}
+ [Documentation] Check Linear Topology.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check Linear Topology On Member ${SWITCHES}
+
+Check Stats Are Not Frozen After Disconnect
+ [Documentation] Check that duration flow stat is increasing
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Flow Stats Are Not Frozen
Remove Flows And Groups After Mininet Is Disconnected
- [Documentation] Remove 1 group 1&2 and 1 flow in every switch after mininet is disconnected.
- Remove Single Group And Flow
+ [Documentation] Remove 1 group 1&2 and 1 flow in every switch.
+ Remove Single Group And Flow On Member
Check Flows In Operational DS After Mininet Is Disconnected
- [Documentation] Check Flows in Operational DS after mininet is disconnected.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Number Of Flows ${less_flows}
+ [Documentation] Check Flows in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 30s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Flows On Member
+ ... ${less_flows}
Check Groups In Operational DS After Mininet Is Disconnected
- [Documentation] Check Groups in Operational DS after mininet is disconnected.
- Check Number Of Groups ${less_groups}
+ [Documentation] Check Groups in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${less_groups}
Check Flows In Switch After Mininet Is Disconnected
- [Documentation] Check Flows in switch after mininet is disconnected
+ [Documentation] Check Flows in switch.
MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${less_flows}
Reconnect Mininet To Owner
[Documentation] Reconnect mininet to switch 1 owner.
- Disconnect Cluster Mininet restore ${owner_list}
-
-Check Linear Topology After Reconnect
- [Documentation] Check Linear Topology After Reconnect.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Linear Topology ${SWITCHES}
+ MininetKeywords.Disconnect Cluster Mininet restore ${original_owner_list}
-Add Flows And Groups After Reconnect
+Check Entity Owner Status And Find Owner and Successor After Reconnect
+ [Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... 1
+
+Add Flows And Groups After Owner Reconnect
[Documentation] Add 1 group type 1&2 and 1 flow in every switch.
- Add Single Group And Flow
+ Add Single Group And Flow On Member
+
+Check Stats Are Not Frozen After Owner Reconnect
+ [Documentation] Check that duration flow stat is increasing.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Flow Stats Are Not Frozen
+
+Check Flows After Owner Reconnect In Operational DS
+ [Documentation] Check Flows in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check Number Of Flows On Member ${all_flows}
+
+Check Groups After Owner Reconnect In Operational DS
+ [Documentation] Check Groups in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${all_groups}
+
+Check Flows After Owner Reconnect In Switch
+ [Documentation] Check Flows in switch.
+ MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${all_flows}
-Check Flows After Reconnect In Operational DS
- [Documentation] Check Flows in Operational DS after mininet is reconnected.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Number Of Flows ${all_flows}
+Check Switches Generate Slave Connection
+ [Documentation] Check switches are connected to new Slave.
+ ${original_master}= BuiltIn.Set Variable ${ODL_SYSTEM_${original_owner}_IP}
+ FOR ${switch} IN RANGE 1 ${switches+1}
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 50s
+ ... 1s
+ ... OvsManager.Should Be Slave
+ ... s${switch}
+ ... ${original_master}
+ ... update_data=${True}
+ END
+
+Disconnect Mininet From Successor
+ [Documentation] Disconnect mininet from the Successor.
+ MininetKeywords.Disconnect Cluster Mininet break ${new_successor_list}
+
+Check Entity Owner Status And Find New Owner and Successor After Disconnect
+ [Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
+ ${current_owner} ${current_successor_list}= BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... 1
+ ... ${owner_list}
+ ... after_stop=True
+ BuiltIn.Set Suite Variable ${current_owner}
+ BuiltIn.Set Suite Variable ${current_successor_list}
+
+Disconnect Mininet From Current Owner
+ [Documentation] Disconnect mininet from the owner
+ ${current_owner_list}= BuiltIn.Create List ${current_owner}
+ MininetKeywords.Disconnect Cluster Mininet break ${current_owner_list}
+
+Check Entity Owner Status And Find Current Owner and Successor After Disconnect
+ [Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
+ ${current_new_owner} ${current_new_successor_list}= BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... 1
+ ... ${original_owner_list}
+ ... after_stop=True
+ BuiltIn.Set Suite Variable ${current_new_owner}
+ BuiltIn.Set Suite Variable ${current_new_successor_list}
+
+Check Switch Moves To Current Master
+ [Documentation] Check switch s1 is connected to original Master.
+ ${current_new_master}= BuiltIn.Set Variable ${ODL_SYSTEM_${current_new_owner}_IP}
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... OvsManager.Should Be Master
+ ... s1
+ ... ${current_new_master}
+ ... update_data=${True}
+ BuiltIn.Should Be Equal ${current_new_owner} ${original_owner}
+
+Check Linear Topology After Owner Disconnect
+ [Documentation] Check Linear Topology.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check Linear Topology On Member ${SWITCHES}
-Check Groups After Reconnect In Operational DS
- [Documentation] Check Groups in Operational DS after mininet is reconnected.
- Check Number Of Groups ${all_groups}
+Check Stats Are Not Frozen After Owner Disconnect
+ [Documentation] Check that duration flow stat is increasing.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Flow Stats Are Not Frozen
-Check Flows After Reconnect In Switch
- [Documentation] Check Flows in switch after mininet is reconnected.
- MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${all_flows}
+Remove Flows And Groups After Owner Disconnected
+ [Documentation] Remove 1 group 1&2 and 1 flow in every switch.
+ Remove Single Group And Flow On Member
+
+Check Flows In Operational DS After Owner Disconnected
+ [Documentation] Check Flows in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 30s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Flows On Member
+ ... ${less_flows}
+
+Check Groups In Operational DS After Owner Disconnected
+ [Documentation] Check Groups in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${less_groups}
+
+Check Flows In Switch After Owner Disconnected
+ [Documentation] Check Flows in switch.
+ MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${less_flows}
Disconnect Mininet From Cluster
[Documentation] Disconnect Mininet from Cluster.
- Disconnect Cluster Mininet
+ MininetKeywords.Disconnect Cluster Mininet break ${original_owner_list}
Check No Switches After Disconnect
- [Documentation] Check no switches in topology after disconnecting mininet from cluster.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check No Switches In Topology ${SWITCHES}
-
-Remove Flows And Groups While Mininet Is Disconnected
- [Documentation] Remove a group and flow while mininet Is Disconnected from cluster.
- Remove Single Group And Flow
+ [Documentation] Check no switches in topology.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check No Switches On Member ${SWITCHES}
+
+Check Switch Is Not Connected
+ [Documentation] Check switch s1 is not connected to any controller.
+ FOR ${index} IN @{cluster_index_list}
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... OvsManager.Should Be Disconnected
+ ... s1
+ ... ${ODL_SYSTEM_${index}_IP}
+ ... update_data=${True}
+ END
Reconnect Mininet To Cluster
[Documentation] Reconnect mininet to cluster by removing Iptables drop rules that were used to disconnect
- Disconnect Cluster Mininet restore
+ MininetKeywords.Disconnect Cluster Mininet restore
Check Linear Topology After Mininet Reconnects
- [Documentation] Check Linear Topology after reconnect.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Linear Topology ${SWITCHES}
+ [Documentation] Check Linear Topology.
+ BuiltIn.Wait Until Keyword Succeeds 10s 1s ClusterOpenFlow.Check Linear Topology On Member ${SWITCHES}
+
+Add Flows And Groups After Mininet Reconnects
+ [Documentation] Add 1 group type 1&2 and 1 flow in every switch.
+ Add Single Group And Flow On Member
Check Flows In Operational DS After Mininet Reconnects
- [Documentation] Check Flows in Operational DS after mininet is reconnected.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Number Of Flows ${less_flows}
+ [Documentation] Check Flows in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check Number Of Flows On Member ${all_flows}
Check Groups In Operational DS After Mininet Reconnects
- [Documentation] Check Groups in Operational DS after mininet is reconnected to cluster.
- Check Number Of Groups ${less_groups}
+ [Documentation] Check Groups in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${all_groups}
Check Flows In Switch After Mininet Reconnects
- [Documentation] Check Flows in switch after mininet is reconnected to cluster.
- MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${less_flows}
+ [Documentation] Check Flows in switch.
+ MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${all_flows}
-Check Entity Owner Status And Find Owner and Successor Before Stop
+Check Entity Owner Status And Find Owner and Successor Before Owner Stop
[Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
- ${original_owner} ${original_successor_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 1
+ ${original_owner} ${original_successor_list}= ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... 1
${original_successor}= Collections.Get From List ${original_successor_list} 0
BuiltIn.Set Suite Variable ${original_owner}
BuiltIn.Set Suite Variable ${original_successor_list}
BuiltIn.Set Suite Variable ${original_successor}
-Check Shards Status before Stop
+Check Switch Generates Slave Connection Before Owner Stop
+ [Documentation] Check switch s1 is connected to Slave.
+ ${original_slave}= BuiltIn.Set Variable ${ODL_SYSTEM_${original_successor}_IP}
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 50s
+ ... 1s
+ ... OvsManager.Should Be Slave
+ ... s1
+ ... ${original_slave}
+ ... update_data=${True}
+
+Check Shards Status Before Owner Stop
[Documentation] Check Status for all shards in OpenFlow application.
ClusterOpenFlow.Check OpenFlow Shards Status
Check Entity Owner Status And Find Owner and Successor After Stop
[Documentation] Check Entity Owner Status and identify owner and successor.
- ${new_owner} ${new_successor_list} ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device openflow:1 ${original_successor} ${new_cluster_list} after_stop=True
+ ${new_owner} ${new_successor_list}= ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... ${original_successor}
+ ... ${new_cluster_list}
+ ... after_stop=True
BuiltIn.Set Suite Variable ${new_owner}
-Check Linear Topology After Owner Stop
- [Documentation] Check Linear Topology.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Linear Topology ${SWITCHES} ${new_owner}
+Check Stats Are Not Frozen After Owner Stop
+ [Documentation] Check that duration flow stat is increasing in new owner.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Flow Stats Are Not Frozen ${new_owner}
-Add Configuration In Owner and Verify After Fail
- [Documentation] Add Flow in Owner and verify it gets applied from all instances.
- Add Single Group And Flow ${new_owner}
-
-Check Flows In Operational DS After Owner Is Stopped
- [Documentation] Check Flows in Operational DS after Owner is Stopped.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Number Of Flows ${all_flows} ${new_owner}
-
-Check Groups In Operational DS After Owner Is Stopped
- [Documentation] Check Groups in Operational DS after Owner is Stopped.
- Check Number Of Groups ${all_groups} ${new_owner}
-
-Check Flows In Switch After Owner Is Stopped
- [Documentation] Check Flows in switch after Owner is Stopped
- MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${all_flows}
+Remove Configuration In Owner and Verify After Owner Stop
+ [Documentation] Remove 1 group 1&2 and 1 flow in every switch.
+ Remove Single Group And Flow On Member ${new_owner}
+
+Check Flows After Owner Stop In Operational DS
+ [Documentation] Check Flows in Operational DS in new owner.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 30s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Flows On Member
+ ... ${less_flows}
+ ... ${new_owner}
+
+Check Groups After Owner Stop In Operational DS
+ [Documentation] Check Groups in Operational DS in new owner.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${less_groups}
+ ... ${new_owner}
+
+Check Flows In Switch After Owner Stop
+ [Documentation] Check Flows in switch.
+ MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${less_flows}
Start Old Owner Instance
- [Documentation] Start old Owner Instance and verify it is up
+ [Documentation] Start old Owner Instance and verify it is up.
ClusterManagement.Start Single Member ${original_owner}
+Check Entity Owner Status And Find Owner and Successor After Start Owner
+ [Documentation] Check Entity Owner Status and identify owner and successor for first switch s1.
+ ${owner} ${successor_list}= BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device
+ ... openflow:1
+ ... 1
+
Check Linear Topology After Owner Restart
[Documentation] Check Linear Topology.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Linear Topology ${SWITCHES}
+ BuiltIn.Wait Until Keyword Succeeds 10s 1s ClusterOpenFlow.Check Linear Topology On Member ${SWITCHES}
-Remove Configuration In Owner and Verify After Owner Restart
- [Documentation] Add Flow in Owner and verify it gets applied from all instances.
- Remove Single Group And Flow ${new_owner}
-
-Check Flows After Owner Restart In Operational DS
- [Documentation] Check Flows in Operational DS after owner is restarted.
- BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Number Of Flows ${less_flows}
-
-Check Groups After Owner Restart In Operational DS
- [Documentation] Check Groups in Operational DS after owner is restarted.
- Check Number Of Groups ${less_groups}
-
-Check Flows In Switch After Owner Is Restarted
- [Documentation] Check Flows in switch after Owner is restarted
- MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${less_flows}
+Add Configuration In Owner and Verify After Owner Restart
+ [Documentation] Add 1 group type 1&2 and 1 flow in every switch.
+ Add Single Group And Flow On Member ${new_owner}
+
+Check Stats Are Not Frozen After Owner Restart
+ [Documentation] Check that duration flow stat is increasing.
+ Log ${new_owner}
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Flow Stats Are Not Frozen
+
+Check Flows In Operational DS After Owner Restart
+ [Documentation] Check Flows in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check Number Of Flows On Member ${all_flows}
+
+Check Groups In Operational DS After Owner Restart
+ [Documentation] Check Groups in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${all_groups}
+
+Check Flows In Switch After Owner Restart
+ [Documentation] Check Flows in switch.
+ MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${all_flows}
Restart Cluster
[Documentation] Stop and Start cluster.
# Try to stop contoller, if stop does not work or takes too long, kill controller.
${status} ${result}= BuiltIn.Run Keyword And Ignore Error ClusterManagement.Stop_Members_From_List_Or_All
- BuiltIn.Run Keyword If '${status}' != 'PASS' ClusterManagement.Kill_Members_From_List_Or_All
+ IF '${status}' != 'PASS'
+ ClusterManagement.Kill_Members_From_List_Or_All
+ END
ClusterManagement.Start_Members_From_List_Or_All wait_for_sync=False
Check Linear Topology After Controller Restarts
- [Documentation] Check Linear Topology after controller restarts.
- BuiltIn.Wait Until Keyword Succeeds 300s 2s Check Linear Topology ${SWITCHES}
+ [Documentation] Check Linear Topology
+ BuiltIn.Wait Until Keyword Succeeds 300s 2s ClusterOpenFlow.Check Linear Topology On Member ${SWITCHES}
-Add Flow And Group After Restart
- [Documentation] Add 1 group type 1&2 and 1 flow in every switch.
- Add Single Group And Flow
+Check Stats Are Not Frozen After Cluster Restart
+ [Documentation] Check that duration flow stat is increasing.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s Check Flow Stats Are Not Frozen
Check Flows In Operational DS After Controller Restarts
- [Documentation] Check Flows in Operational DS after controller is restarted.
- BuiltIn.Wait Until Keyword Succeeds 300s 2s Check Number Of Flows ${all_flows}
+ [Documentation] Check Flows in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds 30s 1s ClusterOpenFlow.Check Number Of Flows On Member ${all_flows}
Check Groups In Operational DS After Controller Restarts
- [Documentation] Check Groups in Operational DS after controller is restarted.
- Check Number Of Groups ${all_groups}
+ [Documentation] Check Groups in Operational DS.
+ BuiltIn.Wait Until Keyword Succeeds
+ ... 10s
+ ... 1s
+ ... ClusterOpenFlow.Check Number Of Groups On Member
+ ... ${all_groups}
Check Flows In Switch After Controller Restarts
- [Documentation] Check Flows in switch after controller is restarted..
+ [Documentation] Check Flows in switch.
MininetKeywords.Check Flows In Mininet ${mininet_conn_id} ${all_flows}
Stop Mininet
Check No Switches
[Documentation] Check no switches in topology.
- BuiltIn.Wait Until Keyword Succeeds 5s 1s Check No Switches In Topology ${SWITCHES}
+ BuiltIn.Wait Until Keyword Succeeds 5s 1s ClusterOpenFlow.Check No Switches On Member ${SWITCHES}
+
*** Keywords ***
Initialization Phase
[Documentation] Create controller session and set variables.
- ClusterManagement.ClusterManagement_Setup
+ SetupUtils.Setup_Utils_For_Setup_And_Teardown
RequestsLibrary.Create Session session http://${ODL_SYSTEM_IP}:${RESTCONFPORT} auth=${AUTH}
- ${switches} Convert To Integer ${SWITCHES}
- ${iter} Convert To Integer ${ITER}
+ ${switches}= Convert To Integer ${SWITCHES}
+ ${iter}= Convert To Integer ${ITER}
${all_groups}= BuiltIn.Evaluate ${switches} * ${iter} * 2
${less_groups}= BuiltIn.Evaluate ${all_groups} - ${switches} * 2
# Stale flows/groups feature enabled in Boron onwards.
- ${less_groups}= CompareStream.Set Variable If At Least Boron ${less_groups} ${all_groups}
${all_flows}= BuiltIn.Evaluate ${switches} * ${iter+1}
${less_flows}= BuiltIn.Evaluate ${all_flows} - ${switches}
# Stale flows/groups feature enabled in Boron onwards.
- ${less_flows}= CompareStream.Set Variable If At Least Boron ${less_flows} ${all_flows}
BuiltIn.Set Suite Variable ${switches}
BuiltIn.Set Suite Variable ${iter}
BuiltIn.Set Suite Variable ${all_groups}
Final Phase
[Documentation] Delete all sessions.
- ${command} = BuiltIn.Set Variable sudo iptables -v -F
- Utils.Run Command On Controller cmd=${command}
- CompareStream.Run Keyword If At Least Boron TemplatedRequests.Put As Json Templated folder=${VAR_DIR}/frm-config mapping={"STALE":"false"} session=session
- BuiltIn.Run Keyword And Ignore Error RequestsLibrary.Delete Request session ${CONFIG_NODES_API}
+ ${command}= BuiltIn.Set Variable sudo iptables -v -F
+ BuiltIn.Run Keyword And Ignore Error ClusterManagement.Run_Bash_Command_On_List_Or_All ${command}
+ BuiltIn.Run Keyword And Ignore Error RequestsLibrary.Delete Request session ${RFC8040_NODES_API}
RequestsLibrary.Delete All Sessions
-Disconnect Cluster Mininet
- [Arguments] ${action}=break ${member_index_list}=${EMPTY}
- [Documentation] Break and restore controller to mininet connection via iptables.
- ${index_list} = ClusterManagement.List_Indices_Or_All given_list=${member_index_list}
- : FOR ${index} IN @{index_list}
- \ ${rule} = BuiltIn.Set Variable OUTPUT -p all --source ${ODL_SYSTEM_${index}_IP} --destination ${TOOLS_SYSTEM_IP} -j DROP
- \ ${command} = BuiltIn.Set Variable If '${action}'=='restore' sudo /sbin/iptables -D ${rule} sudo /sbin/iptables -I ${rule}
- \ Log To Console ${ODL_SYSTEM_${index}_IP}
- \ Utils.Run Command On Controller ${ODL_SYSTEM_${index}_IP} cmd=${command}
- \ ${command} = BuiltIn.Set Variable sudo /sbin/iptables -L -n
- \ ${output} = Utils.Run Command On Controller cmd=${command}
- \ BuiltIn.Log ${output}
-
-Add Groups And Flows
- [Arguments] ${iter}=1 ${member_index}=1
+Add Groups And Flows On Member
[Documentation] Add ${ITER} groups type 1 & 2 and flows in every switch.
- ${session} = Resolve_Http_Session_For_Member member_index=${member_index}
- : FOR ${switch} IN RANGE 1 ${switches+1}
- \ TemplatedRequests.Post As Json Templated folder=${VAR_DIR}/add-group-1 mapping={"SWITCH":"${switch}"} session=${session} iterations=${iter}
- \ TemplatedRequests.Post As Json Templated folder=${VAR_DIR}/add-group-2 mapping={"SWITCH":"${switch}"} session=${session} iterations=${iter}
- \ TemplatedRequests.Post As Json Templated folder=${VAR_DIR}/add-flow mapping={"SWITCH":"${switch}"} session=${session} iterations=${iter}
-
-Add Single Group And Flow
- [Arguments] ${member_index}=1
+ [Arguments] ${iter}=1 ${member_index}=1
+ ${session}= Resolve_Http_Session_For_Member member_index=${member_index}
+ FOR ${switch} IN RANGE 1 ${switches+1}
+ &{mapping}= BuiltIn.Create_Dictionary NODE=openflow:${switch}
+ TemplatedRequests.Post As Json Templated
+ ... folder=${VAR_DIR}/add-group-1
+ ... mapping=${mapping}
+ ... session=${session}
+ ... iterations=${iter}
+ TemplatedRequests.Post As Json Templated
+ ... folder=${VAR_DIR}/add-group-2
+ ... mapping=${mapping}
+ ... session=${session}
+ ... iterations=${iter}
+ TemplatedRequests.Post As Json Templated
+ ... folder=${VAR_DIR}/add-flow
+ ... mapping=${mapping}
+ ... session=${session}
+ ... iterations=${iter}
+ END
+
+Add Single Group And Flow On Member
[Documentation] Add 1 group 1&2 and 1 flow in every switch.
- Add Groups And Flows 1 ${member_index}
-
-Remove Single Group And Flow
[Arguments] ${member_index}=1
- [Documentation] Remove 1 group 1&2 and 1 flow in every switch.
- ${session} = Resolve_Http_Session_For_Member member_index=${member_index}
- : FOR ${switch} IN RANGE 1 ${switches+1}
- \ RequestsLibrary.Delete Request ${session} ${CONFIG_NODES_API}/node/openflow:${switch}/table/0/flow/1
- \ RequestsLibrary.Delete Request ${session} ${CONFIG_NODES_API}/node/openflow:${switch}/group/1
- \ RequestsLibrary.Delete Request ${session} ${CONFIG_NODES_API}/node/openflow:${switch}/group/1000
+ Add Groups And Flows On Member 1 ${member_index}
-Check Linear Topology
- [Arguments] ${switches} ${member_index}=1
- [Documentation] Check Linear topology.
- ${session} = Resolve_Http_Session_For_Member member_index=${member_index}
- ${resp} RequestsLibrary.Get Request ${session} ${OPERATIONAL_TOPO_API}
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- : FOR ${switch} IN RANGE 1 ${switches+1}
- \ Should Contain ${resp.content} "node-id":"openflow:${switch}"
- \ Should Contain ${resp.content} "tp-id":"openflow:${switch}:1"
- \ Should Contain ${resp.content} "tp-id":"openflow:${switch}:2"
- \ Should Contain ${resp.content} "source-tp":"openflow:${switch}:2"
- \ Should Contain ${resp.content} "dest-tp":"openflow:${switch}:2"
- \ ${edge} Evaluate ${switch}==1 or ${switch}==${switches}
- \ Run Keyword Unless ${edge} Should Contain ${resp.content} "tp-id":"openflow:${switch}:3"
- \ Run Keyword Unless ${edge} Should Contain ${resp.content} "source-tp":"openflow:${switch}:3"
- \ Run Keyword Unless ${edge} Should Contain ${resp.content} "dest-tp":"openflow:${switch}:3
-
-Check No Switches In Inventory
- [Arguments] ${switches} ${member_index}=1
- [Documentation] Check no switch is in inventory
- ${session} = Resolve_Http_Session_For_Member member_index=${member_index}
- ${resp} RequestsLibrary.Get Request ${session} ${OPERATIONAL_NODES_API}
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- : FOR ${switch} IN RANGE 1 ${switches+1}
- \ Should Not Contain ${resp.content} "openflow:${switch}"
-
-Check Number Of Flows
- [Arguments] ${flows} ${member_index}=1
- [Documentation] Check number of flows in the inventory.
- ${session} = Resolve_Http_Session_For_Member member_index=${member_index}
- ${resp}= RequestsLibrary.Get Request ${session} ${OPERATIONAL_NODES_API}
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- ${count}= Get Count ${resp.content} "priority"
- Should Be Equal As Integers ${count} ${flows}
-
-Check Number Of Groups
- [Arguments] ${groups} ${member_index}=1
- [Documentation] Check number of groups in the inventory.
- ${session} = Resolve_Http_Session_For_Member member_index=${member_index}
- ${resp}= RequestsLibrary.Get Request ${session} ${OPERATIONAL_NODES_API}
- Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- ${group_count}= Get Count ${resp.content} "group-type"
- ${count}= CompareStream.Set_Variable_If_At Least_Boron ${group_count} ${group_count/2}
- Should Be Equal As Integers ${count} ${groups}
-
-Check No Switches In Topology
- [Arguments] ${switches} ${member_index}=1
- [Documentation] Check no switch is in topology
- ${session} = Resolve_Http_Session_For_Member member_index=${member_index}
- ${resp} RequestsLibrary.Get Request ${session} ${OPERATIONAL_TOPO_API}
+Remove Single Group And Flow On Member
+ [Documentation] Remove 1 group 1&2 and 1 flow in every switch.
+ [Arguments] ${member_index}=1
+ ${session}= Resolve_Http_Session_For_Member member_index=${member_index}
+ FOR ${switch} IN RANGE 1 ${switches+1}
+ RequestsLibrary.Delete Request
+ ... ${session}
+ ... ${RFC8040_NODES_API}/node=openflow%3A${switch}/flow-node-inventory:table=0/flow=1
+ RequestsLibrary.Delete Request
+ ... ${session}
+ ... ${RFC8040_NODES_API}/node=openflow%3A${switch}/flow-node-inventory:group=1
+ RequestsLibrary.Delete Request
+ ... ${session}
+ ... ${RFC8040_NODES_API}/node=openflow%3A${switch}/flow-node-inventory:group=1000
+ END
+
+Check Flow Stats Are Not Frozen
+ [Documentation] Verify flow stats are not frozen for flow 1 and switch 1.
+ [Arguments] ${member_index}=1 ${period_in_seconds}=5
+ ${duration_1}= Extract Flow Duration ${member_index}
+ ${duration_1}= Builtin.Convert To Integer ${duration_1}
+ BuiltIn.Sleep ${period_in_seconds}
+ ${duration_2}= Extract Flow Duration ${member_index}
+ ${duration_2}= Builtin.Convert To Integer ${duration_2}
+ Should Not Be Equal As Integers ${duration_1} ${duration_2}
+
+Extract Flow Duration
+ [Documentation] Extract duration for flow 1 in switch 1.
+ [Arguments] ${member_index}
+ ${session}= Resolve_Http_Session_For_Member member_index=${member_index}
+ ${resp}= RequestsLibrary.Get Request
+ ... ${session}
+ ... ${RFC8040_NODES_API}/node=openflow%3A1/flow-node-inventory:table=0/flow=1?content=nonconfig
+ ... headers=${headers}
Log ${resp.content}
- Should Be Equal As Strings ${resp.status_code} 200
- : FOR ${switch} IN RANGE 1 ${switches+1}
- \ Should Not Contain ${resp.content} openflow:${switch}
+ ${json_resp}= RequestsLibrary.To_Json ${resp.content}
+ ${flow_list}= Collections.Get_From_Dictionary ${json_resp} flow-node-inventory:flow
+ ${flow_stats}= Collections.Get_From_Dictionary
+ ... ${flow_list}[0]
+ ... opendaylight-flow-statistics:flow-statistics
+ ${duration}= Collections.Get_From_Dictionary ${flow_stats}[duration] second
+ RETURN ${duration}