From d883b8eb7f8831ced0ea7289045c7074fbd4ba43 Mon Sep 17 00:00:00 2001 From: Luis Gomez Date: Sun, 28 Feb 2016 21:45:30 -0800 Subject: [PATCH] Fix and clean OVSDB cluster suite - Simplify OVSDB cluster keywords. - Modify Entity Owner KW to support uuid. - Restore OVSDB clean environment. Change-Id: I2c3034c5d09a4fce4e0b96f40bced30fff3204dd Signed-off-by: Luis Gomez --- csit/libraries/ClusterOvsdb.robot | 230 ++++++----------- csit/libraries/OVSDB.robot | 42 ++-- .../010__Ovsdb_Southbound_Cluster.robot | 236 ++++++++---------- .../020__Southbound_Cluster_Extension.robot | 174 +++++++++---- 4 files changed, 317 insertions(+), 365 deletions(-) diff --git a/csit/libraries/ClusterOvsdb.robot b/csit/libraries/ClusterOvsdb.robot index 3d516c464b..a685e20787 100644 --- a/csit/libraries/ClusterOvsdb.robot +++ b/csit/libraries/ClusterOvsdb.robot @@ -7,13 +7,17 @@ Resource Utils.robot Resource OVSDB.robot Variables ../variables/Variables.py +*** Variables *** +${BRIDGE} br01 +${OVSDB_CONFIG_DIR} ${CURDIR}/../variables/ovsdb + *** Keywords *** Check Ovsdb Shards Status [Arguments] ${controller_index_list} [Documentation] Check Status for all shards in Ovsdb application. - ${topo_conf_leader} ${topo_conf_followers_list} Get Cluster Shard Status ${controller_index_list} config topology - ${topo_oper_leader} ${topo_oper_followers_list} Get Cluster Shard Status ${controller_index_list} operational topology - ${owner_oper_leader} ${owner_oper_followers_list} Get Cluster Shard Status ${controller_index_list} operational entity-ownership + ${topo_conf_leader} ${topo_conf_followers_list} ClusterKeywords.Get Cluster Shard Status ${controller_index_list} config topology + ${topo_oper_leader} ${topo_oper_followers_list} ClusterKeywords.Get Cluster Shard Status ${controller_index_list} operational topology + ${owner_oper_leader} ${owner_oper_followers_list} ClusterKeywords.Get Cluster Shard Status ${controller_index_list} operational entity-ownership Log config topology Leader is ${topo_conf_leader} and followers are ${topo_conf_followers_list} Log operational topology Leader is ${topo_oper_leader} and followers are ${topo_oper_followers_list} Log operational entity-ownership Leader is ${owner_oper_leader} and followers are ${owner_oper_followers_list} @@ -23,24 +27,15 @@ Check Ovsdb Shards Status After Cluster Event [Documentation] Check Shard Status after some cluster event. Wait Until Keyword Succeeds 90s 1s Check Ovsdb Shards Status ${controller_index_list} -Get Ovsdb Entity Owner Status For One Device - [Arguments] ${controller_index_list} - [Documentation] Check Entity Owner Status and identify owner and candidate. - ${owner} ${candidates_list} Wait Until Keyword Succeeds 20s 1s Get Cluster Entity Owner For Ovsdb ${controller_index_list} - ... ovsdb ovsdb:1 - [Return] ${owner} ${candidates_list} - Get Cluster Entity Owner For Ovsdb [Arguments] ${controller_index_list} ${device_type} ${device} [Documentation] Checks Entity Owner status for a ${device} and returns owner index and list of candidates from a ${controller_index_list}. ... ${device_type} is openflow, ovsdb, etc... ${length}= Get Length ${controller_index_list} ${candidates_list}= Create List - ${data}= Get Data From URI controller@{controller_index_list}[0] /restconf/operational/entity-owners:entity-owners + ${data}= Utils.Get Data From URI controller@{controller_index_list}[0] /restconf/operational/entity-owners:entity-owners Log ${data} - ${data}= Replace String ${data} /network-topology:network-topology/network-topology:topology[network-topology:topology-id=' ${EMPTY} - # the UUID will not always be the same so need to use regexp to remove this string - ${data}= Replace String Using Regexp ${data} \/network-topology:node\\[network-topology:node-id='ovsdb://uuid/........-....-....-....-............ ${EMPTY} + ${data}= Replace String ${data} /network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id=' ${EMPTY} Log ${data} ${clear_data}= Replace String ${data} '] ${EMPTY} Log ${clear_data} @@ -66,174 +61,99 @@ Get Cluster Entity Owner For Ovsdb Remove Values From List ${candidates_list} ${owner} [Return] ${owner} ${candidates_list} -Create Bridge And Verify - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] Create bridge in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - # need to get UUID which should be the same on all controllers in cluster, so asking controller1 - ${ovsdb_uuid}= Get OVSDB UUID controller_http_session=controller${controller_index} - Set Suite Variable ${ovsdb_uuid} +Get Ovsdb Entity Owner Status For One Device + [Arguments] ${controller_index_list} ${device} + [Documentation] Check Entity Owner Status and identify owner and candidate for an ovs device ${device}. + ${owner} ${candidates_list} Wait Until Keyword Succeeds 20s 1s Get Cluster Entity Owner For Ovsdb ${controller_index_list} + ... ovsdb ${device} + [Return] ${owner} ${candidates_list} + +Create Sample Bridge Manually And Verify + [Arguments] ${controller_index_list} + [Documentation] Create bridge br-s1 using OVS command and verify it gets created in all instances in ${controller_index_list}. + Utils.Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl add-br br-s1 + ${dictionary_operational}= Create Dictionary br-s1=5 + ${dictionary_config}= Create Dictionary br-s1=0 + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_config} ${CONFIG_TOPO_API} + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_operational} ${OPERATIONAL_TOPO_API} + +Add Sample Port To The Manual Bridge And Verify + [Arguments] ${controller_index_list} + [Documentation] Add Port vx1 to br-s1 using OVS command and verify it gets added in all instances in ${controller_index_list}. + Utils.Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl add-port br-s1 vx1 -- set Interface vx1 type=vxlan + ${dictionary_operational}= Create Dictionary vx1=2 + ${dictionary_config}= Create Dictionary vx1=0 + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_config} ${CONFIG_TOPO_API} + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_operational} ${OPERATIONAL_TOPO_API} + +Delete Sample Bridge Manually And Verify + [Arguments] ${controller_index_list} + [Documentation] Delete bridge br-s1 using OVS command and verify it gets applied in all instances in ${controller_index_list}. + Utils.Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl del-br br-s1 + ${dictionary}= Create Dictionary br-s1=0 + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} + +Create Sample Bridge And Verify + [Arguments] ${controller_index_list} ${controller_index} + [Documentation] Create bridge ${BRIDGE} in controller ${controller_index} and verify it gets created in all instances in ${controller_index_list}. ${body}= OperatingSystem.Get File ${CURDIR}/../variables/ovsdb/create_bridge_3node.json ${body} Replace String ${body} ovsdb://127.0.0.1:61644 ovsdb://uuid/${ovsdb_uuid} - ${body} Replace String ${body} tcp:controller1:6633 tcp:${ODL_SYSTEM_1_IP}:6640 - ${body} Replace String ${body} tcp:controller2:6633 tcp:${ODL_SYSTEM_2_IP}:6640 - ${body} Replace String ${body} tcp:controller3:6633 tcp:${ODL_SYSTEM_3_IP}:6640 + ${body} Replace String ${body} tcp:controller1:6633 tcp:${ODL_SYSTEM_1_IP}:6633 + ${body} Replace String ${body} tcp:controller2:6633 tcp:${ODL_SYSTEM_2_IP}:6633 + ${body} Replace String ${body} tcp:controller3:6633 tcp:${ODL_SYSTEM_3_IP}:6633 ${body} Replace String ${body} 127.0.0.1 ${TOOLS_SYSTEM_IP} - ${BRIDGE}= Set Variable If "${status}"=="AfterFail" br02 br01 - Log ${BRIDGE} ${body} Replace String ${body} br01 ${BRIDGE} ${body} Replace String ${body} 61644 ${OVSDB_PORT} Log ${body} ${TOOLS_SYSTEM_IP1} Replace String ${TOOLS_SYSTEM_IP} ${TOOLS_SYSTEM_IP} "${TOOLS_SYSTEM_IP}" ${dictionary}= Create Dictionary ${TOOLS_SYSTEM_IP1}=1 ${OVSDBPORT}=4 ${BRIDGE}=1 - Wait Until Keyword Succeeds 20s 1s Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE} + Wait Until Keyword Succeeds 20s 1s ClusterKeywords.Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE} ... ${body} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid} + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid} -Create Bridge Manually And Verify +Create Sample Port And Verify [Arguments] ${controller_index_list} ${controller_index} - [Documentation] Create bridge in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl add-br br-s1 - ${dictionary_operational}= Create Dictionary br-s1=5 - ${dictionary_config}= Create Dictionary br-s1=0 - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_config} ${CONFIG_TOPO_API} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_operational} ${OPERATIONAL_TOPO_API} - -Delete Bridge Manually And Verify - [Arguments] ${controller_index_list} ${controller_index} - [Documentation] Delete bridge in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl del-br br-s1 - ${dictionary}= Create Dictionary br-s1=0 - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} - -Delete Bridge Via Rest Call And Verify - [Arguments] ${controller_index_list} ${controller_index} - [Documentation] Delete bridge in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - # need to get UUID which should be the same on all controllers in cluster, so asking controller1 - ${ovsdb_uuid}= Get OVSDB UUID controller_http_session=controller${controller_index} - ${dictionary}= Create Dictionary ${BRIDGE}=0 - Wait Until Keyword Succeeds 20s 1s Delete And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid} - -Create Port Via Controller - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] This will add port/interface to the config datastore + [Documentation] Add Port vx2 to bridge ${BRIDGE} in controller ${controller_index} and verify it gets added in all instances in ${controller_index_list}. ${sample} OperatingSystem.Get File ${OVSDB_CONFIG_DIR}/create_port_3node.json ${body} Replace String ${sample} 192.168.1.10 ${TOOLS_SYSTEM_IP} Log ${body} - ${BRIDGE}= Set Variable If "${status}"=="AfterFail" br02 br01 - Log ${BRIDGE} Log URL is ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ ${port_dictionary}= Create Dictionary ${BRIDGE}=7 vx2=3 - Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ ${body} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${port_dictionary} ${OPERATIONAL_TOPO_API} + ClusterKeywords.Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ ${body} + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${port_dictionary} ${OPERATIONAL_TOPO_API} -Modify the destination IP of Port - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] This will modify the dst ip of existing port +Modify the destination IP of Sample Port + [Arguments] ${controller_index_list} ${controller_index} + [Documentation] Modify the dst ip of port vx2 in bridge ${BRIDGE} in controller ${controller_index}. ${sample} OperatingSystem.Get File ${OVSDB_CONFIG_DIR}/create_port_3node.json ${body} Replace String ${sample} 192.168.1.10 10.0.0.19 - ${BRIDGE}= Set Variable If "${status}"=="AfterFail" br02 br01 Log URL is ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ Log ${body} - Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ ${body} + ClusterKeywords.Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ ${body} -Delete Port And Verify - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] Delete port in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - ${dictionary}= Create Dictionary vx2=0 - ${BRIDGE}= Set Variable If "${status}"=="AfterFail" br02 br01 - Delete And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} +Verify Sample Port Is Modified + [Arguments] ${controller_index_list} + [Documentation] Verify dst ip of port vx2 in bridge ${BRIDGE} gets modified in all instances in ${controller_index_list}. + ${port_dictionary} Create Dictionary br01=7 vx2=3 10.0.0.19=1 + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${port_dictionary} ${OPERATIONAL_TOPO_API} -Add Port To The Manual Bridge And Verify +Delete Sample Port And Verify [Arguments] ${controller_index_list} ${controller_index} - [Documentation] Add Port in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl add-port br-s1 vx1 -- set Interface vx1 type=vxlan - ${dictionary_operational}= Create Dictionary vx1=2 - ${dictionary_config}= Create Dictionary vx1=0 - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_config} ${CONFIG_TOPO_API} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary_operational} ${OPERATIONAL_TOPO_API} - -Get Operational Topology with modified Port - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] This request will fetch the operational topology after the Port is added to the bridge - ${port_dictionary_before_fail} Create Dictionary br01=7 vx2=3 10.0.0.19=1 - ${port_dictionary_after_fail} Create Dictionary br02=7 vx2=3 10.0.0.19=1 - ${port_dictionary}= Set Variable If "${status}"=="AfterFail" ${port_dictionary_after_fail} ${port_dictionary_before_fail} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${port_dictionary} ${OPERATIONAL_TOPO_API} - -Verify Bridge in Restarted Node - [Arguments] ${controller_index_list} ${status}=${NONE} - [Documentation] Verify Bridge in Restarted node, which is created when the node is down. - ${BRIDGE}= Set Variable If "${status}"=="AfterCandidateRecover" br04 br02 - ${dictionary} Create Dictionary ${BRIDGE}=6 - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} - -Verify Port in Restarted Node - [Arguments] ${controller_index_list} ${status}=${NONE} - [Documentation] Verify Port in Restarted node, which is created when the node is down. - ${BRIDGE}= Set Variable If "${status}"=="AfterCandidateRecover" br04 br02 - ${dictionary} Create Dictionary ${BRIDGE}=6 vx2=3 - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} - -Create Bridge In Candidate - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] Create bridge in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - # need to get UUID which should be the same on all controllers in cluster, so asking controller1 - ${ovsdb_uuid}= Get OVSDB UUID controller_http_session=controller${controller_index} - Set Suite Variable ${ovsdb_uuid} - ${body}= OperatingSystem.Get File ${CURDIR}/../variables/ovsdb/create_bridge_3node.json - ${body} Replace String ${body} ovsdb://127.0.0.1:61644 ovsdb://uuid/${ovsdb_uuid} - ${body} Replace String ${body} tcp:controller1:6633 tcp:${ODL_SYSTEM_1_IP}:6640 - ${body} Replace String ${body} tcp:controller2:6633 tcp:${ODL_SYSTEM_2_IP}:6640 - ${body} Replace String ${body} tcp:controller3:6633 tcp:${ODL_SYSTEM_3_IP}:6640 - ${body} Replace String ${body} 127.0.0.1 ${MININET} - ${BRIDGE}= Set Variable If "${status}"=="AfterCandidateFail" br04 br03 - Log ${BRIDGE} - ${body} Replace String ${body} br01 ${BRIDGE} - ${body} Replace String ${body} 61644 ${OVSDB_PORT} - Log ${body} - ${TOOLS_SYSTEM_IP1} Replace String ${TOOLS_SYSTEM_IP} ${TOOLS_SYSTEM_IP} "${TOOLS_SYSTEM_IP}" - ${dictionary}= Create Dictionary ${TOOLS_SYSTEM_IP1}=1 ${OVSDBPORT}=4 ${BRIDGE}=1 - Wait Until Keyword Succeeds 20s 1s Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE} - ... ${body} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid} - -Create Port Vxlan in Candidate - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] This will add port/interface to the config datastore - ${sample} OperatingSystem.Get File ${OVSDB_CONFIG_DIR}/create_port_3node.json - ${body} Replace String ${sample} 192.168.1.10 ${TOOLS_SYSTEM_IP} - Log ${body} - ${BRIDGE}= Set Variable If "${status}"=="AfterCandidateFail" br04 br03 - Log ${BRIDGE} - Log URL is ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ - ${port_dictionary}= Create Dictionary ${BRIDGE}=7 vx2=3 - Put And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ ${body} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${port_dictionary} ${OPERATIONAL_TOPO_API} + [Documentation] Delete port vx2 from bridge ${BRIDGE} in controller ${controller_index} and verify it gets deleted in all instances in ${controller_index_list}. + ${dictionary}= Create Dictionary vx2=0 + ClusterKeywords.Delete And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} -Delete Bridge in Candidate - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] Delete bridge in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - # need to get UUID which should be the same on all controllers in cluster, so asking controller1 - ${ovsdb_uuid}= Get OVSDB UUID controller_http_session=controller${controller_index} - ${BRIDGE}= Set Variable If "${status}"=="AfterCandidateFail" br04 br03 +Delete Sample Bridge And Verify + [Arguments] ${controller_index_list} ${controller_index} + [Documentation] Delete bridge ${BRIDGE} in ${controller_index} and verify it gets deleted in all instances in ${controller_index_list}. ${dictionary}= Create Dictionary ${BRIDGE}=0 - Wait Until Keyword Succeeds 20s 1s Delete And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid} - -Delete Port In Candidate - [Arguments] ${controller_index_list} ${controller_index} ${status}=${NONE} - [Documentation] Delete port in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. - ${dictionary}= Create Dictionary vx2=0 - ${BRIDGE}= Set Variable If "${status}"=="AfterCandidateFail" br04 br03 - Delete And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point/vx2/ - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} + Wait Until Keyword Succeeds 20s 1s ClusterKeywords.Delete And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE} + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid} Configure Exit OVSDB Connection - [Arguments] ${controller_index_list} ${controller_index} + [Arguments] ${controller_index_list} [Documentation] Cleans up test environment, close existing sessions. - Clean OVSDB Test Environment ${TOOLS_SYSTEM_IP} + OVSDB.Clean OVSDB Test Environment ${TOOLS_SYSTEM_IP} ${dictionary}= Create Dictionary ovsdb://uuid=0 - Delete And Check At URI In Cluster ${controller_index_list} ${controller_index} ${CONFIG_TOPO_API}/topology/ovsdb:1/node/ovsdb:%2F%2Fuuid%2F${ovsdb_uuid} - Wait Until Keyword Succeeds 5s 1s Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} + Wait Until Keyword Succeeds 5s 1s ClusterKeywords.Check Item Occurrence At URI In Cluster ${controller_index_list} ${dictionary} ${OPERATIONAL_TOPO_API} diff --git a/csit/libraries/OVSDB.robot b/csit/libraries/OVSDB.robot index 575d666cad..58cd7ba75a 100644 --- a/csit/libraries/OVSDB.robot +++ b/csit/libraries/OVSDB.robot @@ -64,7 +64,7 @@ Add Vxlan To Bridge Verify OVS Reports Connected [Arguments] ${tools_system}=${TOOLS_SYSTEM_IP} [Documentation] Uses "vsctl show" to check for string "is_connected" - ${output}= Run Command On Remote System ${tools_system} sudo ovs-vsctl show + ${output}= Utils.Run Command On Remote System ${tools_system} sudo ovs-vsctl show Should Contain ${output} is_connected Get OVSDB UUID @@ -81,11 +81,11 @@ Get OVSDB UUID ${topology}= Get From List ${topologies} 0 ${node_list}= Get From Dictionary ${topology} node Log ${node_list} + # Since bridges are also listed as nodes, but will not have the extra "ovsdb:connection-info data, + # we need to use "Run Keyword And Ignore Error" below. : FOR ${node} IN @{node_list} \ ${node_id}= Get From Dictionary ${node} node-id \ ${node_uuid}= Replace String ${node_id} ovsdb://uuid/ ${EMPTY} - \ # Since bridges are also listed as nodes, but will not have the extra "ovsdb:connection-info data, we need to - \ # use "Run Keyword And Ignore Error" below. \ ${status} ${connection_info} Run Keyword And Ignore Error Get From Dictionary ${node} ovsdb:connection-info \ ${status} ${remote_ip} Run Keyword And Ignore Error Get From Dictionary ${connection_info} remote-ip \ ${uuid}= Set Variable If '${remote_ip}' == '${ovs_system_ip}' ${node_uuid} ${uuid} @@ -94,9 +94,9 @@ Get OVSDB UUID Collect OVSDB Debugs [Arguments] ${switch}=br-int [Documentation] Used to log useful test debugs for OVSDB related system tests. - ${output}= Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl show + ${output}= Utils.Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-vsctl show Log ${output} - ${output}= Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-ofctl -O OpenFlow13 dump-flows ${switch} | cut -d',' -f3- + ${output}= Utils.Run Command On Mininet ${TOOLS_SYSTEM_IP} sudo ovs-ofctl -O OpenFlow13 dump-flows ${switch} | cut -d',' -f3- Log ${output} Clean OVSDB Test Environment @@ -104,33 +104,31 @@ Clean OVSDB Test Environment [Documentation] General Use Keyword attempting to sanitize test environment for OVSDB related ... tests. Not every step will always be neccessary, but should not cause any problems for ... any new ovsdb test suites. - Clean Mininet System ${tools_system} - Run Command On Mininet ${tools_system} sudo ovs-vsctl del-manager - Run Command On Mininet ${tools_system} sudo /usr/share/openvswitch/scripts/ovs-ctl stop - Run Command On Mininet ${tools_system} sudo rm -rf /etc/openvswitch/conf.db - Run Command On Mininet ${tools_system} sudo /usr/share/openvswitch/scripts/ovs-ctl start + Utils.Clean Mininet System ${tools_system} + Utils.Run Command On Mininet ${tools_system} sudo ovs-vsctl del-manager + Utils.Run Command On Mininet ${tools_system} sudo /usr/share/openvswitch/scripts/ovs-ctl stop + Utils.Run Command On Mininet ${tools_system} sudo rm -rf /etc/openvswitch/conf.db + Utils.Run Command On Mininet ${tools_system} sudo /usr/share/openvswitch/scripts/ovs-ctl start Set Controller In OVS Bridge - [Arguments] ${mininet} ${bridge} ${controller_opt} + [Arguments] ${tools_system} ${bridge} ${controller_opt} [Documentation] Sets controller for a given OVS ${bridge} using controller options in ${controller_opt} - Run Command On Mininet ${mininet} sudo ovs-vsctl del-controller ${bridge} - Run Command On Mininet ${mininet} sudo ovs-vsctl set-controller ${bridge} ${controller_opt} + Utils.Run Command On Mininet ${tools_system} sudo ovs-vsctl del-controller ${bridge} + Utils.Run Command On Mininet ${tools_system} sudo ovs-vsctl set-controller ${bridge} ${controller_opt} Add Multiple Managers to OVS - [Arguments] ${mininet} ${controller_index_list} ${ovs_mgr_port}=6640 - [Documentation] Start Mininet with custom topology and connect to all controllers in the ${controller_index_list}. + [Arguments] ${tools_system} ${controller_index_list} ${ovs_mgr_port}=6640 + [Documentation] Connect OVS to all controllers in the ${controller_index_list}. Log Clear any existing mininet - Clean Mininet System ${mininet} - ${mininet_conn_id}= Open Connection ${mininet} prompt=${TOOLS_SYSTEM_PROMPT} timeout=${DEFAULT_TIMEOUT} - Set Suite Variable ${mininet_conn_id} - Flexible Mininet Login + Utils.Clean Mininet System ${tools_system} ${ovs_opt}= Set Variable : FOR ${index} IN @{controller_index_list} \ ${ovs_opt}= Catenate ${ovs_opt} ${SPACE}tcp:${ODL_SYSTEM_${index}_IP}:${ovs_mgr_port} \ Log ${ovs_opt} Log Configure OVS Managers in the OVS - Run Command On Mininet ${mininet} sudo ovs-vsctl set-manager ${ovs_opt} + Utils.Run Command On Mininet ${tools_system} sudo ovs-vsctl set-manager ${ovs_opt} Log Check OVS configuratiom - ${output}= Run Command On Mininet ${mininet} sudo ovs-vsctl show + ${output}= Utils.Run Command On Mininet ${tools_system} sudo ovs-vsctl show Log ${output} - [Return] ${mininet_conn_id} + ${ovsdb_uuid}= Get OVSDB UUID controller_http_session=controller1 + [Return] ${ovsdb_uuid} diff --git a/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot b/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot index fc36c395dd..80ff923f25 100644 --- a/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot +++ b/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot @@ -1,221 +1,181 @@ *** Settings *** -Documentation Test suite for Ovsdb Southbound Cluster +Documentation Test suite for Ovsdb Southbound Cluster - Owner failover and recover Suite Setup Create Controller Sessions Suite Teardown Delete All Sessions Library RequestsLibrary Resource ../../../libraries/ClusterOvsdb.robot Resource ../../../libraries/ClusterKeywords.robot -Resource ../../../libraries/MininetKeywords.robot Variables ../../../variables/Variables.py -*** Variables *** -${OVSDB_CONFIG_DIR} ${CURDIR}/../../../variables/ovsdb -${BRIDGE} br01 - *** Test Cases *** Create Original Cluster List [Documentation] Create original cluster list. - ${original_cluster_list} Create Controller Index List + ${original_cluster_list} ClusterKeywords.Create Controller Index List Set Suite Variable ${original_cluster_list} Log ${original_cluster_list} Check Shards Status Before Fail [Documentation] Check Status for all shards in Ovsdb application. - Check Ovsdb Shards Status ${original_cluster_list} + ClusterOvsdb.Check Ovsdb Shards Status ${original_cluster_list} Start OVS Multiple Connections - [Documentation] Start mininet with connection to all cluster instances. - ${mininet_conn_id} Add Multiple Managers to OVS ${TOOLS_SYSTEM_IP} ${original_cluster_list} - Set Suite Variable ${mininet_conn_id} + [Documentation] Connect OVS to all cluster instances. + ${ovsdb_uuid} Ovsdb.Add Multiple Managers to OVS ${TOOLS_SYSTEM_IP} ${original_cluster_list} + Set Suite Variable ${ovsdb_uuid} Check Entity Owner Status And Find Owner and Candidate Before Fail [Documentation] Check Entity Owner Status and identify owner and candidate. - ${original_owner} ${original_candidates_list} Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} + ${original_owner} ${original_candidates_list} ClusterOvsdb.Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} ovsdb://uuid/${ovsdb_uuid} ${original_candidate}= Get From List ${original_candidates_list} 0 Set Suite Variable ${original_owner} Set Suite Variable ${original_candidate} -Create Bridge Manually In Owner and Verify Before Fail - [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge Manually And Verify ${original_cluster_list} ${original_owner} +Create Bridge Manually and Verify Before Fail + [Documentation] Create bridge with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Bridge Manually And Verify ${original_cluster_list} + +Add Port Manually and Verify Before Fail + [Documentation] Add port with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Add Sample Port To The Manual Bridge And Verify ${original_cluster_list} -Add Port Manually In Owner and Verify Before Fail - [Documentation] Add Port in Owner and verify it gets applied from all instances. - Add Port To The Manual Bridge And Verify ${original_cluster_list} ${original_owner} +Delete the Bridge Manually and Verify Before Fail + [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge Manually And Verify ${original_cluster_list} -Create Bridge Via Controller In Owner and Verify Before Fail +Create Bridge In Owner and Verify Before Fail [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge And Verify ${original_cluster_list} ${original_owner} + ClusterOvsdb.Create Sample Bridge And Verify ${original_cluster_list} ${original_owner} -Create Port Via Controller In Owner and Verify Before Fail +Create Port In Owner and Verify Before Fail [Documentation] Create Port in Owner and verify it gets applied from all instances. - Create Port Via Controller ${original_cluster_list} ${original_owner} + ClusterOvsdb.Create Sample Port And Verify ${original_cluster_list} ${original_owner} -Modify the destination IP of Port Before Fail - [Documentation] This will modify the dst ip of existing port - Modify the destination IP of Port ${original_cluster_list} ${original_owner} +Modify the destination IP of Port In Owner Before Fail + [Documentation] Modify the dst ip of existing port in Owner. + ClusterOvsdb.Modify the destination IP of Sample Port ${original_cluster_list} ${original_owner} -Get Operational Topology with modified Port Before Fail - [Documentation] This request will fetch the operational topology after the modified port is added to the bridge - Get Operational Topology with modified Port ${original_cluster_list} ${original_owner} +Verify Port Is Modified Before Fail + [Documentation] Verify port is modified in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${original_cluster_list} -Delete the Port Before Fail - [Documentation] This request will delete the port node from the bridge node and data store. - Delete Port And Verify ${original_cluster_list} ${original_owner} +Delete Port In Owner Before Fail + [Documentation] Delete port in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Port And Verify ${original_cluster_list} ${original_owner} -Delete the Bridge In Owner and Verify Before Fail - [Documentation] This request will delete the bridge node from the operational data store. - Delete Bridge Manually And Verify ${original_cluster_list} ${original_owner} - -Delete Bridge Via Rest Call And Verify In Owner Before Fail - [Documentation] This request will delete the bridge node from the config data store and operational data store. - Delete Bridge Via Rest Call And Verify ${original_cluster_list} ${original_owner} +Delete Bridge In Owner And Verify Before Fail + [Documentation] Delete bridge in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge And Verify ${original_cluster_list} ${original_owner} Kill Owner Instance [Documentation] Kill Owner Instance and verify it is dead - Kill Multiple Controllers ${original_owner} - ${new_cluster_list} Create Controller Index List + ClusterKeywords.Kill Multiple Controllers ${original_owner} + ${new_cluster_list} ClusterKeywords.Create Controller Index List Remove Values From List ${new_cluster_list} ${original_owner} Set Suite Variable ${new_cluster_list} Check Shards Status After Fail [Documentation] Create original cluster list and check Status for all shards in Ovsdb application. - Check Ovsdb Shards Status After Cluster Event ${new_cluster_list} + ClusterOvsdb.Check Ovsdb Shards Status After Cluster Event ${new_cluster_list} Check Entity Owner Status And Find Owner and Candidate After Fail [Documentation] Check Entity Owner Status and identify owner and candidate. - ${new_owner} ${new_candidates_list} Get Ovsdb Entity Owner Status For One Device ${new_cluster_list} + ${new_owner} ${new_candidates_list} ClusterOvsdb.Get Ovsdb Entity Owner Status For One Device ${new_cluster_list} ovsdb://uuid/${ovsdb_uuid} Run Keyword And Continue On Failure List Should Not Contain Value ${new_candidates_list} ${original_owner} Original owner ${original_owner} still in candidate list. Remove Values From List ${new_candidates_list} ${original_owner} ${new_candidate}= Get From List ${new_candidates_list} 0 Set Suite Variable ${new_owner} Set Suite Variable ${new_candidate} -Create Bridge Manually In Owner and Verify After Fail - [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge Manually And Verify ${new_cluster_list} ${new_owner} +Create Bridge Manually and Verify After Fail + [Documentation] Create bridge with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Bridge Manually And Verify ${new_cluster_list} -Add Port Manually In Owner and Verify After Fail - [Documentation] Add Port in Owner and verify it gets applied from all instances. - Add Port To The Manual Bridge And Verify ${new_cluster_list} ${new_owner} +Add Port Manually and Verify After Fail + [Documentation] Add port with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Add Sample Port To The Manual Bridge And Verify ${new_cluster_list} -Create Bridge Via Controller In Owner and Verify After Fail +Delete the Bridge Manually and Verify After Fail + [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge Manually And Verify ${new_cluster_list} + +Create Bridge In Owner and Verify After Fail [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge And Verify ${new_cluster_list} ${new_owner} AfterFail + ClusterOvsdb.Create Sample Bridge And Verify ${new_cluster_list} ${new_owner} -Create Port Via Controller In Owner and Verify After Fail +Create Port In Owner and Verify After Fail [Documentation] Create Port in Owner and verify it gets applied from all instances. - Create Port Via Controller ${new_cluster_list} ${new_owner} AfterFail - -Modify the destination IP of Port After Fail - [Documentation] This will modify the dst ip of existing port - Modify the destination IP of Port ${new_cluster_list} ${new_owner} AfterFail - -Get Operational Topology with modified Port After Fail - [Documentation] This request will fetch the operational topology after the modified port is added to the bridge - Get Operational Topology with modified Port ${new_cluster_list} ${new_owner} AfterFail + ClusterOvsdb.Create Sample Port And Verify ${new_cluster_list} ${new_owner} -Delete the Port After Fail - [Documentation] This request will delete the port node from the bridge node and data store. - Delete Port And Verify ${new_cluster_list} ${new_owner} AfterFail +Modify the destination IP of Port In Owner After Fail + [Documentation] Modify the dst ip of existing port in Owner. + ClusterOvsdb.Modify the destination IP of Sample Port ${new_cluster_list} ${new_owner} -Delete the Bridge In Owner and Verify After Fail - [Documentation] This request will delete the bridge node from the config data store and operational data store. - Delete Bridge Manually And Verify ${new_cluster_list} ${new_owner} - -Delete Bridge Via Rest Call And Verify In Owner After Fail - [Documentation] This request will delete the bridge node from the config data store and operational data store. - Delete Bridge Via Rest Call And Verify ${new_cluster_list} ${new_owner} +Verify Port Is Modified After Fail + [Documentation] Verify port is modified in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${new_cluster_list} Start Old Owner Instance [Documentation] Start Owner Instance and verify it is active - Start Multiple Controllers 300s ${original_owner} + ClusterKeywords.Start Multiple Controllers 300s ${original_owner} Check Shards Status After Recover [Documentation] Create original cluster list and check Status for all shards in Ovsdb application. - Check Ovsdb Shards Status After Cluster Event ${original_cluster_list} + ClusterOvsdb.Check Ovsdb Shards Status After Cluster Event ${original_cluster_list} Check Entity Owner Status After Recover [Documentation] Check Entity Owner Status and identify owner and candidate. - ${new_owner} ${new_candidates_list} Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} + ${new_owner} ${new_candidates_list} ClusterOvsdb.Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} ovsdb://uuid/${ovsdb_uuid} Set Suite Variable ${new_owner} -Create Bridge Manually In Owner and Verify After Recover - [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge Manually And Verify ${original_cluster_list} ${new_owner} - -Add Port Manually In Owner and Verify After Recover - [Documentation] Add Port in Owner and verify it gets applied from all instances. - Add Port To The Manual Bridge And Verify ${original_cluster_list} ${new_owner} - -Create Bridge Via Controller In Owner and Verify After Recover - [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge And Verify ${original_cluster_list} ${new_owner} - -Verify Bridge in Restarted Node Which Is Killed Earlier - [Documentation] Verify Bridge in Restarted node, which is created when the node is down. - Verify Bridge in Restarted Node ${original_cluster_list} - -Create Port Via Controller In Owner and Verify After Recover - [Documentation] Create Port in Owner and verify it gets applied from all instances. - Create Port Via Controller ${original_cluster_list} ${new_owner} - -Verify Port in Restarted Node Which Is Killed Earlier - [Documentation] Verify Port in Restarted node, which is created when the node is down. - Verify Port in Restarted Node ${original_cluster_list} - -Modify the destination IP of Port After Recover - [Documentation] This will modify the dst ip of existing port - Modify the destination IP of Port ${original_cluster_list} ${new_owner} +Create Bridge Manually and Verify After Recover + [Documentation] Create bridge with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Bridge Manually And Verify ${original_cluster_list} -Get Operational Topology with modified Port After Recover - [Documentation] This request will fetch the operational topology after the modified port is added to the bridge - Get Operational Topology with modified Port ${original_cluster_list} ${new_owner} +Add Port Manually and Verify After Recover + [Documentation] Add port with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Add Sample Port To The Manual Bridge And Verify ${original_cluster_list} -Delete the Port After Recover - [Documentation] This request will delete the port node from the bridge node and data store. - Delete Port And Verify ${original_cluster_list} ${new_owner} +Delete the Bridge Manually and Verify After Recover + [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge Manually And Verify ${original_cluster_list} -Delete the Bridge In Owner and Verify After Recover - [Documentation] This request will delete the bridge node from the operational data store. - Delete Bridge Manually And Verify ${original_cluster_list} ${new_owner} +Verify Modified Port After Recover + [Documentation] Verify modified port exists in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${original_cluster_list} -Delete Bridge Via Rest Call And Verify In Owner After Recover - [Documentation] This request will delete the bridge node from the config data store and operational data store. - Delete Bridge Via Rest Call And Verify ${original_cluster_list} ${new_owner} - -Create Bridge Manually In Old Owner and Verify After Recover - [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge Manually And Verify ${original_cluster_list} ${original_owner} +Delete Port In New Owner After Recover + [Documentation] Delete port in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Port And Verify ${original_cluster_list} ${new_owner} -Add Port Manually In Old Owner and Verify After Recover - [Documentation] Add Port in Owner and verify it gets applied from all instances. - Add Port To The Manual Bridge And Verify ${original_cluster_list} ${original_owner} +Delete Bridge In New Owner And Verify After Recover + [Documentation] Delete bridge in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge And Verify ${original_cluster_list} ${new_owner} -Create Bridge Via Controller In Old Owner and Verify After Recover +Create Bridge In Old Owner and Verify After Recover [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge And Verify ${original_cluster_list} ${original_owner} + ClusterOvsdb.Create Sample Bridge And Verify ${original_cluster_list} ${original_owner} -Create Port Via Controller In Old Owner and Verify After Recover +Create Port In Old Owner and Verify After Recover [Documentation] Create Port in Owner and verify it gets applied from all instances. - Create Port Via Controller ${original_cluster_list} ${original_owner} + ClusterOvsdb.Create Sample Port And Verify ${original_cluster_list} ${original_owner} -Modify the destination IP of Port After Recover - [Documentation] This will modify the dst ip of existing port - Modify the destination IP of Port ${original_cluster_list} ${original_owner} +Modify the destination IP of Port In Old Owner After Recover + [Documentation] Modify the dst ip of existing port in Owner. + ClusterOvsdb.Modify the destination IP of Sample Port ${original_cluster_list} ${original_owner} -Get Operational Topology with modified Port After Recover - [Documentation] This request will fetch the operational topology after the modified port is added to the bridge - Get Operational Topology with modified Port ${original_cluster_list} ${original_owner} +Verify Port Is Modified After Recover + [Documentation] Verify port is modified in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${original_cluster_list} -Delete the Port After Recover - [Documentation] This request will delete the port node from the bridge node and data store. - Delete Port And Verify ${original_cluster_list} ${original_owner} +Delete Port In Old Owner After Recover + [Documentation] Delete port in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Port And Verify ${original_cluster_list} ${original_owner} -Delete the Bridge In Old Owner and Verify After Recover - [Documentation] This request will delete the bridge node from the config data store and operational data store. - Delete Bridge Manually And Verify ${original_cluster_list} ${original_owner} +Delete Bridge In Old Owner And Verify After Recover + [Documentation] Delete bridge in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge And Verify ${original_cluster_list} ${original_owner} -Delete Bridge Via Rest Call And Verify In Old Owner After Recover - [Documentation] This request will delete the bridge node from the config data store and operational data store. - Delete Bridge Via Rest Call And Verify ${original_cluster_list} ${original_owner} +Cleans Up Test Environment For Next Suite + [Documentation] Cleans up test environment, close existing sessions in teardown. + [Tags] exclude + ClusterOvsdb.Configure Exit OVSDB Connection ${original_cluster_list} diff --git a/csit/suites/ovsdb/Southbound_Cluster/020__Southbound_Cluster_Extension.robot b/csit/suites/ovsdb/Southbound_Cluster/020__Southbound_Cluster_Extension.robot index 4932d682ad..c119482447 100644 --- a/csit/suites/ovsdb/Southbound_Cluster/020__Southbound_Cluster_Extension.robot +++ b/csit/suites/ovsdb/Southbound_Cluster/020__Southbound_Cluster_Extension.robot @@ -1,106 +1,180 @@ *** Settings *** -Documentation Test suite for Ovsdb Southbound Cluster Extension +Documentation Test suite for Ovsdb Southbound Cluster - Candidate failover and recover Suite Setup Create Controller Sessions Suite Teardown Delete All Sessions Library RequestsLibrary Resource ../../../libraries/ClusterOvsdb.robot Resource ../../../libraries/ClusterKeywords.robot -Resource ../../../libraries/MininetKeywords.robot Variables ../../../variables/Variables.py -*** Variables *** -${OVSDB_CONFIG_DIR} ${CURDIR}/../../../variables/ovsdb -${BRIDGE} br01 - *** Test Cases *** Create Original Cluster List [Documentation] Create original cluster list. - ${original_cluster_list} Create Controller Index List + ${original_cluster_list} ClusterKeywords.Create Controller Index List Set Suite Variable ${original_cluster_list} Log ${original_cluster_list} Check Shards Status Before Fail [Documentation] Check Status for all shards in Ovsdb application. - Check Ovsdb Shards Status ${original_cluster_list} + ClusterOvsdb.Check Ovsdb Shards Status ${original_cluster_list} + +Start OVS Multiple Connections + [Documentation] Connect OVS to all cluster instances. + ${ovsdb_uuid} Ovsdb.Add Multiple Managers to OVS ${TOOLS_SYSTEM_IP} ${original_cluster_list} + Set Suite Variable ${ovsdb_uuid} Check Entity Owner Status And Find Owner and Candidate Before Fail [Documentation] Check Entity Owner Status and identify owner and candidate. - ${original_owner} ${original_candidates_list} Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} + ${original_owner} ${original_candidates_list} ClusterOvsdb.Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} ovsdb://uuid/${ovsdb_uuid} ${original_candidate}= Get From List ${original_candidates_list} 0 Set Suite Variable ${original_owner} Set Suite Variable ${original_candidate} -Create Bridge Via Controller In Owner and Verify Before Fail +Create Bridge Manually and Verify Before Fail + [Documentation] Create bridge with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Bridge Manually And Verify ${original_cluster_list} + +Add Port Manually and Verify Before Fail + [Documentation] Add port with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Add Sample Port To The Manual Bridge And Verify ${original_cluster_list} + +Delete the Bridge Manually and Verify Before Fail + [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge Manually And Verify ${original_cluster_list} + +Create Bridge In Owner and Verify Before Fail [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge In Candidate ${original_cluster_list} ${original_owner} BeforeCandidateFail + ClusterOvsdb.Create Sample Bridge And Verify ${original_cluster_list} ${original_owner} -Create Port Via Controller In Owner and Verify Before Fail +Create Port In Owner and Verify Before Fail [Documentation] Create Port in Owner and verify it gets applied from all instances. - Create Port Vxlan in Candidate ${original_cluster_list} ${original_owner} BeforeCandidateFail + ClusterOvsdb.Create Sample Port And Verify ${original_cluster_list} ${original_owner} + +Modify the destination IP of Port In Owner Before Fail + [Documentation] Modify the dst ip of existing port in Owner. + ClusterOvsdb.Modify the destination IP of Sample Port ${original_cluster_list} ${original_owner} + +Verify Port Is Modified Before Fail + [Documentation] Verify port is modified in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${original_cluster_list} -Kill Non Owner Instance - [Documentation] Kill Non Owner Instance and verify it is dead - Kill Multiple Controllers ${original_candidate} - ${new_cluster_list} Create Controller Index List +Delete Port In Owner Before Fail + [Documentation] Delete port in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Port And Verify ${original_cluster_list} ${original_owner} + +Delete Bridge In Owner And Verify Before Fail + [Documentation] Delete bridge in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge And Verify ${original_cluster_list} ${original_owner} + +Kill Candidate Instance + [Documentation] Kill Owner Instance and verify it is dead + ClusterKeywords.Kill Multiple Controllers ${original_candidate} + ${new_cluster_list} ClusterKeywords.Create Controller Index List Remove Values From List ${new_cluster_list} ${original_candidate} Set Suite Variable ${new_cluster_list} Check Shards Status After Fail [Documentation] Create original cluster list and check Status for all shards in Ovsdb application. - Check Ovsdb Shards Status After Cluster Event ${new_cluster_list} + ClusterOvsdb.Check Ovsdb Shards Status After Cluster Event ${new_cluster_list} Check Entity Owner Status And Find Owner and Candidate After Fail [Documentation] Check Entity Owner Status and identify owner and candidate. - ${new_owner} ${new_candidates_list} Get Ovsdb Entity Owner Status For One Device ${new_cluster_list} - Run Keyword And Continue On Failure List Should Not Contain Value ${new_candidates_list} ${original_candidate} Original candidate ${original_candidate} still in candidate list. - Remove Values From List ${new_candidates_list} ${original_candidate} + ${new_owner} ${new_candidates_list} ClusterOvsdb.Get Ovsdb Entity Owner Status For One Device ${new_cluster_list} ovsdb://uuid/${ovsdb_uuid} + Run Keyword And Continue On Failure List Should Not Contain Value ${new_candidates_list} ${original_owner} Original owner ${original_owner} still in candidate list. + Remove Values From List ${new_candidates_list} ${original_owner} ${new_candidate}= Get From List ${new_candidates_list} 0 Set Suite Variable ${new_owner} Set Suite Variable ${new_candidate} -Create Bridge Via Controller In Owner and Verify After Fail +Create Bridge Manually and Verify After Fail + [Documentation] Create bridge with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Bridge Manually And Verify ${new_cluster_list} + +Add Port Manually and Verify After Fail + [Documentation] Add port with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Add Sample Port To The Manual Bridge And Verify ${new_cluster_list} + +Delete the Bridge Manually and Verify After Fail + [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge Manually And Verify ${new_cluster_list} + +Create Bridge In Owner and Verify After Fail [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge In Candidate ${new_cluster_list} ${new_owner} AfterCandidateFail + ClusterOvsdb.Create Sample Bridge And Verify ${new_cluster_list} ${new_owner} -Start Non Old Owner Instance - [Documentation] Start Non Owner Instance and verify it is active - Start Multiple Controllers 300s ${original_candidate} +Create Port In Owner and Verify After Fail + [Documentation] Create Port in Owner and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Port And Verify ${new_cluster_list} ${new_owner} + +Modify the destination IP of Port In Owner After Fail + [Documentation] Modify the dst ip of existing port in Owner. + ClusterOvsdb.Modify the destination IP of Sample Port ${new_cluster_list} ${new_owner} + +Verify Port Is Modified After Fail + [Documentation] Verify port is modified in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${new_cluster_list} + +Start Old Candidate Instance + [Documentation] Start Owner Instance and verify it is active + ClusterKeywords.Start Multiple Controllers 300s ${original_candidate} Check Shards Status After Recover [Documentation] Create original cluster list and check Status for all shards in Ovsdb application. - Check Ovsdb Shards Status After Cluster Event ${original_cluster_list} + ClusterOvsdb.Check Ovsdb Shards Status After Cluster Event ${original_cluster_list} Check Entity Owner Status After Recover - [Documentation] Check Entity Owner Status and verify owner is not changed. - ${new_owner} ${new_candidates_list} Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} + [Documentation] Check Entity Owner Status and identify owner and candidate. + ${new_owner} ${new_candidates_list} ClusterOvsdb.Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} ovsdb://uuid/${ovsdb_uuid} Set Suite Variable ${new_owner} -Create Bridge Via Controller In Owner and Verify After Recover - [Documentation] Create Bridge in Owner and verify it gets applied from all instances. - Create Bridge In Candidate ${original_cluster_list} ${new_owner} +Create Bridge Manually and Verify After Recover + [Documentation] Create bridge with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Bridge Manually And Verify ${original_cluster_list} + +Add Port Manually and Verify After Recover + [Documentation] Add port with OVS command and verify it gets applied from all instances. + ClusterOvsdb.Add Sample Port To The Manual Bridge And Verify ${original_cluster_list} -Verify Bridge in Restarted Node Which Is Killed Earlier - [Documentation] Verify Bridge in Restarted node, which is created when the node is down. - Verify Bridge in Restarted Node ${original_cluster_list} AfterCandidateRecover +Delete the Bridge Manually and Verify After Recover + [Documentation] Delete bridge with OVS command and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge Manually And Verify ${original_cluster_list} -Create Port Via Controller In Owner and Verify After Recover +Verify Modified Port After Recover + [Documentation] Verify modified port exists in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${original_cluster_list} + +Delete Port In New Owner After Recover + [Documentation] Delete port in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Port And Verify ${original_cluster_list} ${new_owner} + +Delete Bridge In New Owner And Verify After Recover + [Documentation] Delete bridge in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge And Verify ${original_cluster_list} ${new_owner} + +Create Bridge In Old Candidate and Verify After Recover + [Documentation] Create Bridge in Owner and verify it gets applied from all instances. + ClusterOvsdb.Create Sample Bridge And Verify ${original_cluster_list} ${original_candidate} + +Create Port In Old Owner and Verify After Recover [Documentation] Create Port in Owner and verify it gets applied from all instances. - Create Port Vxlan in Candidate ${original_cluster_list} ${new_owner} + ClusterOvsdb.Create Sample Port And Verify ${original_cluster_list} ${original_candidate} + +Modify the destination IP of Port In Old Owner After Recover + [Documentation] Modify the dst ip of existing port in Owner. + ClusterOvsdb.Modify the destination IP of Sample Port ${original_cluster_list} ${original_candidate} -Verify Port in Restarted Node Which Is Killed Earlier - [Documentation] Verify Port in Restarted node, which is created when the node is down. - Verify Port in Restarted Node ${original_cluster_list} AfterCandidateRecover +Verify Port Is Modified After Recover + [Documentation] Verify port is modified in all instances. + ClusterOvsdb.Verify Sample Port Is Modified ${original_cluster_list} -Delete the Port After Recover - [Documentation] This request will delete the port node from the bridge node and data store. - Delete Port In Candidate ${original_cluster_list} ${new_owner} +Delete Port In Old Owner After Recover + [Documentation] Delete port in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Port And Verify ${original_cluster_list} ${original_candidate} -Delete Bridge Via Rest Call And Verify In Owner After Recover - [Documentation] This request will delete the bridge node from the config data store and operational data store. - Delete Bridge In Candidate ${original_cluster_list} ${new_owner} +Delete Bridge In Old Owner And Verify After Recover + [Documentation] Delete bridge in Owner and verify it gets deleted from all instances. + ClusterOvsdb.Delete Sample Bridge And Verify ${original_cluster_list} ${original_candidate} Cleans Up Test Environment For Next Suite - [Documentation] Cleans up test environment, close existing sessions in teardown. This step needs to be excluded - ... until the keyword "Get Cluster Entity Owner For Ovsdb" is fixed to search using ovs uuid as argument. - [Tags] exclude - Configure Exit OVSDB Connection ${original_cluster_list} ${new_owner} + [Documentation] Cleans up test environment, close existing sessions in teardown. + ClusterOvsdb.Configure Exit OVSDB Connection ${original_cluster_list} -- 2.36.6