From e32b9850edcf465ef63b2d4fd4c77853c2a9e784 Mon Sep 17 00:00:00 2001 From: Luis Gomez Date: Mon, 18 Jan 2016 20:16:24 -0800 Subject: [PATCH] Refactor OVSDB Cluster test - Add entity-owner shard check. - Increase shard timers so we know when there is delay or real failure. - Continue on entity-owner failure to update candidate list. - Small fixes to suites variables. Change-Id: If809be266a3fcc7d02bca284b9637f65ad8f1ce0 Signed-off-by: Luis Gomez --- csit/libraries/ClusterOvsdb.robot | 16 ++++++++++------ .../010__Ovsdb_Southbound_Cluster.robot | 14 ++++++++------ 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/csit/libraries/ClusterOvsdb.robot b/csit/libraries/ClusterOvsdb.robot index 9737dc55de..945ac7f7ec 100644 --- a/csit/libraries/ClusterOvsdb.robot +++ b/csit/libraries/ClusterOvsdb.robot @@ -11,12 +11,17 @@ Variables ../variables/Variables.py Check Ovsdb Shards Status [Arguments] ${controller_index_list} [Documentation] Check Status for all shards in Ovsdb application. - ${topo_conf_leader} ${topo_conf_followers_list} Wait Until Keyword Succeeds 10s 1s Get Cluster Shard Status ${controller_index_list} - ... config topology - ${topo_oper_leader} ${topo_oper_followers_list} Wait Until Keyword Succeeds 10s 1s Get Cluster Shard Status ${controller_index_list} - ... operational topology + ${topo_conf_leader} ${topo_conf_followers_list} Get Cluster Shard Status ${controller_index_list} config topology + ${topo_oper_leader} ${topo_oper_followers_list} Get Cluster Shard Status ${controller_index_list} operational topology + ${owner_oper_leader} ${owner_oper_followers_list} Get Cluster Shard Status ${controller_index_list} operational entity-ownership Log config topology Leader is ${topo_conf_leader} and followers are ${topo_conf_followers_list} Log operational topology Leader is ${topo_oper_leader} and followers are ${topo_oper_followers_list} + Log operational entity-ownership Leader is ${owner_oper_leader} and followers are ${owner_oper_followers_list} + +Check Ovsdb Shards Status After Cluster Event + [Arguments] ${controller_index_list} + [Documentation] Check Shard Status after some cluster event. + Wait Until Keyword Succeeds 90s 1s Check Ovsdb Shards Status ${controller_index_list} Get Ovsdb Entity Owner Status For One Device [Arguments] ${controller_index_list} @@ -56,7 +61,6 @@ Get Cluster Entity Owner For Ovsdb : FOR ${entity_candidate} IN @{entity_candidates_list} \ ${candidate}= Replace String &{entity_candidate}[name] member- ${EMPTY} \ ${candidate}= Convert To Integer ${candidate} - \ List Should Contain Value ${controller_index_list} ${candidate} Candidate ${candidate} not exisiting in ${controller_index_list} \ Run Keyword If '${candidate}' != '${owner}' Append To List ${candidates_list} ${candidate} [Return] ${owner} ${candidates_list} @@ -64,7 +68,7 @@ Create Bridge And Verify [Arguments] ${controller_index_list} ${controller_index} [Documentation] Create bridge in ${controller_index} and verify it gets applied in all instances in ${controller_index_list}. # need to get UUID which should be the same on all controllers in cluster, so asking controller1 - ${ovsdb_uuid}= Get OVSDB UUID controller_http_session=controller1 + ${ovsdb_uuid}= Get OVSDB UUID controller_http_session=controller${controller_index} ${body}= OperatingSystem.Get File ${CURDIR}/../variables/ovsdb/create_bridge_3node.json ${body} Replace String ${body} ovsdb://127.0.0.1:61644 ovsdb://uuid/${ovsdb_uuid} ${body} Replace String ${body} tcp:controller1:6633 tcp:${ODL_SYSTEM_1_IP}:6640 diff --git a/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot b/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot index 2a830f2f58..27385d75be 100644 --- a/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot +++ b/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot @@ -64,11 +64,13 @@ Kill Owner Instance Check Shards Status After Fail [Documentation] Create original cluster list and check Status for all shards in Ovsdb application. - Wait Until Keyword Succeeds 10s 1s Check Ovsdb Shards Status ${new_cluster_list} + Check Ovsdb Shards Status After Cluster Event ${new_cluster_list} Check Entity Owner Status And Find Owner and Candidate After Fail [Documentation] Check Entity Owner Status and identify owner and candidate. - ${new_owner} ${new_candidates_list} Wait Until Keyword Succeeds 10s 1s Get Ovsdb Entity Owner Status For One Device ${new_cluster_list} + ${new_owner} ${new_candidates_list} Get Ovsdb Entity Owner Status For One Device ${new_cluster_list} + Run Keyword And Continue On Failure List Should Not Contain Value ${new_candidates_list} ${original_owner} Original owner ${original_owner} still in candidate list. + Remove Values From List ${new_candidates_list} ${original_owner} ${new_candidate}= Get From List ${new_candidates_list} 0 Set Suite Variable ${new_owner} Set Suite Variable ${new_candidate} @@ -79,7 +81,7 @@ Create Bridge Manually In Owner and Verify After Fail Add Port Manually In Owner and Verify After Fail [Documentation] Add Port in Owner and verify it gets applied from all instances. - Add Port To The Manual Bridge And Verify ${original_cluster_list} ${original_owner} + Add Port To The Manual Bridge And Verify ${new_cluster_list} ${new_owner} Create Bridge Via Controller In Owner and Verify After Fail [Documentation] Create Bridge in Owner and verify it gets applied from all instances. @@ -99,11 +101,11 @@ Start Old Owner Instance Check Shards Status After Recover [Documentation] Create original cluster list and check Status for all shards in Ovsdb application. - Wait Until Keyword Succeeds 10s 1s Check Ovsdb Shards Status ${original_cluster_list} + Check Ovsdb Shards Status After Cluster Event ${original_cluster_list} Check Entity Owner Status After Recover [Documentation] Check Entity Owner Status and identify owner and candidate. - ${new_owner} ${new_candidates_list} Wait Until Keyword Succeeds 10s 1s Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} + ${new_owner} ${new_candidates_list} Get Ovsdb Entity Owner Status For One Device ${original_cluster_list} Set Suite Variable ${new_owner} Create Bridge Manually In Owner and Verify After Recover @@ -112,7 +114,7 @@ Create Bridge Manually In Owner and Verify After Recover Add Port Manually In Owner and Verify After Recover [Documentation] Add Port in Owner and verify it gets applied from all instances. - Add Port To The Manual Bridge And Verify ${original_cluster_list} ${original_owner} + Add Port To The Manual Bridge And Verify ${original_cluster_list} ${new_owner} Create Bridge Via Controller In Owner and Verify After Recover [Documentation] Create Bridge in Owner and verify it gets applied from all instances. -- 2.36.6