X-Git-Url: https://git.opendaylight.org/gerrit/gitweb?a=blobdiff_plain;f=csit%2Flibraries%2Fcontroller%2FDrbCommons.robot;h=d835db8585c63735a4345724b051bfb24b8164e3;hb=39de3404b2681d425af82a66f7f665ecd4a2da3a;hp=9fbcd4ed30e00057c854a0448fa9a81f7a849da2;hpb=197d624d58b6595a2ef07847fd10eeb6ee651ef3;p=integration%2Ftest.git diff --git a/csit/libraries/controller/DrbCommons.robot b/csit/libraries/controller/DrbCommons.robot index 9fbcd4ed30..d835db8585 100644 --- a/csit/libraries/controller/DrbCommons.robot +++ b/csit/libraries/controller/DrbCommons.robot @@ -1,12 +1,12 @@ *** Settings *** Documentation DOMRpcBroker testing: Common keywords -... +... ... Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved. -... +... ... This program and the accompanying materials are made available under the ... terms of the Eclipse Public License v1.0 which accompanies this distribution, ... and is available at http://www.eclipse.org/legal/epl-v10.html -... +... ... The aim of this resource is to groups reusable blocks of commands into ... keywords. It should be initiated by DrbCommons_Init. It creates ... ${all_indices}, ${registered_indices}, ${nonregistered_indices} and @@ -20,6 +20,14 @@ Documentation DOMRpcBroker testing: Common keywords ... constant from isolated node with regirered rpc is ... invalid ... ${active_indices} - list of indexes of non-isolated, non-stopped/killed nodes +... +... Akka can create spurious UnreachableMember events, see +... https://bugs.opendaylight.org/show_bug.cgi?id=8430 +... so some keywords contain "tolerance" argument which applies BuiltIn.Wait_Until_Keyword_Succeeds. +... +... The delay before subsequent ReachableMember is significantly higher than +... RPC registration delay documented at +... http://docs.opendaylight.org/en/latest/developer-guide/controller.html#rpcs-and-cluster Library Collections Resource ${CURDIR}/../ClusterManagement.robot Resource ${CURDIR}/../MdsalLowlevel.robot @@ -27,6 +35,8 @@ Resource ${CURDIR}/../ShardStability.robot *** Variables *** ${CONSTANT_PREFIX} constant- +${CONTEXT} context +${BUG_8430_TOLERANCE} 10 *** Keywords *** DrbCommons_Init @@ -57,17 +67,47 @@ Unregister_Rpc_And_Update_Possible_Constants DrbCommons__Rem_Possible_Constant ${member_index} DrbCommons__Deregister_Index ${member_index} +Register_Action_And_Update_Possible_Constants + [Arguments] ${member_index} + [Documentation] Register routed rpc on given node of the cluster. + MdsalLowlevel.Register_Bound_Constant ${member_index} ${CONTEXT} ${CONSTANT_PREFIX}${member_index} + DrbCommons__Add_Possible_Constant ${member_index} + DrbCommons__Register_Index ${member_index} + +Unregister_Action_And_Update_Possible_Constants + [Arguments] ${member_index} + [Documentation] Unregister routed rpc on given node of the cluster. + MdsalLowlevel.Unregister_Bound_Constant ${member_index} ${CONTEXT} + DrbCommons__Rem_Possible_Constant ${member_index} + DrbCommons__Deregister_Index ${member_index} + Register_Rpc_On_Nodes [Arguments] ${index_list} [Documentation] Register global rpc on given nodes of the cluster. - : FOR ${index} IN @{index_list} - \ Register_Rpc_And_Update_Possible_Constants ${index} + FOR ${index} IN @{index_list} + Register_Rpc_And_Update_Possible_Constants ${index} + END Unregister_Rpc_On_Nodes [Arguments] ${index_list} [Documentation] Unregister global rpc on given nodes of the cluster. - : FOR ${index} IN @{index_list} - \ Unregister_Rpc_And_Update_Possible_Constants ${index} + FOR ${index} IN @{index_list} + Unregister_Rpc_And_Update_Possible_Constants ${index} + END + +Register_Action_On_Nodes + [Arguments] ${index_list} + [Documentation] Register global rpc on given nodes of the cluster. + FOR ${index} IN @{index_list} + Register_Action_And_Update_Possible_Constants ${index} + END + +Unregister_Action_On_Nodes + [Arguments] ${index_list} + [Documentation] Unregister global rpc on given nodes of the cluster. + FOR ${index} IN @{index_list} + Unregister_Action_And_Update_Possible_Constants ${index} + END Verify_Constant_On_Registered_Node [Arguments] ${member_index} @@ -84,30 +124,71 @@ Verify_Constant_On_Unregistered_Node Collections.List_Should_Contain_Value ${possible_constants} ${constant} BuiltIn.Return_From_Keyword ${constant} +Verify_Contexted_Constant_On_Registered_Node + [Arguments] ${member_index} + [Documentation] Verify that the rpc response comes from the local node. + ${constant} = MdsalLowlevel.Get_Contexted_Constant ${member_index} ${CONTEXT} + BuiltIn.Should_Be_Equal_As_Strings ${CONSTANT_PREFIX}${member_index} ${constant} + BuiltIn.Return_From_Keyword ${constant} + +Verify_Contexted_Constant_On_Unregistered_Node + [Arguments] ${member_index} + [Documentation] Verify that the response comes from other nodes with rpc registered. Verification + ... passes for registered nodes too. + ${constant} = MdsalLowlevel.Get_Contexted_Constant ${member_index} ${CONTEXT} + Collections.List_Should_Contain_Value ${possible_constants} ${constant} + BuiltIn.Return_From_Keyword ${constant} + Verify_Constant_On_Registered_Nodes [Arguments] ${index_list} [Documentation] Verify that the rpc response comes from the local node for every node in the list. - : FOR ${index} IN @{index_list} - \ Verify_Constant_On_Registered_Node ${index} + FOR ${index} IN @{index_list} + Verify_Constant_On_Registered_Node ${index} + END + +Verify_Contexted_Constant_On_Registered_Nodes + [Arguments] ${index_list} + [Documentation] Verify that the rpc response comes from the local node for every node in the list. + FOR ${index} IN @{index_list} + Verify_Contexted_Constant_On_Registered_Node ${index} + END Verify_Constant_On_Unregistered_Nodes [Arguments] ${index_list} [Documentation] Verify that the rpc response comes from the remote node for every node in the list. - : FOR ${index} IN @{index_list} - \ Verify_Constant_On_Unregistered_Node ${index} + FOR ${index} IN @{index_list} + Verify_Constant_On_Unregistered_Node ${index} + END Verify_Constant_On_Active_Nodes + [Arguments] ${tolerance}=${BUG_8430_TOLERANCE} + [Documentation] Verify that the rpc response comes from the local node for every node in the list. + ... As a workaround for Bug 8430, \${tolerance} can be set as duration (number of seconds) for WUKS. + # TODO: Rename most Verify_* keywords to Check_* and use the Verify prefix for the WUKS versions. + BuiltIn.Run_Keyword_And_Return_If ${tolerance} BuiltIn.Wait_Until_Keyword_Succeeds ${tolerance} 1s Verify_Constant_On_Active_Nodes tolerance=0 + FOR ${index} IN @{active_indices} + BuiltIn.Run_Keyword_If ${index} in ${registered_indices} Verify_Constant_On_Registered_Node ${index} + ... ELSE Verify_Constant_On_Unregistered_Node ${index} + END + +Verify_Contexted_Constant_On_Active_Nodes + [Arguments] ${tolerance}=${BUG_8430_TOLERANCE} [Documentation] Verify that the rpc response comes from the local node for every node in the list. - : FOR ${index} IN @{active_indices} - \ BuiltIn.Run_Keyword_If ${index} in ${registered_indices} Verify_Constant_On_Registered_Node ${index} - \ ... ELSE Verify_Constant_On_Unregistered_Node ${index} + ... As a workaround for Bug 8430, \${tolerance} can be set as duration (number of seconds) for WUKS. + # TODO: Rename most Verify_* keywords to Check_* and use the Verify prefix for the WUKS versions. + BuiltIn.Run_Keyword_And_Return_If ${tolerance} BuiltIn.Wait_Until_Keyword_Succeeds ${tolerance} 1s Verify_Contexted_Constant_On_Active_Nodes tolerance=0 + FOR ${index} IN @{active_indices} + BuiltIn.Run_Keyword_If ${index} in ${registered_indices} Verify_Contexted_Constant_On_Registered_Node ${index} + ... ELSE Verify_Contexted_Constant_On_Unregistered_Node ${index} + END Verify_Expected_Constant_On_Nodes [Arguments] ${index_list} ${exp_constant} [Documentation] Verify that the rpc response comes only from one node only for every node in the list. - : FOR ${index} IN @{index_list} - \ ${const_index} = Get_Constant_Index_From_Node ${index} - \ BuiltIn.Should_Be_Equal_As_Strings ${exp_constant} ${CONSTANT_PREFIX}${const_index} + FOR ${index} IN @{index_list} + ${const_index} = Get_Constant_Index_From_Node ${index} + BuiltIn.Should_Be_Equal_As_Strings ${exp_constant} ${CONSTANT_PREFIX}${const_index} + END Get_Constant_Index_From_Node [Arguments] ${member_index} @@ -118,11 +199,20 @@ Get_Constant_Index_From_Node ${index} = BuiltIn.Convert_To_Integer ${index} BuiltIn.Return_From_Keyword ${index} +Get_Contexted_Constant_Index_From_Node + [Arguments] ${member_index} + [Documentation] Ivoke get-contexted-constant rpc on given member index. Returns the index of + ... the node where the constant came from. + ${constant} = MdsalLowlevel.Get_Contexted_Constant ${member_index} ${CONTEXT} + ${index} = String.Replace_String ${constant} ${CONSTANT_PREFIX} ${EMPTY} + ${index} = BuiltIn.Convert_To_Integer ${index} + BuiltIn.Return_From_Keyword ${index} + Isolate_Node [Arguments] ${member_index} [Documentation] Isolate a member and update appropriate suite variables. ClusterManagement.Isolate_Member_From_List_Or_All ${member_index} - DrbCommons__Upadte_Active_Nodes_List deactivate_idx=${member_index} + DrbCommons__Update_Active_Nodes_List deactivate_idx=${member_index} BuiltIn.Return_From_Keyword_If ${member_index} not in ${registered_indices} DrbCommons__Rem_Possible_Constant ${member_index} @@ -131,11 +221,11 @@ Rejoin_Node [Documentation] Rejoin a member and update appropriate suite variables. ClusterManagement.Rejoin_Member_From_List_Or_All ${member_index} BuiltIn.Wait_Until_Keyword_Succeeds 70s 10s ShardStability.Shards_Stability_Get_Details ${DEFAULT_SHARD_LIST} - DrbCommons__Upadte_Active_Nodes_List activate_idx=${member_index} + DrbCommons__Update_Active_Nodes_List activate_idx=${member_index} BuiltIn.Return_From_Keyword_If ${member_index} not in ${registered_indices} DrbCommons__Add_Possible_Constant ${member_index} -DrbCommons__Upadte_Active_Nodes_List +DrbCommons__Update_Active_Nodes_List [Arguments] ${activate_idx}=${EMPTY} ${deactivate_idx}=${EMPTY} [Documentation] Add or remove member index to/from the list of active nodes. BuiltIn.Run_Keyword_If "${activate_idx}" != "${EMPTY}" Collections.Append_To_List ${active_indices} ${activate_idx}