*** Settings ***
Documentation DOMRpcBroker testing: Common keywords
-...
+...
... Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.
-...
+...
... This program and the accompanying materials are made available under the
... terms of the Eclipse Public License v1.0 which accompanies this distribution,
... and is available at http://www.eclipse.org/legal/epl-v10.html
-...
+...
... The aim of this resource is to groups reusable blocks of commands into
... keywords. It should be initiated by DrbCommons_Init. It creates
... ${all_indices}, ${registered_indices}, ${nonregistered_indices} and
... constant from isolated node with regirered rpc is
... invalid
... ${active_indices} - list of indexes of non-isolated, non-stopped/killed nodes
+...
+... Akka can create spurious UnreachableMember events, see
+... https://bugs.opendaylight.org/show_bug.cgi?id=8430
+... so some keywords contain "tolerance" argument which applies BuiltIn.Wait_Until_Keyword_Succeeds.
+...
+... The delay before subsequent ReachableMember is significantly higher than
+... RPC registration delay documented at
+... http://docs.opendaylight.org/en/latest/developer-guide/controller.html#rpcs-and-cluster
Library Collections
Resource ${CURDIR}/../ClusterManagement.robot
Resource ${CURDIR}/../MdsalLowlevel.robot
*** Variables ***
${CONSTANT_PREFIX} constant-
+${CONTEXT} context
+${BUG_8430_TOLERANCE} 10
*** Keywords ***
DrbCommons_Init
DrbCommons__Rem_Possible_Constant ${member_index}
DrbCommons__Deregister_Index ${member_index}
+Register_Action_And_Update_Possible_Constants
+ [Arguments] ${member_index}
+ [Documentation] Register routed rpc on given node of the cluster.
+ MdsalLowlevel.Register_Bound_Constant ${member_index} ${CONTEXT} ${CONSTANT_PREFIX}${member_index}
+ DrbCommons__Add_Possible_Constant ${member_index}
+ DrbCommons__Register_Index ${member_index}
+
+Unregister_Action_And_Update_Possible_Constants
+ [Arguments] ${member_index}
+ [Documentation] Unregister routed rpc on given node of the cluster.
+ MdsalLowlevel.Unregister_Bound_Constant ${member_index} ${CONTEXT}
+ DrbCommons__Rem_Possible_Constant ${member_index}
+ DrbCommons__Deregister_Index ${member_index}
+
Register_Rpc_On_Nodes
[Arguments] ${index_list}
[Documentation] Register global rpc on given nodes of the cluster.
- : FOR ${index} IN @{index_list}
- \ Register_Rpc_And_Update_Possible_Constants ${index}
+ FOR ${index} IN @{index_list}
+ Register_Rpc_And_Update_Possible_Constants ${index}
+ END
Unregister_Rpc_On_Nodes
[Arguments] ${index_list}
[Documentation] Unregister global rpc on given nodes of the cluster.
- : FOR ${index} IN @{index_list}
- \ Unregister_Rpc_And_Update_Possible_Constants ${index}
+ FOR ${index} IN @{index_list}
+ Unregister_Rpc_And_Update_Possible_Constants ${index}
+ END
+
+Register_Action_On_Nodes
+ [Arguments] ${index_list}
+ [Documentation] Register global rpc on given nodes of the cluster.
+ FOR ${index} IN @{index_list}
+ Register_Action_And_Update_Possible_Constants ${index}
+ END
+
+Unregister_Action_On_Nodes
+ [Arguments] ${index_list}
+ [Documentation] Unregister global rpc on given nodes of the cluster.
+ FOR ${index} IN @{index_list}
+ Unregister_Action_And_Update_Possible_Constants ${index}
+ END
Verify_Constant_On_Registered_Node
[Arguments] ${member_index}
Collections.List_Should_Contain_Value ${possible_constants} ${constant}
BuiltIn.Return_From_Keyword ${constant}
+Verify_Contexted_Constant_On_Registered_Node
+ [Arguments] ${member_index}
+ [Documentation] Verify that the rpc response comes from the local node.
+ ${constant} = MdsalLowlevel.Get_Contexted_Constant ${member_index} ${CONTEXT}
+ BuiltIn.Should_Be_Equal_As_Strings ${CONSTANT_PREFIX}${member_index} ${constant}
+ BuiltIn.Return_From_Keyword ${constant}
+
+Verify_Contexted_Constant_On_Unregistered_Node
+ [Arguments] ${member_index}
+ [Documentation] Verify that the response comes from other nodes with rpc registered. Verification
+ ... passes for registered nodes too.
+ ${constant} = MdsalLowlevel.Get_Contexted_Constant ${member_index} ${CONTEXT}
+ Collections.List_Should_Contain_Value ${possible_constants} ${constant}
+ BuiltIn.Return_From_Keyword ${constant}
+
Verify_Constant_On_Registered_Nodes
[Arguments] ${index_list}
[Documentation] Verify that the rpc response comes from the local node for every node in the list.
- : FOR ${index} IN @{index_list}
- \ Verify_Constant_On_Registered_Node ${index}
+ FOR ${index} IN @{index_list}
+ Verify_Constant_On_Registered_Node ${index}
+ END
+
+Verify_Contexted_Constant_On_Registered_Nodes
+ [Arguments] ${index_list}
+ [Documentation] Verify that the rpc response comes from the local node for every node in the list.
+ FOR ${index} IN @{index_list}
+ Verify_Contexted_Constant_On_Registered_Node ${index}
+ END
Verify_Constant_On_Unregistered_Nodes
[Arguments] ${index_list}
[Documentation] Verify that the rpc response comes from the remote node for every node in the list.
- : FOR ${index} IN @{index_list}
- \ Verify_Constant_On_Unregistered_Node ${index}
+ FOR ${index} IN @{index_list}
+ Verify_Constant_On_Unregistered_Node ${index}
+ END
Verify_Constant_On_Active_Nodes
+ [Arguments] ${tolerance}=${BUG_8430_TOLERANCE}
+ [Documentation] Verify that the rpc response comes from the local node for every node in the list.
+ ... As a workaround for Bug 8430, \${tolerance} can be set as duration (number of seconds) for WUKS.
+ # TODO: Rename most Verify_* keywords to Check_* and use the Verify prefix for the WUKS versions.
+ BuiltIn.Run_Keyword_And_Return_If ${tolerance} BuiltIn.Wait_Until_Keyword_Succeeds ${tolerance} 1s Verify_Constant_On_Active_Nodes tolerance=0
+ FOR ${index} IN @{active_indices}
+ BuiltIn.Run_Keyword_If ${index} in ${registered_indices} Verify_Constant_On_Registered_Node ${index}
+ ... ELSE Verify_Constant_On_Unregistered_Node ${index}
+ END
+
+Verify_Contexted_Constant_On_Active_Nodes
+ [Arguments] ${tolerance}=${BUG_8430_TOLERANCE}
[Documentation] Verify that the rpc response comes from the local node for every node in the list.
- : FOR ${index} IN @{active_indices}
- \ BuiltIn.Run_Keyword_If ${index} in ${registered_indices} Verify_Constant_On_Registered_Node ${index}
- \ ... ELSE Verify_Constant_On_Unregistered_Node ${index}
+ ... As a workaround for Bug 8430, \${tolerance} can be set as duration (number of seconds) for WUKS.
+ # TODO: Rename most Verify_* keywords to Check_* and use the Verify prefix for the WUKS versions.
+ BuiltIn.Run_Keyword_And_Return_If ${tolerance} BuiltIn.Wait_Until_Keyword_Succeeds ${tolerance} 1s Verify_Contexted_Constant_On_Active_Nodes tolerance=0
+ FOR ${index} IN @{active_indices}
+ BuiltIn.Run_Keyword_If ${index} in ${registered_indices} Verify_Contexted_Constant_On_Registered_Node ${index}
+ ... ELSE Verify_Contexted_Constant_On_Unregistered_Node ${index}
+ END
Verify_Expected_Constant_On_Nodes
[Arguments] ${index_list} ${exp_constant}
[Documentation] Verify that the rpc response comes only from one node only for every node in the list.
- : FOR ${index} IN @{index_list}
- \ ${const_index} = Get_Constant_Index_From_Node ${index}
- \ BuiltIn.Should_Be_Equal_As_Strings ${exp_constant} ${CONSTANT_PREFIX}${const_index}
+ FOR ${index} IN @{index_list}
+ ${const_index} = Get_Constant_Index_From_Node ${index}
+ BuiltIn.Should_Be_Equal_As_Strings ${exp_constant} ${CONSTANT_PREFIX}${const_index}
+ END
Get_Constant_Index_From_Node
[Arguments] ${member_index}
${index} = BuiltIn.Convert_To_Integer ${index}
BuiltIn.Return_From_Keyword ${index}
+Get_Contexted_Constant_Index_From_Node
+ [Arguments] ${member_index}
+ [Documentation] Ivoke get-contexted-constant rpc on given member index. Returns the index of
+ ... the node where the constant came from.
+ ${constant} = MdsalLowlevel.Get_Contexted_Constant ${member_index} ${CONTEXT}
+ ${index} = String.Replace_String ${constant} ${CONSTANT_PREFIX} ${EMPTY}
+ ${index} = BuiltIn.Convert_To_Integer ${index}
+ BuiltIn.Return_From_Keyword ${index}
+
Isolate_Node
[Arguments] ${member_index}
[Documentation] Isolate a member and update appropriate suite variables.
ClusterManagement.Isolate_Member_From_List_Or_All ${member_index}
- DrbCommons__Upadte_Active_Nodes_List deactivate_idx=${member_index}
+ DrbCommons__Update_Active_Nodes_List deactivate_idx=${member_index}
BuiltIn.Return_From_Keyword_If ${member_index} not in ${registered_indices}
DrbCommons__Rem_Possible_Constant ${member_index}
[Arguments] ${member_index}
[Documentation] Rejoin a member and update appropriate suite variables.
ClusterManagement.Rejoin_Member_From_List_Or_All ${member_index}
- BuiltIn.Wait_Until_Keyword_Succeeds 60s 10s ShardStability.Shards_Stability_Get_Details ${DEFAULT_SHARD_LIST}
- DrbCommons__Upadte_Active_Nodes_List activate_idx=${member_index}
+ BuiltIn.Wait_Until_Keyword_Succeeds 70s 10s ShardStability.Shards_Stability_Get_Details ${DEFAULT_SHARD_LIST}
+ DrbCommons__Update_Active_Nodes_List activate_idx=${member_index}
BuiltIn.Return_From_Keyword_If ${member_index} not in ${registered_indices}
DrbCommons__Add_Possible_Constant ${member_index}
-DrbCommons__Upadte_Active_Nodes_List
+DrbCommons__Update_Active_Nodes_List
[Arguments] ${activate_idx}=${EMPTY} ${deactivate_idx}=${EMPTY}
[Documentation] Add or remove member index to/from the list of active nodes.
BuiltIn.Run_Keyword_If "${activate_idx}" != "${EMPTY}" Collections.Append_To_List ${active_indices} ${activate_idx}
Code Review / integration / test.git / blobdiff