${sync_status} = Collections.Get_From_Dictionary dictionary=${value_object} key=SyncStatus
[Return] ${sync_status}
+List_All_Indices
+ [Documentation] Create a new list of all indices.
+ ${return_list_copy} = ClusterManagement__Given_Or_Internal_Index_List
+ BuiltIn.Return_From_Keyword ${return_list_copy}
+
List_Indices_Minus_Member
[Arguments] ${member_index} ${member_index_list}=${EMPTY}
[Documentation] Create a new list which contains indices from ${member_index_list} (or all) without ${member_index}.
- ${index_list} = ClusterManagement__Given_Or_Empty_List ${member_index_list}
+ ${index_list} = ClusterManagement__Given_Or_Internal_Index_List ${member_index_list}
Collections.Remove Values From List ${index_list} ${member_index}
[Return] ${index_list}
... This program and the accompanying materials are made available under the
... terms of the Eclipse Public License v1.0 which accompanies this distribution,
... and is available at http://www.eclipse.org/legal/epl-v10.html
+Library XML
Resource ${CURDIR}/ClusterManagement.robot
Resource ${CURDIR}/TemplatedRequests.robot
*** Keywords ***
Get_Constant
- [Arguments] ${member_index}
- [Documentation] TODO: more desctiptive comment than: Invoke get-constant rpc.
+ [Arguments] ${member_index} ${explicit_status_codes}=${NO_STATUS_CODES}
+ [Documentation] Invoke get-constant rpc on the requested member and return the registered constant. The ${explicit_status_codes} is a list
+ ... of http status codes for which the rpc call is considered as passed and is used for calls with expected failures on odl's side, such as
+ ... calling the rpc on isolated node etc.
${session} = ClusterManagement.Resolve_Http_Session_For_Member member_index=${member_index}
- ${text} = TemplatedRequests.Post_As_Xml_Templated ${GET_CONSTANT_DIR} session=${session}
- BuiltIn.Fail TODO: to format output data
- BuiltIn.Return_From_Keyword ${formatted_output}
+ ${text} = TemplatedRequests.Post_As_Xml_Templated ${GET_CONSTANT_DIR} session=${session} explicit_status_codes=${explicit_status_codes}
+ ${xml} = XML.Parse_Xml ${text}
+ ${constant} = XML.Get_Element_Text ${xml} xpath=constant
+ BuiltIn.Return_From_Keyword ${constant}
Get_Contexted_Constant
[Arguments] ${member_index} ${context}
Register_Constant
[Arguments] ${member_index} ${constant}
- [Documentation] TODO: more desctiptive comment than: Invoke register-constant rpc.
+ [Documentation] Register the get-constant rpc on the requested node with given constant.
${session} = ClusterManagement.Resolve_Http_Session_For_Member member_index=${member_index}
&{mapping} BuiltIn.Create_Dictionary CONSTANT=${constant}
TemplatedRequests.Post_As_Xml_Templated ${REGISTER_CONSTANT_DIR} mapping=${mapping} session=${session}
Unregister_Constant
[Arguments] ${member_index}
- [Documentation] TODO: more desctiptive comment than: Invoke unregister-constant rpc.
+ [Documentation] Unregister the get-constant rpc on the given node.
${session} = ClusterManagement.Resolve_Http_Session_For_Member member_index=${member_index}
- ${uri} = TemplatedRequests.Resolve_Text_From_Template_Folder folder=${UNREGISTER_CONSTANT_DIR} base_name=location extension=uri
- ${text} = TemplatedRequests.Post_To_Uri uri=${uri} data=${EMPTY} accept=${ACCEPT_JSON} content_type=${HEADERS_YANG_JSON} session=${session}
+ TemplatedRequests.Post_As_Xml_Templated ${UNREGISTER_CONSTANT_DIR} session=${session}
Register_Singleton_Constant
[Arguments] ${member_index} ${constant}
Post_As_Xml_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
+ ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES}
[Documentation] Add arguments sensible for XML data, return Post_Templated response text.
... Optionally, verification against response.xml (no iteration) is called.
# In case of iterations, we use endlines in data to send, as it should not matter and it is more readable.
${response_text} = Post_Templated folder=${folder} base_name=data extension=xml accept=${ACCEPT_XML} content_type=${HEADERS_XML}
... mapping=${mapping} session=${session} normalize_json=False endline=${\n} iterations=${iterations} iter_start=${iter_start}
+ ... additional_allowed_status_codes=${additional_allowed_status_codes} explicit_status_codes=${explicit_status_codes}
BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Xml_Templated response=${response_text} folder=${folder} base_name=response mapping=${mapping}
[Return] ${response_text}
Post_Templated
[Arguments] ${folder} ${base_name} ${extension} ${content_type} ${accept} ${mapping}={}
- ... ${session}=default ${normalize_json}=False ${endline}=${\n} ${iterations}=${EMPTY} ${iter_start}=1
+ ... ${session}=default ${normalize_json}=False ${endline}=${\n} ${iterations}=${EMPTY} ${iter_start}=1 ${additional_allowed_status_codes}=${NO_STATUS_CODES}
+ ... ${explicit_status_codes}=${NO_STATUS_CODES}
[Documentation] Resolve URI and data from folder, call Post_To_Uri, return response text.
${uri} = Resolve_Text_From_Template_Folder folder=${folder} base_name=location extension=uri mapping=${mapping}
${data} = Resolve_Text_From_Template_Folder folder=${folder} name_prefix=post_ base_name=${base_name} extension=${extension} mapping=${mapping}
... endline=${endline} iterations=${iterations} iter_start=${iter_start}
${response_text} = Post_To_Uri uri=${uri} data=${data} content_type=${content_type} accept=${accept} session=${session}
- ... normalize_json=${normalize_json}
+ ... normalize_json=${normalize_json} additional_allowed_status_codes=${additional_allowed_status_codes} explicit_status_codes=${explicit_status_codes}
[Return] ${response_text}
Verify_Response_Templated
Post_To_Uri
[Arguments] ${uri} ${data} ${content_type} ${accept} ${session}=default ${normalize_json}=False
+ ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES}
[Documentation] POST data to given URI, check status code and return response text.
... \${content_type} and \${accept} are mandatory Python objects with headers to use.
... If \${normalize_json}, normalize text before returning.
BuiltIn.Log ${accept}
${headers} = Join_Two_Headers first=${content_type} second=${accept}
${response} = RequestsLibrary.Post_Request alias=${session} uri=${uri} data=${data} headers=${headers}
- Check_Status_Code ${response}
+ Check_Status_Code ${response} additional_allowed_status_codes=${additional_allowed_status_codes} explicit_status_codes=${explicit_status_codes}
BuiltIn.Run_Keyword_Unless ${normalize_json} BuiltIn.Return_From_Keyword ${response.text}
${text_normalized} = norm_json.normalize_json_text ${response.text}
[Return] ${text_normalized}
--- /dev/null
+*** Settings ***
+Documentation DOMRpcBroker testing: RPC Provider Partition And Heal
+...
+... Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.
+...
+... This program and the accompanying materials are made available under the
+... terms of the Eclipse Public License v1.0 which accompanies this distribution,
+... and is available at http://www.eclipse.org/legal/epl-v10.html
+...
+... This tests establishes that the RPC service operates correctly when faced
+... with node failures.
+... This suite supports more than three node cluster setup too.
+Suite Setup Setup_Kw
+Suite Teardown SSHLibrary.Close_All_Connections
+Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+Test Teardown SetupUtils.Teardown_Test_Show_Bugs_If_Test_Failed
+Default Tags critical
+Library Collections
+Library SSHLibrary
+Resource ${CURDIR}/../../../libraries/MdsalLowlevel.robot
+Resource ${CURDIR}/../../../libraries/SetupUtils.robot
+Resource ${CURDIR}/../../../libraries/TemplatedRequests.robot
+Resource ${CURDIR}/../../../libraries/WaitForFailure.robot
+
+*** Variables ***
+@{INSTALLED_RPC_MEMEBER_IDX_LIST} ${1} ${2}
+${TESTED_MEMBER_WITHOUT_RPC_IDX} ${3}
+${CONSTANT_PREFIX} member-
+@{NON_WORKING_RPC_STATUS_CODE} ${501}
+
+*** Test Cases ***
+Register_Rpc_On_Two_Nodes
+ [Documentation] Register rpc on two nodes of the odl cluster.
+ : FOR ${index} IN @{INSTALLED_RPC_MEMEBER_IDX_LIST}
+ \ MdsalLowlevel.Register_Constant ${index} ${CONSTANT_PREFIX}${index}
+
+Invoke_Rpc_On_Each_Node
+ [Documentation] Invoke get-constant rpc on every node of the cluster. When requested on the node with
+ ... local instance the local value is expected. If invoked on the node with no local instance, any remote
+ ... value is expected. From the constant returned from the ${TESTED_MEMBER_WITHOUT_RPC_IDX} node (with no rpc instance) an index of
+ ... the node to be isolated is derived. And in the tc Invoke_Rpc_On_Remaining_Nodes a different constant
+ ... is expected. The second for loop makes the suite suitable for more that 3 nodes cluster.
+ : FOR ${index} IN @{INSTALLED_RPC_MEMEBER_IDX_LIST}
+ \ Verify_Local_Rpc_Invoked ${index}
+ : FOR ${index} IN @{non_installed_rpc_member_idx_list}
+ \ ${constant} = Verify_Any_Remote_Rpc_Invoked ${index}
+ \ BuiltIn.Run_Keyword_If "${index}" == "${TESTED_MEMBER_WITHOUT_RPC_IDX}" BuiltIn.Set_Suite_Variable ${initial_const_on_tested_non_rpc_member} ${constant}
+ ${isolated_idx} = String.Replace_String ${initial_const_on_tested_non_rpc_member} ${CONSTANT_PREFIX} ${EMPTY}
+ BuiltIn.Set_Suite_Variable ${isolated_idx} ${${isolated_idx}}
+
+Isolate_One_Node
+ [Documentation] Isolate one node with registered rpc.
+ ClusterManagement.Isolate_Member_From_List_Or_All ${isolated_idx}
+
+Invoke_Rpc_On_Isolated_Node
+ [Documentation] Invoke rpc on isolated node. Because rpc is registered on this node, local constant
+ ... is expected.
+ BuiltIn.Wait_Until_Keyword_Succeeds 3x 2s Verify_Local_Rpc_Invoked ${isolated_idx}
+
+Invoke_Rpc_On_Remaining_Nodes
+ [Documentation] Invoke rpc on non-islolated nodes. As the only instance of rpc remained in the non-isolated
+ ... cluster nodes, only this value is expected.
+ ${index_list} = ClusterManagement.List_Indices_Minus_Member ${isolated_idx} ${all_indices}
+ : FOR ${index} IN @{index_list}
+ \ ${constant} = Verify_Any_Remote_Rpc_Invoked ${index}
+ \ BuiltIn.Should_Not_Be_Equal_As_Strings ${CONSTANT_PREFIX}${isolated_idx} ${constant}
+
+Rejoin_Isolated_Member
+ [Documentation] Rejoin isolated node
+ ClusterManagement.Rejoin_Member_From_List_Or_All ${isolated_idx}
+
+Invoke_Rpc_On_Each_Node_Again
+ [Documentation] Invoke rpc get-constant on every node. When requested on the node with
+ ... local instance the local value is expected. If invoked on the node with no local instance, any remote
+ ... value is expected.
+ : FOR ${index} IN @{all_indices}
+ \ BuiltIn.Run_Keyword_If ${index} in ${INSTALLED_RPC_MEMEBER_IDX_LIST} Verify_Local_Rpc_Invoked ${index}
+ \ BuiltIn.Run_Keyword_Unless ${index} in ${INSTALLED_RPC_MEMEBER_IDX_LIST} WaitForFailure.Verify_Keyword_Does_Not_Fail_Within_Timeout 20s 3s Verify_Any_Remote_Rpc_Invoked
+ \ ... ${index}
+
+Isolate_Member_Without_Registered_Rpc
+ [Documentation] Isolate one node with unregistered rpc.
+ ClusterManagement.Isolate_Member_From_List_Or_All ${TESTED_MEMBER_WITHOUT_RPC_IDX}
+
+Verify_Rpc_Fails_On_Isolated_Member_Without_Rpc
+ [Documentation] Rpc should fail as it is requested on isolated node without rpc instance.
+ BuiltIn.Wait_Until_Keyword_Succeeds 15s 2s MdsalLowlevel.Get_Constant ${TESTED_MEMBER_WITHOUT_RPC_IDX} explicit_status_codes=${NON_WORKING_RPC_STATUS_CODE}
+
+Rejoin_Isolated_Member_Without_Registered_Rpc
+ [Documentation] Rejoin isolated node.
+ ClusterManagement.Rejoin_Member_From_List_Or_All ${TESTED_MEMBER_WITHOUT_RPC_IDX}
+
+Verify_Rpc_Again_Passes_On_Member_Without_Rpc
+ [Documentation] Verify rpc works after the node rejoin.
+ ${constant} = BuiltIn.Wait_Until_Keyword_Succeeds 10x 3s MdsalLowlevel.Get_Constant ${TESTED_MEMBER_WITHOUT_RPC_IDX}
+ Collections.List_Should_Contain_Value ${possible_constants} ${constant}
+
+Unregister_Rpc_On_Each_Node
+ [Documentation] Inregister rpc on both nodes.
+ : FOR ${index} IN @{INSTALLED_RPC_MEMEBER_IDX_LIST}
+ \ MdsalLowlevel.Unregister_Constant ${index}
+
+*** Keywords ***
+Setup_Kw
+ [Documentation] Setup keyword. Create ${possible_constants} list with possible variables of remote constants.
+ SetupUtils.Setup_Utils_For_Setup_And_Teardown
+ ${all_indices} = ClusterManagement.List_All_Indices
+ BuiltIn.Set_Suite_Variable ${all_indices}
+ ${non_installed_rpc_member_idx_list} = ClusterManagement.List_All_Indices
+ ${possible_constants} = BuiltIn.Create_List
+ : FOR ${index} IN @{INSTALLED_RPC_MEMEBER_IDX_LIST}
+ \ Collections.Append_To_List ${possible_constants} ${CONSTANT_PREFIX}${index}
+ \ ${non_installed_rpc_member_idx_list} = ClusterManagement.List_Indices_Minus_Member ${index} ${non_installed_rpc_member_idx_list}
+ BuiltIn.Set_Suite_Variable ${possible_constants}
+ BuiltIn.Set_Suite_Variable ${non_installed_rpc_member_idx_list}
+
+Verify_Local_Rpc_Invoked
+ [Arguments] ${member_index}
+ [Documentation] Verify that local constant is received.
+ ${constant} = MdsalLowlevel.Get_Constant ${member_index}
+ BuiltIn.Should_Be_Equal_As_Strings ${CONSTANT_PREFIX}${member_index} ${constant}
+ BuiltIn.Return_From_Keyword ${constant}
+
+Verify_Any_Remote_Rpc_Invoked
+ [Arguments] ${member_index}
+ [Documentation] Verify that any valid constant is received.
+ ${constant} = MdsalLowlevel.Get_Constant ${member_index}
+ Collections.List_Should_Contain_Value ${possible_constants} ${constant}
+ BuiltIn.Return_From_Keyword ${constant}
--- /dev/null
+*** Settings ***
+Documentation DOMRpcBroker testing: RPC Provider Precedence
+...
+... Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.
+...
+... This program and the accompanying materials are made available under the
+... terms of the Eclipse Public License v1.0 which accompanies this distribution,
+... and is available at http://www.eclipse.org/legal/epl-v10.html
+...
+... The aim is to establish that remote RPC implementations have lower priority
+... than local ones, which is to say that any movement of RPCs on remote nodes
+... does not affect routing as long as a local implementation is available.
+Suite Setup Setup_Keyword
+Suite Teardown SSHLibrary.Close_All_Connections
+Test Setup SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+Test Teardown SetupUtils.Teardown_Test_Show_Bugs_If_Test_Failed
+Default Tags critical
+Library Collections
+Library SSHLibrary
+Resource ${CURDIR}/../../../libraries/ClusterManagement.robot
+Resource ${CURDIR}/../../../libraries/MdsalLowlevel.robot
+Resource ${CURDIR}/../../../libraries/SetupUtils.robot
+Resource ${CURDIR}/../../../libraries/TemplatedRequests.robot
+
+*** Variables ***
+${UNREGISTERED_RPC_NODE} ${1}
+${CONSTANT_PREFIX} constant-
+
+*** Test Cases ***
+Register_Rpc_On_Each_Node
+ [Documentation] Register global rpc on each node of the cluster.
+ : FOR ${index} IN @{full_cluster_index_list}
+ \ MdsalLowlevel.Register_Constant ${index} ${CONSTANT_PREFIX}${index}
+
+Invoke_Rpc_On_Each_Node
+ [Documentation] Verify that the rpc response comes from the local node.
+ : FOR ${index} IN @{full_cluster_index_list}
+ \ ${constant} = MdsalLowlevel.Get_Constant ${index}
+ \ BuiltIn.Should_Be_Equal_As_Strings ${CONSTANT_PREFIX}${index} ${constant}
+
+Unregister_Rpc_On_Node
+ [Documentation] Unregister the rpc on one of the cluster nodes.
+ MdsalLowlevel.Unregister_Constant ${UNREGISTERED_RPC_NODE}
+
+Invoke_Rpc_On_Node_With_Unregistered_Rpc
+ [Documentation] Invoke rcp on the node with unregistered rpc. The response is expected
+ ... to come from other nodes where the rpc remained registered.
+ ${constant} = MdsalLowlevel.Get_Constant ${UNREGISTERED_RPC_NODE}
+ Collections.List_Should_Contain_Value ${allowed_values} ${constant}
+
+Invoke_Rpc_On_Remaining_Nodes
+ [Documentation] Verify that the rpc response comes from the local node.
+ : FOR ${index} IN @{allowed_index_list}
+ \ ${constant} = MdsalLowlevel.Get_Constant ${index}
+ \ BuiltIn.Should_Be_Equal_As_Strings ${CONSTANT_PREFIX}${index} ${constant}
+
+Reregister_Rpc_On_Node
+ [Documentation] Reregister the rpc.
+ MdsalLowlevel.Register_Constant ${UNREGISTERED_RPC_NODE} ${CONSTANT_PREFIX}${UNREGISTERED_RPC_NODE}
+
+Invoke_Rpc_On_Each_Node_Again
+ [Documentation] Verify that the rpc response comes from the local node.
+ : FOR ${index} IN @{full_cluster_index_list}
+ \ ${constant} = MdsalLowlevel.Get_Constant ${index}
+ \ BuiltIn.Should_Be_Equal_As_Strings ${CONSTANT_PREFIX}${index} ${constant}
+
+Unregister_Rpc_On_Each_Node
+ [Documentation] Unregister rpc on every node.
+ : FOR ${index} IN @{full_cluster_index_list}
+ \ MdsalLowlevel.Unregister_Constant ${index}
+
+*** Keywords ***
+Setup_Keyword
+ [Documentation] Create a list of possible constant responses on the node with unregistered rpc.
+ SetupUtils.Setup_Utils_For_Setup_And_Teardown
+ ${full_cluster_index_list} = ClusterManagement.List_All_Indices
+ BuiltIn.Set_Suite_Variable ${full_cluster_index_list}
+ ${allowed_values} = BuiltIn.Create_List
+ ${allowed_index_list} = ClusterManagement.List_Indices_Minus_Member ${UNREGISTERED_RPC_NODE} ${full_cluster_index_list}
+ : FOR ${index} IN @{allowed_index_list}
+ \ Collections.Append_To_List ${allowed_values} ${CONSTANT_PREFIX}${index}
+ BuiltIn.Set_Suite_Variable ${allowed_index_list}
+ BuiltIn.Set_Suite_Variable ${allowed_values}
integration/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot
integration/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot
integration/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot
+integration/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot
+integration/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot
--- /dev/null
+<input xmlns="tag:opendaylight.org,2017:controller:yang:lowlevel:target">
+</input>
-<input>
+<input xmlns="tag:opendaylight.org,2017:controller:yang:lowlevel:control">
<constant>$CONSTANT</constant>
</input>
--- /dev/null
+<input xmlns="tag:opendaylight.org,2017:controller:yang:lowlevel:control">
+</input>
Code Review / integration / test.git / commitdiff