Add test suite for MD-SAL netty-replicate functionality.
Jira: MDSAL-648
Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
Change-Id: I10f2f06cd32fb686b13a12d13bde50a7f6d66db8
--- /dev/null
+*** Settings ***
+Documentation Netty replication library. This library can be used to setup and teardown netty replication connections.
+Resource KarafKeywords.robot
+Resource ClusterManagement.robot
+
+*** Variables ***
+${DEFAULT_NETTY_SOURCE_NODE_INDEX} ${1}
+@{DEFAULT_NETTY_SINK_NODE_INDEXES} ${2}
+
+*** Keywords ***
+Setup_Netty_Replication
+ [Arguments] ${source_memeber_index}=${DEFAULT_NETTY_SOURCE_NODE_INDEX} ${sink_members_indexes}=${DEFAULT_NETTY_SINK_NODE_INDEXES}
+ [Documentation] Set up netty replication connections betwean source and sinks.
+ Open_Source_Connection ${source_memeber_index}
+ FOR ${sink_member_index} IN @{sink_members_indexes}
+ Open_Sink_Connection ${sink_member_index} source_memeber_index=${source_memeber_index}
+ END
+
+Teardown_Netty_Replication
+ [Arguments] ${source_memeber_index}=${DEFAULT_NETTY_SOURCE_NODE_INDEX} ${sink_members_indexes}=${DEFAULT_NETTY_SINK_NODE_INDEXES}
+ [Documentation] Tear down netty replication connections betwean source and sinks.
+ Close_Source_Connection ${source_memeber_index}
+ FOR ${sink_member_index} IN @{sink_members_indexes}
+ Close_Sink_Connection ${sink_member_index}
+ END
+
+Open_Source_Connection
+ [Arguments] ${source_memeber_index}=${DEFAULT_NETTY_SOURCE_NODE_INDEX}
+ [Documentation] Open source part of netty replicate connection on specific node.
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:edit org.opendaylight.mdsal.replicate.netty.source member_index=${source_memeber_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:property-set enabled true member_index=${source_memeber_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:update member_index=${source_memeber_index}
+
+Close_Source_Connection
+ [Arguments] ${source_memeber_index}=${DEFAULT_NETTY_SOURCE_NODE_INDEX}
+ [Documentation] Close source part of netty replicate connection on specific node.
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:edit org.opendaylight.mdsal.replicate.netty.source member_index=${source_memeber_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:property-set enabled false member_index=${source_memeber_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:update member_index=${source_memeber_index}
+
+Open_Sink_Connection
+ [Arguments] ${sink_member_index}=@{DEFAULT_NETTY_SINK_NODE_INDEXES}[0] ${source_memeber_index}=${DEFAULT_NETTY_SOURCE_NODE_INDEX}
+ [Documentation] Open sink part of netty replicate connection on specific node.
+ ${replicate_source_ip}= ClusterManagement.Resolve_Ip_Address_For_Member ${source_memeber_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:edit org.opendaylight.mdsal.replicate.netty.sink member_index=${sink_member_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:property-set enabled true member_index=${sink_member_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:property-set source-host ${replicate_source_ip} member_index=${sink_member_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:update member_index=${sink_member_index}
+
+Close_Sink_Connection
+ [Arguments] ${sink_member_index}=@{DEFAULT_NETTY_SINK_NODE_INDEXES}[0]
+ [Documentation] Close sink part of netty replicate connection on specific node.
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:edit org.opendaylight.mdsal.replicate.netty.sink member_index=${sink_member_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:property-set enabled false member_index=${sink_member_index}
+ KarafKeywords.Execute_Controller_Karaf_Command_On_Background config:update member_index=${sink_member_index}
Get_As_Json_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
- ... ${http_timeout}=${EMPTY} ${log_response}=True
+ ... ${http_timeout}=${EMPTY} ${log_response}=True ${iter_j_offset}=0
[Documentation] Add arguments sensible for JSON data, return Get_Templated response text.
... Optionally, verification against JSON data (may be iterated) is called.
... Only subset of JSON data is verified and returned if JMES path is specified in
${response_text} = Get_Templated folder=${folder} mapping=${mapping} accept=${ACCEPT_EMPTY} session=${session} normalize_json=True
... http_timeout=${http_timeout} log_response=${log_response}
BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Json_Templated response=${response_text} folder=${folder} base_name=data mapping=${mapping}
- ... iterations=${iterations} iter_start=${iter_start}
+ ... iterations=${iterations} iter_start=${iter_start} iter_j_offset=${iter_j_offset}
[Return] ${response_text}
Get_As_Xml_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
- ... ${http_timeout}=${EMPTY}
+ ... ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Add arguments sensible for XML data, return Get_Templated response text.
... Optionally, verification against XML data (may be iterated) is called.
${response_text} = Get_Templated folder=${folder} mapping=${mapping} accept=${ACCEPT_XML} session=${session} normalize_json=False
... http_timeout=${http_timeout}
BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Xml_Templated response=${response_text} folder=${folder} base_name=data mapping=${mapping}
- ... iterations=${iterations} iter_start=${iter_start}
+ ... iterations=${iterations} iter_start=${iter_start} iter_j_offset=${iter_j_offset}
[Return] ${response_text}
Put_As_Json_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
- ... ${http_timeout}=${EMPTY}
+ ... ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Add arguments sensible for JSON data, return Put_Templated response text.
... Optionally, verification against response.json (no iteration) is called.
... Only subset of JSON data is verified and returned if JMES path is specified in
... file ${folder}${/}jmespath.expr.
${response_text} = Put_Templated folder=${folder} base_name=data extension=json accept=${ACCEPT_EMPTY} content_type=${HEADERS_YANG_JSON}
... mapping=${mapping} session=${session} normalize_json=True endline=${\n} iterations=${iterations} iter_start=${iter_start}
- ... http_timeout=${http_timeout}
- BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Json_Templated response=${response_text} folder=${folder} base_name=response mapping=${mapping}
+ ... http_timeout=${http_timeout} iter_j_offset=${iter_j_offset}
+ BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Json_Templated response=${response_text} folder=${folder}
+ ... base_name=response mapping=${mapping} iter_j_offset=${iter_j_offset}
[Return] ${response_text}
Put_As_Xml_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
- ... ${http_timeout}=${EMPTY}
+ ... ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Add arguments sensible for XML data, return Put_Templated response text.
... Optionally, verification against response.xml (no iteration) is called.
# In case of iterations, we use endlines in data to send, as it should not matter and it is more readable.
${response_text} = Put_Templated folder=${folder} base_name=data extension=xml accept=${ACCEPT_XML} content_type=${HEADERS_XML}
... mapping=${mapping} session=${session} normalize_json=False endline=${\n} iterations=${iterations} iter_start=${iter_start}
- ... http_timeout=${http_timeout}
- BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Xml_Templated response=${response_text} folder=${folder} base_name=response mapping=${mapping}
+ ... http_timeout=${http_timeout} iter_j_offset=${iter_j_offset}
+ BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Xml_Templated response=${response_text} folder=${folder}
+ ... base_name=response mapping=${mapping} iter_j_offset=${iter_j_offset}
[Return] ${response_text}
Post_As_Json_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
- ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY}
+ ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Add arguments sensible for JSON data, return Post_Templated response text.
... Optionally, verification against response.json (no iteration) is called.
... Only subset of JSON data is verified and returned if JMES path is specified in
${response_text} = Post_Templated folder=${folder} base_name=data extension=json accept=${ACCEPT_EMPTY} content_type=${HEADERS_YANG_JSON}
... mapping=${mapping} session=${session} normalize_json=True endline=${\n} iterations=${iterations} iter_start=${iter_start}
... additional_allowed_status_codes=${additional_allowed_status_codes} explicit_status_codes=${explicit_status_codes} http_timeout=${http_timeout}
- BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Json_Templated response=${response_text} folder=${folder} base_name=response mapping=${mapping}
+ ... iter_j_offset=${iter_j_offset}
+ BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Json_Templated response=${response_text} folder=${folder}
+ ... base_name=response mapping=${mapping} iter_j_offset=${iter_j_offset}
[Return] ${response_text}
Post_As_Json_Rfc8040_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
- ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY}
+ ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Add arguments sensible for JSON data, return Post_Templated response text.
... Optionally, verification against response.json (no iteration) is called.
... Only subset of JSON data is verified and returned if JMES path is specified in
${response_text} = Post_Templated folder=${folder} base_name=data extension=json accept=${ACCEPT_EMPTY} content_type=${HEADERS_YANG_RFC8040_JSON}
... mapping=${mapping} session=${session} normalize_json=True endline=${\n} iterations=${iterations} iter_start=${iter_start}
... additional_allowed_status_codes=${additional_allowed_status_codes} explicit_status_codes=${explicit_status_codes} http_timeout=${http_timeout}
+ ... iter_j_offset=${iter_j_offset}
BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Json_Templated response=${response_text} folder=${folder} base_name=response mapping=${mapping}
+ ... iter_j_offset=${iter_j_offset}
[Return] ${response_text}
Post_As_Xml_Templated
[Arguments] ${folder} ${mapping}={} ${session}=default ${verify}=False ${iterations}=${EMPTY} ${iter_start}=1
- ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY}
+ ... ${additional_allowed_status_codes}=${NO_STATUS_CODES} ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Add arguments sensible for XML data, return Post_Templated response text.
... Optionally, verification against response.xml (no iteration) is called.
# In case of iterations, we use endlines in data to send, as it should not matter and it is more readable.
${response_text} = Post_Templated folder=${folder} base_name=data extension=xml accept=${ACCEPT_XML} content_type=${HEADERS_XML}
... mapping=${mapping} session=${session} normalize_json=False endline=${\n} iterations=${iterations} iter_start=${iter_start}
... additional_allowed_status_codes=${additional_allowed_status_codes} explicit_status_codes=${explicit_status_codes} http_timeout=${http_timeout}
+ ... iter_j_offset=${iter_j_offset}
BuiltIn.Run_Keyword_If ${verify} Verify_Response_As_Xml_Templated response=${response_text} folder=${folder} base_name=response mapping=${mapping}
+ ... iter_j_offset=${iter_j_offset}
[Return] ${response_text}
Delete_Templated
[Return] ${response_text}
Verify_Response_As_Json_Templated
- [Arguments] ${response} ${folder} ${base_name}=response ${mapping}={} ${iterations}=${EMPTY} ${iter_start}=1
+ [Arguments] ${response} ${folder} ${base_name}=response ${mapping}={} ${iterations}=${EMPTY} ${iter_start}=1 ${iter_j_offset}=0
[Documentation] Resolve expected JSON data, should be equal to provided \${response}.
... JSON normalization is used, endlines enabled for readability.
Verify_Response_Templated response=${response} folder=${folder} base_name=${base_name} extension=json mapping=${mapping} normalize_json=True
- ... endline=${\n} iterations=${iterations} iter_start=${iter_start}
+ ... endline=${\n} iterations=${iterations} iter_start=${iter_start} iter_j_offset=${iter_j_offset}
Verify_Response_As_Xml_Templated
- [Arguments] ${response} ${folder} ${base_name}=response ${mapping}={} ${iterations}=${EMPTY} ${iter_start}=1
+ [Arguments] ${response} ${folder} ${base_name}=response ${mapping}={} ${iterations}=${EMPTY} ${iter_start}=1 ${iter_j_offset}=0
[Documentation] Resolve expected XML data, should be equal to provided \${response}.
... Endline set to empty, as this Resource does not support indented XML comparison.
Verify_Response_Templated response=${response} folder=${folder} base_name=${base_name} extension=xml mapping=${mapping} normalize_json=False
- ... endline=${EMPTY} iterations=${iterations} iter_start=${iter_start}
+ ... endline=${EMPTY} iterations=${iterations} iter_start=${iter_start} iter_j_offset=${iter_j_offset}
Get_As_Json_From_Uri
[Arguments] ${uri} ${session}=default ${http_timeout}=${EMPTY} ${log_response}=True
Put_Templated
[Arguments] ${folder} ${base_name} ${extension} ${content_type} ${accept} ${mapping}={}
- ... ${session}=default ${normalize_json}=False ${endline}=${\n} ${iterations}=${EMPTY} ${iter_start}=1 ${http_timeout}=${EMPTY}
+ ... ${session}=default ${normalize_json}=False ${endline}=${\n} ${iterations}=${EMPTY} ${iter_start}=1 ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Resolve URI and data from folder, call Put_To_Uri, return response text.
${uri} = Resolve_Text_From_Template_Folder folder=${folder} base_name=location extension=uri mapping=${mapping}
${data} = Resolve_Text_From_Template_Folder folder=${folder} base_name=${base_name} extension=${extension} mapping=${mapping} endline=${endline}
- ... iterations=${iterations} iter_start=${iter_start}
+ ... iterations=${iterations} iter_start=${iter_start} iter_j_offset=${iter_j_offset}
${jmes_expression} = Resolve_Jmes_Path ${folder}
${response_text} = Put_To_Uri uri=${uri} data=${data} content_type=${content_type} accept=${accept} session=${session}
... http_timeout=${http_timeout} normalize_json=${normalize_json} jmes_path=${jmes_expression}
Post_Templated
[Arguments] ${folder} ${base_name} ${extension} ${content_type} ${accept} ${mapping}={}
... ${session}=default ${normalize_json}=False ${endline}=${\n} ${iterations}=${EMPTY} ${iter_start}=1 ${additional_allowed_status_codes}=${NO_STATUS_CODES}
- ... ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY}
+ ... ${explicit_status_codes}=${NO_STATUS_CODES} ${http_timeout}=${EMPTY} ${iter_j_offset}=0
[Documentation] Resolve URI and data from folder, call Post_To_Uri, return response text.
${uri} = Resolve_Text_From_Template_Folder folder=${folder} base_name=location extension=uri mapping=${mapping}
${data} = Resolve_Text_From_Template_Folder folder=${folder} name_prefix=post_ base_name=${base_name} extension=${extension} mapping=${mapping}
- ... endline=${endline} iterations=${iterations} iter_start=${iter_start}
+ ... endline=${endline} iterations=${iterations} iter_start=${iter_start} iter_j_offset=${iter_j_offset}
${jmes_expression} = Resolve_Jmes_Path ${folder}
${response_text} = Post_To_Uri uri=${uri} data=${data} content_type=${content_type} accept=${accept} session=${session}
... jmes_path=${jmes_expression} normalize_json=${normalize_json} additional_allowed_status_codes=${additional_allowed_status_codes} explicit_status_codes=${explicit_status_codes} http_timeout=${http_timeout}
Verify_Response_Templated
[Arguments] ${response} ${folder} ${base_name} ${extension} ${mapping}={} ${normalize_json}=False
- ... ${endline}=${\n} ${iterations}=${EMPTY} ${iter_start}=1
+ ... ${endline}=${\n} ${iterations}=${EMPTY} ${iter_start}=1 ${iter_j_offset}=0
[Documentation] Resolve expected text from template, provided response shuld be equal.
... If \${normalize_json}, perform normalization before comparison.
# TODO: Support for XML-aware comparison could be added, but there are issues with namespaces and similar.
${expected_text} = Resolve_Text_From_Template_Folder folder=${folder} base_name=${base_name} extension=${extension} mapping=${mapping} endline=${endline}
- ... iterations=${iterations} iter_start=${iter_start}
+ ... iterations=${iterations} iter_start=${iter_start} iter_j_offset=${iter_j_offset}
BuiltIn.Run_Keyword_And_Return_If """${expected_text}""" == """${EMPTY}""" BuiltIn.Should_Be_Equal ${EMPTY} ${response}
BuiltIn.Run_Keyword_If ${normalize_json} Normalize_Jsons_And_Compare expected_raw=${expected_text} actual_raw=${response}
... ELSE BuiltIn.Should_Be_Equal ${expected_text} ${response}
Resolve_Text_From_Template_Folder
[Arguments] ${folder} ${name_prefix}=${EMPTY} ${base_name}=data ${extension}=json ${mapping}={} ${iterations}=${EMPTY}
- ... ${iter_start}=1 ${endline}=${\n}
+ ... ${iter_start}=1 ${iter_j_offset}=0 ${endline}=${\n}
[Documentation] Read a template from folder, strip endline, make changes according to mapping, return the result.
... If \${iterations} value is present, put text together from "prolog", "item" and "epilog" parts,
... where additional template variable ${i} goes from ${iter_start}, by one ${iterations} times.
+ ... Template variable ${j} is calculated as ${i} incremented by offset ${iter_j_offset} ( j = i + iter_j_offset )
+ ... used to create non uniform data in order to be able to validate UPDATE operations.
... POST (as opposed to PUT) needs slightly different data, \${name_prefix} may be used to distinguish.
... (Actually, it is GET who formats data differently when URI is a top-level container.)
BuiltIn.Run_Keyword_And_Return_If not "${iterations}" Resolve_Text_From_Template_File folder=${folder} file_name=${name_prefix}${base_name}.${extension} mapping=${mapping}
${separator} = BuiltIn.Set_Variable_If '${extension}' != 'json' ${endline} ,${endline}
FOR ${iteration} IN RANGE ${iter_start} ${iterations}+${iter_start}
BuiltIn.Run_Keyword_If ${iteration} > ${iter_start} Collections.Append_To_List ${items} ${separator}
- ${item} = BuiltIn.Evaluate string.Template('''${item_template}''').substitute({"i":"${iteration}"}) modules=string
+ ${j} = BuiltIn.Evaluate ${iteration}+${iter_j_offset}
+ ${item} = BuiltIn.Evaluate string.Template('''${item_template}''').substitute({"i":"${iteration}", "j":${j}}) modules=string
Collections.Append_To_List ${items} ${item}
# TODO: The following makes ugly result for iterations=0. Should we fix that?
END
--- /dev/null
+*** Settings ***
+Documentation Test suite for testing MD-SAL netty replication functionality
+Suite Setup Setup_Suite
+Test Setup Setup_Test
+Test Teardown Teardown_Test
+Default Tags 3node critical netty-replicate
+Library SSHLibrary
+Resource ${CURDIR}/../../../libraries/NettyReplication.robot
+Resource ${CURDIR}/../../../libraries/KarafKeywords.robot
+Resource ${CURDIR}/../../../libraries/ClusterManagement.robot
+Resource ${CURDIR}/../../../libraries/SetupUtils.robot
+Resource ${CURDIR}/../../../libraries/TemplatedRequests.robot
+Resource ${CURDIR}/../../../libraries/CarPeople.robot
+Resource ${CURDIR}/../../../libraries/WaitForFailure.robot
+Resource ${CURDIR}/../../../libraries/Utils.robot
+
+*** Variables ***
+${CARPEOPLE_DEV_FOLDER} ${CURDIR}/../../../variables/carpeople/crud
+${ADDITIONAL_SOURCE_NODE_INDEX} ${3}
+@{MULTIPLE_SINK_NODES_INDEXES} ${2} ${3}
+
+*** Test Cases ***
+Replicate_Config_Addition
+ [Documentation] Adding new configuration on source node shoudl be replicated on sink node.
+ NettyReplication.Setup_Netty_Replication
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${5}
+
+Replicate_Config_Update
+ [Documentation] Updating existing configuration on source node shoudl be replicated on sink node.
+ NettyReplication.Setup_Netty_Replication
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${5} iter_j_offset=${0}
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${5} iter_j_offset=${5}
+
+Replicte_Config_Deletion
+ [Documentation] Updating existing configuration on source node shoudl be replicated on sink node.
+ NettyReplication.Setup_Netty_Replication
+ &{mapping} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${5}
+ Delete_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people
+
+Replicate_Multiple_Changes_to_Config
+ [Documentation] CRUD configuration changes done on source node shoudl be replicated on sink node.
+ NettyReplication.Setup_Netty_Replication
+ &{mapping_1} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${5} iter_j_offset=${0}
+ &{mapping_2} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${7} iter_j_offset=${0}
+ &{mapping_2_updated} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${7} iter_j_offset=${2}
+ Delete_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people
+
+Replicate_Multiple_Changes_to_Multiple_Sinks
+ [Documentation] CRUD configuration changes done on source node should be replicated on multiple sink nodes.
+ NettyReplication.Setup_Netty_Replication source_memeber_index=${DEFAULT_NETTY_SOURCE_NODE_INDEX} sink_members_indexes=@{MULTIPLE_SINK_NODES_INDEXES}
+ &{mapping_1} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people sink_node_indexes=@{MULTIPLE_SINK_NODES_INDEXES} iterations=${5} iter_j_offset=${0}
+ &{mapping_2} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people sink_node_indexes=@{MULTIPLE_SINK_NODES_INDEXES} iterations=${7} iter_j_offset=${0}
+ &{mapping_2_updated} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people sink_node_indexes=@{MULTIPLE_SINK_NODES_INDEXES} iterations=${8} iter_j_offset=${2}
+ Delete_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people
+
+Sink_Catch_Up_To_Changes_After_Opening_Connection
+ [Documentation] After opening sink part of connection, sink should catch up to changes made on the source node, which were made before opening connection.
+ NettyReplication.Open_Source_Connection
+ &{mapping} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people sink_node_indexes=@{EMPTY} iterations=${5}
+ ${netty_sink_session_alias} = Resolve_Http_Session_For_Member member_index=@{DEFAULT_NETTY_SINK_NODE_INDEXES}[0]
+ Verify_Config_Is_Not_Present ${CARPEOPLE_DEV_FOLDER}/people session=${netty_sink_session_alias} wait_time=5s
+ NettyReplication.Open_Sink_Connection
+ Verify_Config_Is_Present ${CARPEOPLE_DEV_FOLDER}/people session=${netty_sink_session_alias} iterations=${5}
+
+Reconnect_After_Lost_Connection
+ [Documentation] Test if sink sucessfuly reconnects after lost connection and if changes made during lost connection are present on reconnected sink's datastore.
+ NettyReplication.Setup_Netty_Replication
+ ClusterManagement.Isolate_Member_From_List_Or_All @{DEFAULT_NETTY_SINK_NODE_INDEXES}[0]
+ &{mapping} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people sink_node_indexes=@{EMPTY} iterations=${5}
+ ${netty_sink_session_alias} = Resolve_Http_Session_For_Member member_index=@{DEFAULT_NETTY_SINK_NODE_INDEXES}[0]
+ Verify_Config_Is_Not_Present ${CARPEOPLE_DEV_FOLDER}/people session=${netty_sink_session_alias} wait_time=5s
+ ClusterManagement.Rejoin_Member_From_List_Or_All @{DEFAULT_NETTY_SINK_NODE_INDEXES}[0]
+ Verify_Config_Is_Present ${CARPEOPLE_DEV_FOLDER}/people session=${netty_sink_session_alias} iterations=${5}
+ [Teardown] Teardown_Isolation_Test
+
+Change_Replication_Source
+ [Documentation] After changing replication source ODL to another ODL, sink should reflect datastore from newer source, forgeting changes from the old datastore.
+ NettyReplication.Setup_Netty_Replication
+ &{mapping_older} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/people iterations=${5}
+ NettyReplication.Teardown_Netty_Replication
+ # switch source node 1 -> 3
+ NettyReplication.Setup_Netty_Replication source_memeber_index=${ADDITIONAL_SOURCE_NODE_INDEX} sink_members_indexes=${DEFAULT_NETTY_SINK_NODE_INDEXES}
+ &{mapping_newer} = BuiltIn.Create_Dictionary
+ Put_Config_And_Verify ${CARPEOPLE_DEV_FOLDER}/cars source_node_index=${ADDITIONAL_SOURCE_NODE_INDEX} iterations=${8}
+ # check if data from old source was forgotten (removed)
+ ${netty_sink_session_alias} = Resolve_Http_Session_For_Member member_index=@{DEFAULT_NETTY_SINK_NODE_INDEXES}[0]
+ Verify_Config_Is_Not_Present ${CARPEOPLE_DEV_FOLDER}/people session=${netty_sink_session_alias} removal=True
+
+*** Keywords ***
+Setup_suite
+ [Documentation] Open karaf connections to each ODL and deconfigure cluster to prevent interferance of shard replication with netty replication.
+ KarafKeywords.Setup_Karaf_Keywords
+ # deconfigure cluster
+ ${members_index_list} = List_Indices_Or_All
+ FOR ${cluster_member_index} IN @{members_index_list}
+ ${member_ip_address} = ClusterManagement.Resolve_Ip_Address_For_Member ${cluster_member_index}
+ # backup old cluster configuration files
+ Run_Bash_Command_On_Member command=pushd ${karaf_home} && tar -cvf /tmp/config_backup.tar ./configuration/initial/ && popd member_index=${cluster_member_index}
+ Run_Bash_Command_On_Member command=pushd ${karaf_home} && ./bin/configure_cluster.sh 1 ${member_ip_address} && popd member_index=${cluster_member_index}
+ END
+
+Setup_Test
+ [Documentation] Clear data and set loging.
+ Clear_Data_On_All_Nodes_And_Restart
+ SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
+
+Teardown_Test
+ [Documentation] Close all netty replication connections and show linked bugs in case of failure.
+ FOR ${sink_member_index} IN @{ClusterManagement__member_index_list}
+ NettyReplication.Close_Source_Connection ${sink_member_index}
+ NettyReplication.Close_Sink_Connection ${sink_member_index}
+ END
+ SetupUtils.Teardown_Test_Show_Bugs_If_Test_Failed
+
+Teardown_Isolation_Test
+ [Documentation] Tear down test when isolation is used. Additionally to traditional teardown resets iptables rules set during isolation.
+ FOR ${member_index} IN @{ClusterManagement__member_index_list}
+ ClusterManagement.Rejoin_Member_From_List_Or_All ${member_index}
+ END
+ Teardown_Test
+
+Clear_Data_On_All_Nodes_And_Restart
+ [Documentation] Clear data after stoping all members and then restart all members.
+ ClusterManagement.Stop_Members_From_List_Or_All
+ ClusterManagement.Clean_Journals_Data_And_Snapshots_On_List_Or_All
+ ClusterManagement.Start_Members_From_List_Or_All
+
+Put_Config_And_Verify
+ [Arguments] ${template_folder} ${mapping}={} ${source_node_index}=${DEFAULT_NETTY_SOURCE_NODE_INDEX} @{sink_node_indexes}=${DEFAULT_NETTY_SINK_NODE_INDEXES}
+ ... ${iterations}=${1} ${iter_j_offset}=${0}
+ [Documentation] Request put config on netty replicate source and verify changes has been made on both source and sinks.
+ ${netty_source_session_alias} = Resolve_Http_Session_For_Member member_index=${source_node_index}
+ TemplatedRequests.Put_As_Json_Templated ${template_folder} session=${netty_source_session_alias} iterations=${iterations} iter_j_offset=${iter_j_offset}
+ Verify_Config_Is_Present template_folder=${template_folder} mapping=${mapping} session=${netty_source_session_alias} iterations=${iterations} iter_j_offset=${iter_j_offset}
+ FOR ${sink_node_index} IN @{sink_node_indexes}
+ ${netty_sink_session_alias} = Resolve_Http_Session_For_Member member_index=${sink_node_index}
+ Verify_Config_Is_Present template_folder=${template_folder} mapping=${mapping} session=${netty_sink_session_alias} iterations=${iterations} iter_j_offset=${iter_j_offset}
+ END
+
+Verify_Config_Is_Present
+ [Arguments] ${template_folder} ${session} ${mapping}={} ${iterations}=${1} ${iter_j_offset}=${0}
+ [Documentation] Verify config is present on target node datastore by using templated request.
+ BuiltIn.Should_Not_Be_Empty ${session} Could not verify, session to node is not opened
+ BuiltIn.Wait_Until_Keyword_Succeeds 10x 3s TemplatedRequests.Get_As_Json_Templated ${template_folder}
+ ... verify=True session=${session} iterations=${iterations} iter_j_offset=${iter_j_offset}
+
+Delete_Config_And_Verify
+ [Arguments] ${template_folder} ${mapping}={} ${source_node_index}=${DEFAULT_NETTY_SOURCE_NODE_INDEX} @{sink_node_indexes}=${DEFAULT_NETTY_SINK_NODE_INDEXES}
+ ... ${iterations}=${1}
+ [Documentation] Request delete config on netty replicate source and verify changes has been made on both source and sink.
+ ${netty_source_session_alias} = Resolve_Http_Session_For_Member member_index=${source_node_index}
+ TemplatedRequests.Delete_Templated ${template_folder} session=${netty_source_session_alias}
+ Verify_Config_Is_Not_Present template_folder=${template_folder} mapping=${mapping} session=${netty_source_session_alias} removal=True
+ FOR ${sink_node_index} IN @{sink_node_indexes}
+ ${netty_sink_session_alias} = Resolve_Http_Session_For_Member member_index=${sink_node_index}
+ Verify_Config_Is_Not_Present template_folder=${template_folder} mapping=${mapping} session=${netty_sink_session_alias} removal=True
+ END
+
+Verify_Config_Is_Not_Present
+ [Arguments] ${template_folder} ${session} ${mapping}={} ${removal}=False ${wait_time}=0s
+ [Documentation] Verify config is not present on the target node datastore by using templated request. Should get return code 404 or 409.
+ ... removal - Retries until config is not present (for cases of cofig deletion when config migh be present at begining, but disapears later)
+ ... wait_time - Repeatedly checks for specific amount of time if config does not appear (for cases of config addition when config is not present, but might appear after some time)
+ BuiltIn.Should_Not_Be_Empty ${session} Could not verify, session to node is not opened
+ ${uri} = TemplatedRequests.Resolve_Text_From_Template_Folder folder=${template_folder} base_name=location extension=uri mapping=${mapping}
+ BuiltIn.Run_Keyword_If ${removal} Until_Confg_Is_Removed ${session} ${uri}
+ ... ELSE IF "${wait_time}" != "0s" Config_Does_Not_Appear_During_Time ${session} ${uri} ${wait_time}
+ ... ELSE Utils.No_Content_From_URI ${session} ${uri}
+
+Until_Confg_Is_Removed
+ [Arguments] ${session} ${uri}
+ [Documentation] Retry until config is not avaibale
+ BuiltIn.Wait_Until_Keyword_Succeeds 10x 3s Utils.No_Content_From_URI ${session} ${uri}
+
+Config_Does_Not_Appear_During_Time
+ [Arguments] ${session} ${uri} ${wait_time}
+ [Documentation] Repeatedly check if config does not appear during the wait time
+ WaitForFailure.Verify_Keyword_Does_Not_Fail_Within_Timeout ${wait_time} 1s Utils.No_Content_From_URI ${session} ${uri}
--- /dev/null
+# Place the suites in run order:
+integration/test/csit/suites/mdsal/netty_replicate/netty-replicate.robot
+
{
"id": "localhost/cars/car_id_$i",
"category": "car_category_$i",
- "model": "car_model_$i",
+ "model": "car_model_$j",
"manufacturer": "car_manufacturer_$i",
- "year": $i
+ "year": $j
}
{
"id": "localhost/people/person_id_$i",
- "gender": "gender_$i",
- "age": $i,
+ "gender": "gender_$j",
+ "age": $j,
"address": "address_$i",
"contactNo": "contact_$i"
}
Code Review / integration / test.git / commitdiff