...
... This resource holds private state (in suite variables),
... which is generated once at Setup.
-... The state includes list with indexes (numbers enumerating cluster members),
-... IP addresses and Http (RequestsLibrary) sessions.
+... The state includes IP addresses and Http (RequestsLibrary) sessions.
... Most functionality deals with stopping/starting controllers
... and finding leaders/followers for a Shard.
...
: FOR ${index} IN @{index_list}
\ Verify_Karaf_Is_Not_Running_On_Member member_index=${index}
+ClusterManagement__Build_List
+ [Arguments] ${member}
+ ${member_int}= BuiltIn.Convert_To_Integer ${member}
+ ${index_list}= BuiltIn.Create_List ${member_int}
+ [Return] ${index_list}
+
+Kill_Single_Member
+ [Arguments] ${member} ${confirm}=True
+ [Documentation] Convenience keyword that kills the specified member of the cluster.
+ ${index_list}= ClusterManagement__Build_List ${member}
+ Kill_Members_From_List_Or_All ${index_list} ${confirm}
+
Clean_Journals_And_Snapshots_On_List_Or_All
[Arguments] ${member_index_list}=${EMPTY}
[Documentation] Delete journal and snapshots directories on every node listed (or all).
BuiltIn.Wait_Until_Keyword_Succeeds ${timeout} 1s Check_Cluster_Is_In_Sync member_index_list=${member_index_list}
# TODO: Do we also want to check Shard Leaders here?
+Start_Single_Member
+ [Arguments] ${member} ${wait_for_sync}=True ${timeout}=300s
+ [Documentation] Convenience keyword that starts the specified member of the cluster.
+ ${index_list}= ClusterManagement__Build_List ${member}
+ Start_Members_From_List_Or_All ${index_list} ${wait_for_sync} ${timeout}
+
Verify_Leader_Exists_For_Each_Shard
[Arguments] ${shard_name_list} ${shard_type}=operational ${member_index_list}=${EMPTY} ${verify_restconf}=True
[Documentation] For each shard name, call Get_Leader_And_Followers_For_Shard.
Reboot_1
[Documentation] Previous car-people Leader is rebooted (without persistence cleanup).
${index_to_reboot} = Collections.Remove_From_List ${list_to_reboot} 0
- ${index_list} = BuiltIn.Create_List ${index_to_reboot}
- ClusterManagement.Kill_Members_From_List_Or_All member_index_list=${index_list} confirm=True
- ClusterManagement.Start_Members_From_List_Or_All member_index_list=${index_list} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
+ ClusterManagement.Kill_Single_Member ${index_to_reboot} confirm=True
+ ClusterManagement.Start_Single_Member ${index_to_reboot} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
BuiltIn.Wait_Until_Keyword_Succeeds 30s 2s ClusterManagement.Verify_Leader_Exists_For_Each_Shard shard_name_list=${SHARD_NAME_LIST} shard_type=config
Buy_Cars_After_1_Reboots_And_Verify
Reboot_2
[Documentation] Previous car-people Leader is rebooted (without persistence cleanup).
${index_to_reboot} = Collections.Remove_From_List ${list_to_reboot} 0
- ${index_list} = BuiltIn.Create_List ${index_to_reboot}
- ClusterManagement.Kill_Members_From_List_Or_All member_index_list=${index_list} confirm=True
- ClusterManagement.Start_Members_From_List_Or_All member_index_list=${index_list} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
+ ClusterManagement.Kill_Single_Member ${index_to_reboot} confirm=True
+ ClusterManagement.Start_Single_Member ${index_to_reboot} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
BuiltIn.Wait_Until_Keyword_Succeeds 30s 2s ClusterManagement.Verify_Leader_Exists_For_Each_Shard shard_name_list=${SHARD_NAME_LIST} shard_type=config
Buy_Cars_After_2_Reboots_And_Verify
Reboot_3
[Documentation] Previous car-people Leader is rebooted (without persistence cleanup).
${index_to_reboot} = Collections.Remove_From_List ${list_to_reboot} 0
- ${index_list} = BuiltIn.Create_List ${index_to_reboot}
- ClusterManagement.Kill_Members_From_List_Or_All member_index_list=${index_list} confirm=True
- ClusterManagement.Start_Members_From_List_Or_All member_index_list=${index_list} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
+ ClusterManagement.Kill_Single_Member ${index_to_reboot} confirm=True
+ ClusterManagement.Start_Single_Member ${index_to_reboot} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
BuiltIn.Wait_Until_Keyword_Succeeds 30s 2s ClusterManagement.Verify_Leader_Exists_For_Each_Shard shard_name_list=${SHARD_NAME_LIST} shard_type=config
Buy_Cars_After_3_Reboots_And_Verify
Kill_Original_Car_Leader
[Documentation] Kill the car Leader to cause a new leader to get elected.
- ${kill_list} = BuiltIn.Create_List ${car_leader_index}
- ClusterManagement.Kill_Members_From_List_Or_All member_index_list=${kill_list} confirm=True
+ ClusterManagement.Kill_Single_Member ${car_leader_index} confirm=True
Wait_For_New_Leader
[Documentation] Wait until new car Leader is elected.
Start_Old_Car_Leader
[Documentation] Start the killed member without deleting the persisted data.
- ${revive_list} = BuiltIn.Create_List ${car_leader_index}
- ClusterManagement.Start_Members_From_List_Or_All member_index_list=${revive_list} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
+ ClusterManagement.Start_Single_Member ${car_leader_index} wait_for_sync=True timeout=${MEMBER_START_TIMEOUT}
BuiltIn.Wait_Until_Keyword_Succeeds 30s 2s ClusterManagement.Verify_Leader_Exists_For_Each_Shard shard_name_list=${SHARD_NAME_LIST} shard_type=config
See_Folower_Cars_On_Old_Leader