Tolerate timeout when accessing isolated member
[integration/test.git] / csit / suites / controller / Clustering_Datastore / entity_isolate.robot
1 *** Settings ***
2 Documentation     Suite for performing member isolation and rejoin, we do with entity-ownership leader.
3 Suite Setup       Setup
4 Suite Teardown    Teardown
5 Test Setup        SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
6 Default Tags      clustering    carpeople    critical
7 Library           Collections
8 Resource          ${CURDIR}/../../../libraries/CarPeople.robot
9 Resource          ${CURDIR}/../../../libraries/ClusterManagement.robot
10 Resource          ${CURDIR}/../../../libraries/SetupUtils.robot
11 Resource          ${CURDIR}/../../../libraries/TemplatedRequests.robot
12 Variables         ${CURDIR}/../../../variables/Variables.py
13
14 *** Variables ***
15 @{SHARD_OPER_LIST}    car    people    car-people    inventory    topology    default    toaster
16 ...               entity-ownership
17 @{SHARD_CONF_LIST}    car    people    car-people    inventory    topology    default    toaster
18
19 *** Test Cases ***
20 Check All Shards Before Isolate
21     [Documentation]    Check all shards in controller.
22     BuiltIn.Wait_Until_Keyword_Succeeds    30s    2s    ClusterManagement.Verify_Leader_Exists_For_Each_Shard    shard_name_list=${SHARD_OPER_LIST}    shard_type=operational
23     BuiltIn.Wait_Until_Keyword_Succeeds    30s    2s    ClusterManagement.Verify_Leader_Exists_For_Each_Shard    shard_name_list=${SHARD_CONF_LIST}    shard_type=config
24
25 Isolate Entity Leader
26     [Documentation]    Isolate the entity-ownership Leader to cause a new leader to get elected.
27     ClusterManagement.Isolate_Member_From_List_Or_All    ${entity-ownership_leader_index}
28
29 Check All Shards After Isolate
30     [Documentation]    Check all shards in controller.
31     BuiltIn.Wait_Until_Keyword_Succeeds    30s    2s    ClusterManagement.Verify_Leader_Exists_For_Each_Shard    shard_name_list=${SHARD_OPER_LIST}    shard_type=operational    member_index_list=${entity-ownership_follower_indices}
32     BuiltIn.Wait_Until_Keyword_Succeeds    30s    2s    ClusterManagement.Verify_Leader_Exists_For_Each_Shard    shard_name_list=${SHARD_CONF_LIST}    shard_type=config    member_index_list=${entity-ownership_follower_indices}
33
34 Rejoin Entity Leader
35     [Documentation]    Rejoin the entity-ownership Leader.
36     ClusterManagement.Rejoin_Member_From_List_Or_All    ${entity-ownership_leader_index}
37
38 Check All Shards After Rejoin
39     [Documentation]    Check all shards in controller.
40     BuiltIn.Wait_Until_Keyword_Succeeds    30s    2s    ClusterManagement.Verify_Leader_Exists_For_Each_Shard    shard_name_list=${SHARD_OPER_LIST}    shard_type=operational
41     BuiltIn.Wait_Until_Keyword_Succeeds    30s    2s    ClusterManagement.Verify_Leader_Exists_For_Each_Shard    shard_name_list=${SHARD_CONF_LIST}    shard_type=config
42
43 *** Keywords ***
44 Setup
45     [Documentation]    Initialize resources, memorize car shard leader and followers.
46     SetupUtils.Setup_Utils_For_Setup_And_Teardown
47     CarPeople.Set_Variables_For_Shard    shard_name=entity-ownership    shard_type=operational
48
49 Teardown
50     [Documentation]    Clear IPTables in all nodes.
51     ClusterManagement.Flush_Iptables_From_List_Or_All