csit/suites/controller/cluster_singleton/partition_and_heal.robot

   1 *** Settings ***
   2 Documentation       Cluster Singleton testing: Partition And Heal
   3 ...
   4 ...                 Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.
   5 ...
   6 ...                 This program and the accompanying materials are made available under the
   7 ...                 terms of the Eclipse Public License v1.0 which accompanies this distribution,
   8 ...                 and is available at http://www.eclipse.org/legal/epl-v10.html
   9 ...
  10 ...                 Cluster Singleton service is designed to ensure that only one instance of
  11 ...                 an application is registered globally in the cluster.
  12 ...                 The goal is to establish the service operates correctly in face of node
  13 ...                 failures.
  14
  15 Library             Collections
  16 Library             SSHLibrary
  17 Library             RequestsLibrary
  18 Resource            ${CURDIR}/../../../libraries/controller/CsCommon.robot
  19 Resource            ${CURDIR}/../../../libraries/ClusterManagement.robot
  20 Resource            ${CURDIR}/../../../libraries/MdsalLowlevel.robot
  21 Resource            ${CURDIR}/../../../libraries/SetupUtils.robot
  22 Resource            ${CURDIR}/../../../libraries/WaitForFailure.robot
  23
  24 Suite Setup         Setup_Keyword
  25 Suite Teardown      SSHLibrary.Close_All_Connections
  26 Test Setup          SetupUtils.Setup_Test_With_Logging_And_Without_Fast_Failing
  27 Test Teardown       SetupUtils.Teardown_Test_Show_Bugs_If_Test_Failed
  28
  29 Default Tags        critical
  30
  31
  32 *** Variables ***
  33 ${STABILITY_TIMEOUT_ISOLATED}       120s
  34 ${STABILITY_TIMEOUT_REJOINED}       60s
  35 @{STATUS_ISOLATED}                  ${501}
  36
  37
  38 *** Test Cases ***
  39 Register_Singleton_Constant_On_Each_Node
  40     [Documentation]    Register a candidate application on each node.
  41     CsCommon.Register_Singleton_Constant_On_Nodes    ${cs_all_indices}
  42
  43 Verify_Singleton_Constant_On_Each_Node
  44     [Documentation]    Store the owner and candidates of the application and initially verify that all
  45     ...    odl nodes are used.
  46     ${owner}    ${candidates}=    CsCommon.Get_And_Save_Present_CsOwner_And_CsCandidates    1
  47     BuiltIn.Wait_Until_Keyword_Succeeds
  48     ...    15
  49     ...    2s
  50     ...    CsCommon.Verify_Singleton_Constant_On_Nodes
  51     ...    ${cs_all_indices}
  52     ...    ${CS_CONSTANT_PREFIX}${owner}
  53
  54 Isolate_Owner_Node
  55     [Documentation]    Isolate the cluster node which is the owner. Wait until the new owner is elected and store
  56     ...    new values of owner and candidates.
  57     CsCommon.Isolate_Owner_And_Verify_Isolated
  58
  59 Monitor_Stability_While_Isolated
  60     [Documentation]    Monitor the stability of the singleton application and fail the the owner is changed during the monitoring.
  61     WaitForFailure.Verify_Keyword_Does_Not_Fail_Within_Timeout
  62     ...    ${STABILITY_TIMEOUT_ISOLATED}
  63     ...    3s
  64     ...    CsCommon.Verify_Singleton_Constant_During_Isolation
  65
  66 Rejoin_Isolated_node
  67     [Documentation]    Rejoin isolated node.
  68     CsCommon.Rejoin_Node_And_Verify_Rejoined
  69
  70 Unregister_Singleton_Constant_On_Each_Node
  71     [Documentation]    Unregister the application on every node.
  72     CsCommon.Unregister_Singleton_Constant_On_Nodes    ${cs_all_indices}
  73
  74
  75 *** Keywords ***
  76 Setup_Keyword
  77     [Documentation]    Suite setup.
  78     SetupUtils.Setup_Utils_For_Setup_And_Teardown    http_timeout=125
  79     CsCommon.Cluster_Singleton_Init