Moved clustering tests to controller directory.
Replaced ${PORT} with ${RESTCONFPORT} in call to Wait For Controller Up.
This commit is a re-try of the following commit:
https://git.opendaylight.org/gerrit/#/c/23001/
It adds KEYFILE_PASS to variables.py
and then puts KEYFILE_PASS into
Run Command On Remote System.
It also contains the following from the
previous commit attempt:
Change global variables to match those used in CI
All ssh sign-ins now use certificates rather than passwords
Remove Sleeps where possible.
Move repeated variables into variables.py.
Move repeated keywords into libraries.
Change-Id: Iacb328c89abe35d8c57eac4b4d081ff219b2e8b5
Signed-off-by: Phillip Shea <phillip.shea@hp.com>
*** Settings ***
+Resource Utils.txt
*** Variables ***
*** Keywords ***
Get Leader And Verify
[Arguments] ${shard_name} ${old_leader}=${EMPTY}
- ${leader} GetLeader ${shard_name} ${3} ${3} ${1} ${PORT} ${MEMBER1} ${MEMBER2} ${MEMBER3}
+ [Documentation] Returns the IP addr or hostname of the leader of the specified shard.
+ [Documentation] Controllers are specifed in the pybot command line.
+ ${leader} GetLeader ${shard_name} ${3} ${3} ${1} ${PORT} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
Should Not Be Equal As Strings ${leader} None
Run Keyword If '${old_leader}'!='${EMPTY}' Should Not Be Equal ${old_leader} ${leader}
[Return] ${leader}
-Wait For Leader
+Wait For Leader To Be Found
[Arguments] ${shard_name}
+ [Documentation] Waits until the leader of the specified shard is found.
${leader} Wait Until Keyword Succeeds 60s 2s Get Leader And Verify ${shard_name}
Log ${leader}
[Return] ${leader}
Switch Leader
[Arguments] ${shard_name} ${current_leader}
- StopController ${current_leader} ${USERNAME} ${PASSWORD} ${KARAFHOME}
+ [Documentation] Forces a change of leadership by shutting down the current leader.
+ Stop One Or More Controllers ${current_leader}
${new_leader} Wait Until Keyword Succeeds 60s 2s Get Leader And Verify ${shard_name} ${current_leader}
+ # TODO: Future enhanement: make sure the other controller is a follower and not a master or candidate.
Log ${new_leader}
[Return] ${new_leader}
Get All Followers
[Arguments] ${shard_name}
- ${followers} GetFollowers ${shard_name} ${3} ${3} ${1} ${PORT} ${MEMBER1} ${MEMBER2} ${MEMBER3}
+ [Documentation] Returns the IP addresses or hostnames of all followers of the specified shard.
+ ${followers} GetFollowers ${shard_name} ${3} ${3} ${1} ${PORT} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
Log ${followers}
Should Not Be Empty ${followers}
[Return] ${followers}
Add Cars And Verify
+ [Documentation] Initializes shard and then adds the specified number of cars and performs a GET as a check.
[Arguments] ${controller_ip} ${num_cars} ${timeout}=60s
${resp} InitCar ${controller_ip} ${PORT}
Should Be Equal As Strings ${resp.status_code} 204
- ${resp} AddCar ${controller_ip} ${PORT} ${num_cars}
+ ${resp} AddCar ${controller_ip} ${RESTCONFPORT} ${num_cars}
Should Be Equal As Strings ${resp.status_code} 204
Wait Until Keyword Succeeds ${timeout} 2s Get Cars And Verify ${controller_ip} ${num_cars}
+Add Cars And Verify Without Init
+ [Documentation] Adds cars to an initialized cars shard then performs a GET as a check.
+ [Arguments] ${controller_ip} ${num_cars} ${timeout}=60s
+ ${resp} AddCar ${controller_ip} ${RESTCONFPORT} ${num_cars}
+ Should Be Equal As Strings ${resp.status_code} 204
+ Wait Until Keyword Succeeds ${timeout} 2s Get Cars And Verify ${controller_ip} ${num_cars}
+
+
Get Cars And Verify
[Arguments] ${controller_ip} ${num_cars}
- ${resp} Getcars ${controller_ip} ${PORT} ${0}
+ [Documentation] Gets cars and verifies that the manufacturer is correct.
+ # TODO: Future enhanement: verify all fields.
+ ${resp} Getcars ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
:FOR ${i} IN RANGE 1 ${num_cars}+1
\ Should Contain ${resp.content} manufacturer${i}
[Documentation] Note: The first AddPerson call passed with 0 posts directly to the data store to get
[Documentation] the people container created so the subsequent AddPerson RPC calls that put
[Documentation] to the person list will succeed.
- ${resp} AddPerson ${controller_ip} ${PORT} ${0}
+ ${resp} AddPerson ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 204
Wait Until Keyword Succeeds 60s 2s Get One Person And Verify ${controller_ip} ${0}
- ${resp} AddPerson ${controller_ip} ${PORT} ${num_people}
+ ${resp} AddPerson ${controller_ip} ${RESTCONFPORT} ${num_people}
Wait Until Keyword Succeeds 60s 2s Get People And Verify ${controller_ip} ${num_people}
Get One Person And Verify
[Arguments] ${controller_ip} ${number}
- ${resp} GetPersons ${controller_ip} ${PORT} ${0}
+ [Documentation] Gets a person and verifies that the user ID is correct.
+ # TODO: Future enhanement: verify all fields.
+ ${resp} GetPersons ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} user${number}
Get People And Verify
[Arguments] ${controller_ip} ${num_people}
- ${resp} GetPersons ${controller_ip} ${PORT} ${0}
+ [Documentation] Gets multiple people and verifies that the user IDs are correct.
+ # TODO: Future enhanement: verify all fields.
+ ${resp} GetPersons ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
:FOR ${i} IN RANGE 1 ${num_people}+1
\ Should Contain ${resp.content} user${i}
[Documentation] Add a car-person via the data store and get the car-person from Leader.
[Documentation] Note: This is done to get the car-people container created so subsequent BuyCar
[Documentation] RPC puts to the car-person list will succeed.
- AddCarPerson ${controller_ip} ${PORT} ${0}
+ AddCarPerson ${controller_ip} ${RESTCONFPORT} ${0}
Wait Until Keyword Succeeds 60s 2s Get One Car-Person Mapping And Verify ${controller_ip} ${0}
Get One Car-Person Mapping And Verify
[Arguments] ${controller_ip} ${number}
- ${resp} GetCarPersonMappings ${controller_ip} ${PORT} ${0}
+ [Documentation] Gets a car person mapping and verifies that the user ID is correct.
+ ${resp} GetCarPersonMappings ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} user${number}
Get Car-Person Mappings And Verify
[Arguments] ${controller_ip} ${num_entries}
- ${resp} GetCarPersonMappings ${controller_ip} ${PORT} ${0}
+ ${resp} GetCarPersonMappings ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
:FOR ${i} IN RANGE 1 ${num_entries}+1
\ Should Contain ${resp.content} user${i}
Buy Cars And Verify
[Arguments] ${controller_ip} ${num_entries} ${start}=${0}
- Wait Until Keyword Succeeds 60s 2s BuyCar ${controller_ip} ${PORT} ${num_entries} ${start}
+ Wait Until Keyword Succeeds 60s 2s BuyCar ${controller_ip} ${RESTCONFPORT} ${num_entries} ${start}
Check Cars Deleted
[Arguments] ${controller_ip}
- ${resp} Getcars ${controller_ip} ${PORT} ${0}
+ ${resp} Getcars ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 404
Delete All Cars And Verify
[Arguments] ${controller_ip}
- DeleteAllCars ${controller_ip} ${PORT} ${0}
+ DeleteAllCars ${controller_ip} ${RESTCONFPORT} ${0}
Wait Until Keyword Succeeds 60s 2s Check Cars Deleted ${controller_ip}
Check People Deleted
[Arguments] ${controller_ip}
- ${resp} GetPersons ${controller_ip} ${PORT} ${0}
+ ${resp} GetPersons ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 404
Delete All People And Verify
[Arguments] ${controller_ip}
- DeleteAllPersons ${controller_ip} ${PORT} ${0}
+ DeleteAllPersons ${controller_ip} ${RESTCONFPORT} ${0}
Wait Until Keyword Succeeds 60s 2s Check People Deleted ${controller_ip}
Check Cars-Persons Deleted
[Arguments] ${controller_ip}
- ${resp} GetCarPersonMappings ${controller_ip} ${PORT} ${0}
+ ${resp} GetCarPersonMappings ${controller_ip} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 404
Delete All Cars-Persons And Verify
[Arguments] ${controller_ip}
- DeleteAllCarsPersons ${controller_ip} ${PORT} ${0}
+ DeleteAllCarsPersons ${controller_ip} ${RESTCONFPORT} ${0}
Wait Until Keyword Succeeds 60s 2s Check Cars-Persons Deleted ${controller_ip}
[Return] ${value}
Get Process ID Based On Regex On Remote System
- [Documentation] Uses ps to find a process that matches the supplied regex. Returns the PID of that process
+ [Arguments] ${remote_system} ${regex_string_to_match_on}
+ [Documentation] Uses ps to find a process that matches the supplied regex. Returns the PID of that process
... The ${regex_string_to_match_on} should produce a unique process otherwise the PID returned may not be
... the expected PID
- [Arguments] ${remote_system} ${regex_string_to_match_on}
# doing the extra -v grep in this command to exclude the grep process itself from the output
${output}= Run Command On Remote System ${remote_system} ps -elf | grep -v grep | grep ${regex_string_to_match_on} | awk '{print $4}'
# ${output} contains the system prompt and all we want is the value of the number
[Return] ${pid}
Get Process Thread Count On Remote System
+ [Arguments] ${remote_system} ${pid}
[Documentation] Executes the ps command to retrieve the lightweight process (aka thread) count.
- [Arguments] ${remote_system} ${pid}
${output}= Run Command On Remote System ${remote_system} ps --no-headers -o nlwp ${pid}
# ${output} contains the system prompt and all we want is the value of the number
${thread_count}= Fetch From Left ${output} \r
[Documentation] Reduces the common work of running a command on a remote system to a single higher level robot keyword,
... taking care to log in with a public key and. The command given is written and the output returned. No test conditions
... are checked.
- Log Attempting to execute ${cmd} on ${remote_system}
+ Log Attempting to execute ${cmd} on ${remote_system} by ${user} with ${keyfile_pass} and ${prompt}
${conn_id}= SSHLibrary.Open Connection ${remote_system} prompt=${prompt} timeout=${prompt_timeout}
- Login With Public Key ${user} ${USER_HOME}/.ssh/id_rsa any
+ Login With Public Key ${user} ${USER_HOME}/.ssh/id_rsa ${KEYFILE_PASS}
SSHLibrary.Write ${cmd}
${output}= SSHLibrary.Read Until ${prompt}
SSHLibrary.Close Connection
${body} OperatingSystem.Get File ${data_file}
${resp} RequestsLibrary.Post session ${dest_uri} data=${body} headers=${headers}
Should Be Equal As Strings ${resp.status_code} 200
+
+Stop One Or More Controllers
+ [Arguments] @{controllers}
+ [Documentation] Give this keyword a scalar or list of controllers to be stopped.
+ ${cmd} = Set Variable ${KARAF_HOME}/bin/stop
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ : FOR ${ip} IN @{controllers}
+ \ Wait Until Keyword Succeeds 60 s 3 s Controller Down Check ${ip}
+
+Start One Or More Controllers
+ [Arguments] @{controllers}
+ [Documentation] Give this keyword a scalar or list of controllers to be started.
+ ${cmd} = Set Variable ${KARAF_HOME}/bin/start
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ # TODO: This should throw an error if controller never comes up.
+ : FOR ${ip} IN @{controllers}
+ \ UtilLibrary.Wait For Controller Up ${ip} ${RESTCONFPORT}
+
+Kill One Or More Controllers
+ [Arguments] @{controllers}
+ [Documentation] Give this keyword a scalar or list of controllers to be stopped.
+ ${cmd} = Set Variable ps axf | grep karaf | grep -v grep | awk '{print \"kill -9 \" $1}' | sh
+ log ${cmd}
+ ${controller_pid}= Get Process ID Based On Regex On Remote System ${CONTROLLER} karaf
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ : FOR ${ip} IN @{controllers}
+ \ Wait Until Keyword Succeeds 60 s 3 s Controller Down Check ${ip}
+
+Controller Down Check
+ [Arguments] ${ip}
+ [Documentation] Checks to see if a controller is down by verifying that the karaf process isn't present.
+ ${cmd} = Set Variable ps axf | grep karaf | grep -v grep | wc -l
+ ${response} Run Command On Remote System ${ip} ${cmd}
+ Log Number of controller instances running: ${response}
+ Should Start With ${response} 0 Controller process found or there may be extra instances of
+ ... karaf running on the host machine.
+
+Clean One Or More Journals
+ [Arguments] @{controllers}
+ [Documentation] Give this keyword a scalar or list of controllers on which to clean journals.
+ ${del_cmd} = Set Variable rm -rf ${KARAF_HOME}/journal
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${del_cmd}
+
+Show Cluster Configuation Files
+ [Arguments] @{controllers}
+ [Documentation] Prints out the cluster configuration files for one or more controllers.
+ Log controllers: @{controllers}
+ ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/akka.conf
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/modules.conf
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/module-shards.conf
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ ${cmd} = Set Variable cat ${KARAF_HOME}/configuration/initial/jolokia.xml
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ ${cmd} = Set Variable cat ${KARAF_HOME}/etc/initial/org.apache.karaf.management.cfg
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+ ${cmd} = Set Variable cat ${KARAF_HOME}/etc/org.apache.karaf.features.cfg
+ : FOR ${ip} IN @{controllers}
+ \ Run Command On Remote System ${ip} ${cmd}
+++ /dev/null
-*** Settings ***
-Documentation Start the controllers
-Default Tags 3-node-cluster
-Library Collections
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Library ../../../libraries/CrudLibrary.py
-Library ../../../libraries/SettingsLibrary.py
-Library ../../../libraries/UtilLibrary.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${REST_CONTEXT} /restconf/config/
-
-*** Test Cases ***
-Stop All Controllers
- [Documentation] Stop all the controllers in the cluster
- StopAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${MEMBER1} ${MEMBER2} ${MEMBER3}
-
-Clean All Journals
- [Documentation] Clean the journals of all the controllers in the cluster
- CleanJournal ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Sleep 5
-
-Start All Controllers
- [Documentation] Start all the controllers in the cluster
- ${rc} StartAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${RESTCONFPORT} ${MEMBER1}
- ... ${MEMBER2} ${MEMBER3}
- Should Be True ${rc}
+++ /dev/null
-*** Settings ***
-Documentation Test cleanup
-Default Tags 3-node-cluster
-Library ../../../libraries/UtilLibrary.py
-Variables ../../../variables/Variables.py
-
-*** Test Cases ***
-Kill All Controllers
- KillController ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- KillController ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- KillController ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
-
-Clean All Journals
- CleanJournal ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
+++ /dev/null
-*** Settings ***
-Documentation Start the controllers
-Library Collections
-Library ../../../libraries/RequestsLibrary.py
-Library ../../../libraries/Common.py
-Library ../../../libraries/CrudLibrary.py
-Library ../../../libraries/SettingsLibrary.py
-Library ../../../libraries/UtilLibrary.py
-Variables ../../../variables/Variables.py
-
-*** Variables ***
-${REST_CONTEXT} /restconf/config/
-
-*** Test Cases ***
-Stop All Controllers
- [Documentation] Stop all the controllers in the cluster
- Stopcontroller ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Stopcontroller ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Stopcontroller ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Sleep 30
- KillController ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- KillController ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- KillController ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
-
-Clean All Journals
- [Documentation] Clean the journals of all the controllers in the cluster
- CleanJournal ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Sleep 5
-
-Start All Controllers
- [Documentation] Start all the controllers in the cluster
- Startcontroller ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Startcontroller ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Startcontroller ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- Sleep 120
--- /dev/null
+*** Settings ***
+Documentation Start the controllers
+Default Tags 3-node-cluster
+Resource ../../../libraries/Utils.txt
+
+*** Variables ***
+@{controllers} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
+${KARAF_HOME} ${WORKSPACE}/${BUNDLEFOLDER}
+
+*** Test Cases ***
+Stop All Controllers
+ [Documentation] Stop all the controllers in the cluster.
+ Stop One Or More Controllers @{controllers}
+
+Clean All Journals
+ [Documentation] Clean the journals of all the controllers in the cluster
+ Clean One Or More Journals @{controllers}
+
+Start All Controllers
+ [Documentation] Start all the controllers in the cluster
+ Start One Or More Controllers @{controllers}
Documentation This test finds the leader for shards in a 3-Node cluster and executes CRUD operations on them
Default Tags 3-node-cluster
Library Collections
-Library ../../../libraries/RequestsLibrary.py
+Library RequestsLibrary
Library ../../../libraries/Common.py
Library ../../../libraries/CrudLibrary.py
Library ../../../libraries/SettingsLibrary.py
Variables ../../../variables/Variables.py
*** Variables ***
-${REST_CONTEXT} /restconf/config/
${SHARD_CAR_NAME} shard-car-config
${SHARD_PEOPLE_NAME} shard-people-config
${SHARD_CAR_PERSON_NAME} shard-car-people-config
*** Test Cases ***
Get Car Leader And Followers
- ${CURRENT_CAR_LEADER} Wait For Leader ${SHARD_CAR_NAME}
+ ${CURRENT_CAR_LEADER} Wait For Leader To Be Found ${SHARD_CAR_NAME}
Set Suite Variable ${CURRENT_CAR_LEADER}
${CAR_FOLLOWERS} Get All Followers ${SHARD_CAR_NAME}
Set Suite Variable ${CAR_FOLLOWERS}
Wait Until Keyword Succeeds 60s 2s Get Cars And Verify @{CAR_FOLLOWERS}[1] ${NUM_ENTRIES}
Get People Leader And Followers
- ${CURRENT_PEOPLE_LEADER} Wait For Leader ${SHARD_PEOPLE_NAME}
+ ${CURRENT_PEOPLE_LEADER} Wait For Leader To Be Found ${SHARD_PEOPLE_NAME}
Set Suite Variable ${CURRENT_PEOPLE_LEADER}
${PEOPLE_FOLLOWERS} Get All Followers ${SHARD_PEOPLE_NAME}
Set Suite Variable ${PEOPLE_FOLLOWERS}
Wait Until Keyword Succeeds 60s 2s Get People And Verify @{PEOPLE_FOLLOWERS}[1] ${NUM_ENTRIES}
Get Car-Person Leader And Followers
- ${CURRENT_CAR_PERSON_LEADER} Wait For Leader ${SHARD_CAR_PERSON_NAME}
+ ${CURRENT_CAR_PERSON_LEADER} Wait For Leader To Be Found ${SHARD_CAR_PERSON_NAME}
Set Suite Variable ${CURRENT_CAR_PERSON_LEADER}
${CAR_PERSON_FOLLOWERS} Get All Followers ${SHARD_CAR_PERSON_NAME}
Set Suite Variable ${CAR_PERSON_FOLLOWERS}
Documentation This test finds the followers of certain shards in a 3-Node cluster and executes CRUD operations on any one follower
Default Tags 3-node-cluster
Library Collections
-Library ../../../libraries/RequestsLibrary.py
Library ../../../libraries/Common.py
Library ../../../libraries/CrudLibrary.py
Library ../../../libraries/SettingsLibrary.py
Variables ../../../variables/Variables.py
*** Variables ***
-${REST_CONTEXT} /restconf/config/
${SHARD_CAR_NAME} shard-car-config
${SHARD_PEOPLE_NAME} shard-people-config
${SHARD_CAR_PERSON_NAME} shard-car-people-config
Get car-person mappings from Leader
[Documentation] Get car-person mappings from the Leader to see all entries
- ${CURRENT_CAR_LEADER} Wait For Leader ${SHARD_CAR_PERSON_NAME}
+ ${CURRENT_CAR_LEADER} Wait For Leader To Be Found ${SHARD_CAR_PERSON_NAME}
Wait Until Keyword Succeeds 60s 2s Get Car-Person Mappings And Verify ${CURRENT_CAR_LEADER} ${NUM_ENTRIES}
Get car-person mappings from Follower2
Library ../../../libraries/UtilLibrary.py
Library ../../../libraries/ClusterStateLibrary.py
Resource ../../../libraries/ClusterKeywords.txt
+Resource ../../../libraries/Utils.txt
*** Variables ***
${CAR_SHARD} shard-car-config
${NUM_CARS} ${50}
${NUM_ORIG_CARS} ${10}
+${KARAF_HOME} ${WORKSPACE}/${BUNDLEFOLDER}
*** Test Cases ***
Get old car leader
- ${OLD_CAR_LEADER} Wait For Leader ${CAR_SHARD}
+ ${OLD_CAR_LEADER} Wait For Leader To Be Found ${CAR_SHARD}
Set Suite Variable ${OLD_CAR_LEADER}
Delete cars on old leader
Wait Until Keyword Succeeds 60s 2s Get Cars And Verify ${NEW_CAR_LEADER} ${NUM_CARS}
Restart old Car leader
- StartController ${OLD_CAR_LEADER} ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${PORT}
+ Start One Or More Controllers ${OLD_CAR_LEADER}
Get added cars from old leader
[Documentation] Get the added cars from the old leader
*** Variables ***
${PEOPLE_SHARD} shard-people-config
${NUM_ENTRIES} ${50}
+${KARAF_HOME} ${WORKSPACE}/${BUNDLEFOLDER}
*** Test Cases ***
Switch People Leader
[Documentation] Stop the leader to cause a new leader to be elected
- ${OLD_PEOPLE_LEADER} Wait For Leader ${PEOPLE_SHARD}
+ ${OLD_PEOPLE_LEADER} Wait For Leader To Be Found ${PEOPLE_SHARD}
${NEW_PEOPLE_LEADER} Switch Leader ${PEOPLE_SHARD} ${OLD_PEOPLE_LEADER}
Set Suite Variable ${OLD_PEOPLE_LEADER}
Set Suite Variable ${NEW_PEOPLE_LEADER}
Wait Until Keyword Succeeds 60s 2s Get People And Verify ${NEW_PEOPLE_LEADER} ${NUM_ENTRIES}
Restart old People leader
- StartController ${OLD_PEOPLE_LEADER} ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${PORT}
+ Start One Or More Controllers ${OLD_PEOPLE_LEADER}
Get added people from old leader
Wait Until Keyword Succeeds 60s 2s Get People And Verify ${OLD_PEOPLE_LEADER} ${NUM_ENTRIES}
*** Variables ***
${CAR_SHARD} shard-car-config
${NUM_CARS} ${50}
+${KARAF_HOME} ${WORKSPACE}/${BUNDLEFOLDER}
+@{controllers} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
*** Test Cases ***
Get car leader
- ${CAR_LEADER} Wait For Leader ${CAR_SHARD}
+ ${CAR_LEADER} Wait For Leader To Be Found ${CAR_SHARD}
Set Suite Variable ${CAR_LEADER}
Delete cars from leader
Delete All Cars And Verify ${CAR_LEADER}
Stop all controllers after delete
- StopAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${MEMBER1} ${MEMBER2} ${MEMBER3}
+ Stop One Or More Controllers @{controllers}
Start all controllers after delete
- ${rc} StartAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${RESTCONFPORT} ${MEMBER1}
- ... ${MEMBER2} ${MEMBER3}
- Should Be True ${rc}
+ Start One Or More Controllers @{controllers}
Verify no cars on leader after restart
- ${resp} Getcars ${CAR_LEADER} ${PORT} ${0}
+ ${resp} Getcars ${CAR_LEADER} ${RESTCONFPORT} ${0}
Should Be Equal As Strings ${resp.status_code} 404
Add cars on leader
Add Cars And Verify ${CAR_LEADER} ${NUM_CARS}
Stop all controllers after add
- StopAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${MEMBER1} ${MEMBER2} ${MEMBER3}
+ Stop One Or More Controllers @{controllers}
Start all controllers after add
- ${rc} StartAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${RESTCONFPORT} ${MEMBER1}
- ... ${MEMBER2} ${MEMBER3}
- Should Be True ${rc}
+ Start One Or More Controllers @{controllers}
Get cars from leader after restart
Wait Until Keyword Succeeds 60s 2s Get Cars And Verify ${CAR_LEADER} ${NUM_CARS}
Variables ../../../variables/Variables.py
*** Variables ***
-${REST_CONTEXT} /restconf/config/
${CAR_SHARD} shard-car-config
${NUM_CARS} ${60}
+@{controllers} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
+${KARAF_HOME} ${WORKSPACE}/${BUNDLEFOLDER}
*** Test Cases ***
Stop All Controllers
[Documentation] Stop all the controllers in the cluster
- StopAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${MEMBER1} ${MEMBER2} ${MEMBER3}
+ Stop One Or More Controllers @{controllers}
Clean All Journals
[Documentation] Clean the journals of all the controllers in the cluster
- CleanJournal ${MEMBER1} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER2} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
- CleanJournal ${MEMBER3} ${USER_NAME} ${PASSWORD} ${KARAF_HOME}
+ Clean One Or More Journals @{controllers}
Start All Controllers
[Documentation] Start all the controllers in the cluster
- ${rc} StartAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${RESTCONFPORT} ${MEMBER1}
- ... ${MEMBER2} ${MEMBER3}
- Should Be True ${rc}
+ Start One Or More Controllers @{controllers}
Get car leader and followers
- ${CURRENT_CAR_LEADER} Wait For Leader ${CAR_SHARD}
+ ${CURRENT_CAR_LEADER} Wait For Leader To Be Found ${CAR_SHARD}
Set Suite Variable ${CURRENT_CAR_LEADER}
${CAR_FOLLOWERS} Get All Followers ${CAR_SHARD}
Set Suite Variable ${CAR_FOLLOWERS}
Stop both of the followers
- StopAllControllers ${USER_NAME} ${PASSWORD} ${KARAF_HOME} @{CAR_FOLLOWERS}[0] @{CAR_FOLLOWERS}[1]
- Wait Until Keyword Succeeds 30s 2s Expect No Leader
+ @{followers} = Create List @{CAR_FOLLOWERS}[0] @{CAR_FOLLOWERS}[1]
+ Stop One Or More Controllers @{followers}
-Attempt to add a car from the leader
+Attempt to add a car to the leader
[Documentation] Should fail as both followers are down
- AddCar ${CURRENT_CAR_LEADER} ${PORT} ${1}
+ AddCar ${CURRENT_CAR_LEADER} ${RESTCONFPORT} ${1}
Sleep 2
- ${resp} Getcars ${CURRENT_CAR_LEADER} ${PORT} ${1}
+ ${resp} Getcars ${CURRENT_CAR_LEADER} ${RESTCONFPORT} ${1}
Should Not Be Equal As Strings ${resp.status_code} 200
Restart the first follower
- StartController @{CAR_FOLLOWERS}[0] ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${PORT}
- Sleep 1
+ Start One Or More Controllers @{CAR_FOLLOWERS}[0]
-Add cars from the first follower
- Wait Until Keyword Succeeds 60s 2s Add Cars And Verify @{CAR_FOLLOWERS}[0] ${NUM_CARS} 4s
+Add cars to the first follower
+ Log Adding ${NUM_CARS} cars to @{CAR_FOLLOWERS}[0]
+ Add Cars And Verify Without Init @{CAR_FOLLOWERS}[0] ${NUM_CARS} 4s
Restart the second follower
- StartController @{CAR_FOLLOWERS}[1] ${USER_NAME} ${PASSWORD} ${KARAF_HOME} ${PORT}
+ Start One Or More Controllers @{CAR_FOLLOWERS}[1]
Get all the cars from the second follower
- Wait Until Keyword Succeeds 60s 2s Get Cars And Verify @{CAR_FOLLOWERS}[1] ${NUM_CARS}
+ Get Cars And Verify @{CAR_FOLLOWERS}[1] ${NUM_CARS}
*** Keywords ***
Expect No Leader
- ${leader} GetLeader ${CAR_SHARD} ${3} ${1} ${1} ${PORT}
+ ${leader} GetLeader ${CAR_SHARD} ${3} ${1} ${1} ${RESTCONFPORT}
... ${CURRENT_CAR_LEADER}
Should Be Equal As Strings ${leader} None
--- /dev/null
+*** Settings ***
+Documentation Test cleanup
+Default Tags 3-node-cluster
+Resource ../../../libraries/Utils.txt
+Library ../../../libraries/UtilLibrary.py
+Variables ../../../variables/Variables.py
+
+*** Variables ***
+@{controllers} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
+${KARAF_HOME} ${WORKSPACE}/${BUNDLEFOLDER}
+
+*** Test Cases ***
+Kill All Controllers
+ [Documentation] Kill all the karaf processes in the cluster
+ Kill One Or More Controllers @{controllers}
+
+Clean All Journals
+ [Documentation] Clean the journals of all the controllers in the cluster
+ Clean One Or More Journals @{controllers}
--- /dev/null
+*** Settings ***
+Documentation Start the controllers
+Default Tags 3-node-cluster
+Resource ../../../libraries/Utils.txt
+
+*** Variables ***
+@{controllers} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
+${KARAF_HOME} ${WORKSPACE}/${BUNDLEFOLDER}
+
+*** Test Cases ***
+Stop All Controllers
+ [Documentation] Stop all the controllers in the cluster
+ Stop One Or More Controllers @{controllers}
+
+Clean All Journals
+ [Documentation] Clean the journals of all the controllers in the cluster
+ Clean One Or More Journals @{controllers}
+
+Start All Controllers
+ [Documentation] Start all the controllers in the cluster
+ Start One Or More Controllers @{controllers}
*** Settings ***
Documentation Test suite for Routed RPC.
Library Collections
-Library ../../../libraries/RequestsLibrary.py
+Library RequestsLibrary
Library ../../../libraries/Common.py
Library ../../../libraries/CrudLibrary.py
Library ../../../libraries/SettingsLibrary.py
Variables ../../../variables/Variables.py
*** Variables ***
-${REST_CONTEXT} /restconf/config/
+@{controllers} ${CONTROLLER} ${CONTROLLER1} ${CONTROLLER2}
*** Test Cases ***
Add cars and get cars from Leader
[Documentation] Add 100 cars and get added cars from Leader
- ${resp} InitCar ${MEMBER1} ${PORT}
- ${resp} AddCar ${MEMBER1} ${PORT} ${100}
- ${resp} Getcars ${MEMBER1} ${PORT} ${0}
+ ${resp} InitCar ${CONTROLLER} ${PORT}
+ ${resp} AddCar ${CONTROLLER} ${PORT} ${100}
+ ${resp} Getcars ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} manufacturer1
+ Should Contain ${resp.content} manufacturer1 cars not added!
Add persons and get persons from Leader
[Documentation] Add 100 persons and get persons Note: There should be one person added first to enable rpc
- ${resp} AddPerson ${MEMBER1} ${PORT} ${0}
- ${resp} AddPerson ${MEMBER1} ${PORT} ${100}
- ${resp} GetPersons ${MEMBER1} ${PORT} ${0}
+ ${resp} AddPerson ${CONTROLLER} ${PORT} ${0}
+ ${resp} AddPerson ${CONTROLLER} ${PORT} ${100}
+ ${resp} GetPersons ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} user5
- SLEEP 10
+ Should Contain ${resp.content} user5 people not added!
Add car-person mapping and get car-person mapping from Follower1
[Documentation] Add car-person and get car-person from Leader Note: This is done to enable working of rpc
- ${resp} AddCarPerson ${MEMBER2} ${PORT} ${0}
- ${resp} GetCarPersonMappings ${MEMBER2} ${PORT} ${0}
+ ${resp} AddCarPerson ${CONTROLLER1} ${PORT} ${0}
+ ${resp} GetCarPersonMappings ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} user0
- SLEEP 5
+ Should Contain ${resp.content} user0 car-person not initialized!
Purchase 100 cars using Follower1
[Documentation] Purchase 100 cars using Follower1
- ${resp} BuyCar ${MEMBER2} ${PORT} ${100}
- ${resp} GetCarPersonMappings ${MEMBER2} ${PORT} ${0}
+ ${resp} BuyCar ${CONTROLLER1} ${PORT} ${100}
+ ${resp} GetCarPersonMappings ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Get Cars from Leader
[Documentation] Get 100 using Leader
- ${resp} Getcars ${MEMBER1} ${PORT} ${0}
+ ${resp} Getcars ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} manufacturer99
Get persons from Leader
[Documentation] Get 101 Persons from Leader
- ${resp} GetPersons ${MEMBER1} ${PORT} ${0}
+ ${resp} GetPersons ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} user100
Get car-person mappings using Leader
[Documentation] Get 101 car-person mappings using Leader to see 100 entry
- ${resp} GetCarPersonMappings ${MEMBER1} ${PORT} ${0}
+ ${resp} GetCarPersonMappings ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} user100
*** Settings ***
Documentation Test suite for Routed RPC.
+Resource ../../../libraries/Utils.txt
Library Collections
-Library ../../../libraries/RequestsLibrary.py
+Library RequestsLibrary
Library ../../../libraries/Common.py
Library ../../../libraries/CrudLibrary.py
Library ../../../libraries/SettingsLibrary.py
Library ../../../libraries/ClusterStateLibrary.py
Variables ../../../variables/Variables.py
-*** Variables ***
-${REST_CONTEXT} /restconf/config/
-
*** Test Cases ***
Add cars and get cars from Leader
[Documentation] Add 100 cars and get added cars from Leader
- ${resp} InitCar ${MEMBER1} ${PORT}
- ${resp} AddCar ${MEMBER1} ${PORT} ${100}
- ${resp} Getcars ${MEMBER1} ${PORT} ${0}
+ ${resp} InitCar ${CONTROLLER} ${PORT}
+ ${resp} AddCar ${CONTROLLER} ${PORT} ${100}
+ ${resp} Getcars ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} manufacturer1
Add persons and get persons from Leader
[Documentation] Add 100 persons and get persons Note: There should be one person added first to enable rpc
- ${resp} AddPerson ${MEMBER1} ${PORT} ${0}
- ${resp} AddPerson ${MEMBER1} ${PORT} ${100}
- ${resp} GetPersons ${MEMBER1} ${PORT} ${0}
+ ${resp} AddPerson ${CONTROLLER} ${PORT} ${0}
+ ${resp} AddPerson ${CONTROLLER} ${PORT} ${100}
+ ${resp} GetPersons ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} user5
Add car-person mapping and get car-person mapping from Follower1
[Documentation] Add car-person and get car-person from Follower1 Note: This is done to enable working of rpc
- ${resp} AddCarPerson ${MEMBER2} ${PORT} ${0}
- ${resp} GetCarPersonMappings ${MEMBER2} ${PORT} ${0}
+ ${resp} AddCarPerson ${CONTROLLER1} ${PORT} ${0}
+ ${resp} GetCarPersonMappings ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} user0
Purchase 100 cars using Follower
[Documentation] Purchase 100 cars using Follower
- SLEEP 10
- ${resp} BuyCar ${MEMBER2} ${PORT} ${100}
- ${resp} GetCarPersonMappings ${MEMBER2} ${PORT} ${0}
+ ${resp} BuyCar ${CONTROLLER1} ${PORT} ${100}
+ ${resp} GetCarPersonMappings ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Get Cars from Leader
[Documentation] Get 100 using Leader
- ${resp} Getcars ${MEMBER1} ${PORT} ${0}
+ ${resp} Getcars ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} manufacturer9
+ Should Contain ${resp.content} manufacturer9 cars not returned!
Get persons from Leader
[Documentation] Get 11 Persons from Leader
- ${resp} GetPersons ${MEMBER1} ${PORT} ${0}
+ ${resp} GetPersons ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} user100
+ Should Contain ${resp.content} user100 people not returned!
Get car-person mappings using Leader
[Documentation] Get car-person mappings using Leader to see 100 entry
- ${resp} GetCarPersonMappings ${MEMBER1} ${PORT} ${0}
+ ${resp} GetCarPersonMappings ${CONTROLLER} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} user100
+ Should Contain ${resp.content} user100 car-people not returned!
Stop Leader
[Documentation] Stop Leader controller
- ${resp} Stopcontroller ${MEMBER1} ${USERNAME} ${PASSWORD} ${KARAF_HOME}
- SLEEP 30
- ${resp} Killcontroller ${MEMBER1} ${USERNAME} ${PASSWORD} ${KARAF_HOME}
+ Stop One Or More Controllers ${CONTROLLER}
+ Kill One Or More Controllers ${CONTROLLER}
Add cars and get cars from Follower1
[Documentation] Add 100 cars and get added cars from Follower
- ${resp InitCar ${MEMBER2} ${PORT}
- ${resp} AddCar ${MEMBER2} ${PORT} ${100}
- ${resp} Getcars ${MEMBER2} ${PORT} ${0}
+ ${resp} InitCar ${CONTROLLER1} ${PORT}
+ ${resp} AddCar ${CONTROLLER1} ${PORT} ${100}
+ ${resp} Getcars ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} manufacturer1
+ Should Contain ${resp.content} manufacturer1 cars not added!
Add persons and get persons from Follower1
[Documentation] Add 100 persons and get persons Note: There should be one person added first to enable rpc
- ${resp} AddPerson ${MEMBER2} ${PORT} ${0}
- ${resp} AddPerson ${MEMBER2} ${PORT} ${100}
- ${resp} GetPersons ${MEMBER2} ${PORT} ${0}
+ ${resp} AddPerson ${CONTROLLER1} ${PORT} ${0}
+ ${resp} AddPerson ${CONTROLLER1} ${PORT} ${100}
+ ${resp} GetPersons ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} user5
- SLEEP 10
+ Should Contain ${resp.content} user5 car-people not initialized!
Purchase 100 cars using Follower2
[Documentation] Purchase 100 cars using Follower2
- ${resp} BuyCar ${MEMBER3} ${PORT} ${100}
- SLEEP 10
- ${resp} GetCarPersonMappings ${MEMBER3} ${PORT} ${0}
+ ${resp} BuyCar ${CONTROLLER2} ${PORT} ${100}
+ ${resp} GetCarPersonMappings ${CONTROLLER2} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
Get Cars from Follower1
[Documentation] Get 100 using Follower1
- ${resp} Getcars ${MEMBER2} ${PORT} ${0}
+ ${resp} Getcars ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} manufacturer9
+ Should Contain ${resp.content} manufacturer9 cars not returned!
Get persons from Follower1
[Documentation] Get 11 Persons from Follower1
- ${resp} GetPersons ${MEMBER2} ${PORT} ${0}
+ ${resp} GetPersons ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} user100
+ Should Contain ${resp.content} user100 people not returned!
Get car-person mappings using Follower1
[Documentation] Get car-person mappings using Follower1 to see 100 entry
- ${resp} GetCarPersonMappings ${MEMBER2} ${PORT} ${0}
+ ${resp} GetCarPersonMappings ${CONTROLLER1} ${PORT} ${0}
Should Be Equal As Strings ${resp.status_code} 200
- Should Contain ${resp.content} user100
+ Should Contain ${resp.content} user100 car-people not returned!
Start Leader
[Documentation] Start Leader controller
- ${resp} Startcontroller ${MEMBER1} ${USERNAME} ${PASSWORD} ${KARAF_HOME}
- SLEEP 20
+ Start One Or More Controllers ${CONTROLLER}
# Place the suites in run order:
-integration/test/csit/suites/clustering/datastore
+integration/test/csit/suites/controller/Clustering_Datastore
CONTAINER = 'default'
USER = 'admin'
PWD = 'admin'
+PASSWORD = 'EMPTY'
AUTH = [u'admin', u'admin']
SCOPE = 'sdn'
HEADERS = {'Content-Type': 'application/json'}
TOPO_TREE_LEVEL = 2
TOPO_TREE_DEPTH = 3
TOPO_TREE_FANOUT = 2
+CONTROLLERS = ['CONTROLLER', 'CONTROLLER1', 'CONTROLLER2']
+KEYFILE_PASS = 'any'
-# KARAF Varaiable
+# KARAF Variaable
KARAF_SHELL_PORT = '8101'
KARAF_PROMPT = 'opendaylight-user'
KARAF_USER = 'karaf'
'network-topology/topology'
'/topology-netconf/node/'
'controller-config/yang-ext:mount')
+CONFIG_API = '/restconf/config/'
+OPERATIONAL_API = '/restconf/operational/'
# TOKEN
AUTH_TOKEN_API = '/oauth2/token'
Code Review / integration / test.git / commitdiff