2 Documentation PCEP performance suite, uses restconf with configurable authentication.
4 ... Copyright (c) 2015 Cisco Systems, Inc. and others. All rights reserved.
6 ... This program and the accompanying materials are made available under the
7 ... terms of the Eclipse Public License v1.0 which accompanies this distribution,
8 ... and is available at http://www.eclipse.org/legal/epl-v10.html
13 ... This is a suite which has both scale and performance aspects.
14 ... Given scale target, suite reports failures if functional error
15 ... is detected, or if various time limits expire.
16 ... For passing test cases, their duration is the performance metric.
18 ... ODL acts as a translation layer between PCEP capable devices
19 ... and users employing RESTCONF.
20 ... Performance measurement focuses on two different workflows.
22 ... The first workflow is initial synchronization, when ODL learns
23 ... the state of PCEP topology as devices connect to it,
24 ... while restconf user reads the state repeatedly.
25 ... The second workflow is mass update, when restconf users issue RPCs
26 ... to updale Layer Switched Paths on Path Computation Clients.
28 ... This suite uses pcc-mock (downloaded from Nexus) to simulate PCCs.
29 ... It needs segment of bindable IP addresses,
30 ... one for each simulated PCC; so running pcc-mock from remote machine
31 ... is only viable when just single PCC is simulated.
32 ... Testing with multiple PCCs works best when pcc-mock
33 ... runs on the same VM as ODL, so 127.0.0.0/8 subnet can be used.
35 ... Library AuthStandalone is used directly for restconf reads
36 ... in the first workflow. That library transparently handles several
37 ... http authentication methods, based on credentials and pybot arguments.
39 ... In the second workflow, updater.py utility is used for issuing
40 ... rapid restconf requests. It can use multiple worker threads,
41 ... as http requests are blocking.
42 ... Due to CPython interpreter itself being single threaded,
43 ... amounts of threads above 8-16 are actually slightly slower
44 ... (which may roughly correspond to network traffic
45 ... being more limiting factor than CPU).
46 ... This suite starts updater utility bound to single CPU,
47 ... as this setup was the most performant in other tests.
49 ... In case of failed test case, other tests are skipped (unless
50 ... this is overriden by [Setup]) to finish test run sooner.
52 ... Variables and test case names refer to Controller(ODL_SYSTEM) and Mininet
53 ... (TOOLS_SYSTEM), those are assumed to be separate remote VMs, one to host ODL,
54 ... other to host tools.
55 ... In case updater and pcc-mock are desired to run
56 ... from separate machines, their parameters use Mininet(TOOLS_SYSTEM)
57 ... values as default.
58 ... If both updater VM and pcc-mock VM parameters are specified,
59 ... Mininet(TOOLS_SYSTEM) parameters may be skipped.
60 ... Variable ${USE_TOOLS_SYSTEM} decides the pcc-mock running machine.
62 ... Some launch scripts put restrictions on how pybot options
63 ... can be specified, so there are utility variables to help with
64 ... copying Controller related value to apply fo updater of pccmock.
65 ... Having a tool co-located with ODL reduces network latency,
66 ... but puts more pressure on CPU and memory on Controller VM.
68 ... In some environments, issues with TIME-WAIT prevent high restconf rates,
69 ... so TCP reuse is temporarily allowed during the suite run, if possible
70 ... (and if not disabled by UPDATERVM_ENABLE_TCP_RW_REUSE option value).
71 ... See http://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux.html
72 ... This suite ignores possible failures when changing reuse.
74 ... Similarly, in some environments, handling of requests.Session object matters
75 ... try changing RESTCONF_REUSE value to see if it helps.
77 ... Variables to override (only if needed) in pybot command:
78 ... (Look into Variables table to see the default values.)
80 ... FIRST_PCC_IP: Set in case bind address is different from public pcc-mock VM address.
81 ... LOG_NAME: Filename (without path) to save pcc-mock output into.
82 ... LOG_PATH: Override if not the same as pccmock VM workspace.
83 ... LSPS: Number of LSPs per PCC to simulate and test.
84 ... MOCK_FILE: Filename to use for mock-pcc executable instead of the timestamped one.
85 ... ODL_SYSTEM_IP: Numeric IP address of VM where ODL runs.
86 ... ODL_SYSTEM_USER: Username for ssh login to ODL VM.
87 ... ODL_SYSTEM_PASSWORD: Ssh password, empty means public keys are used instead.
88 ... ODL_SYSTEM_PROMPT: Substring to identify Linux prompt on ODL VM.
89 ... ODL_SYSTEM_WORKSPACE: Path to where files can be written on ODL VM.
90 ... PCCDOWNLOAD_HOSTHEADER: Download server may check checks this header before showing content.
91 ... PCCDOWNLOAD_URLBASE: URL to pcep-pcc-mock folder in Nexus (use numberic IP if DNS has problems).
92 ... PCCMOCK_COLOCATED: If True, set PCCMOCKVM* to mirror ODL_SYSTEM*
93 ... PCCMOCKVM_IP: Override TOOLS_SYSTEM for pcc-mock usage.
94 ... PCCMOCKVM_*: Override corresponding TOOLS_SYSTEM_* for pcc-mock usage.
95 ... PCCS: Number of PCCs to simulate and test.
96 ... PCEP_READY_VERIFY_TIMEOUT: Grace period for pcep-topology to appear. Lower if ODL is ready.
97 ... RESTCONF_*: USER, PASSWORD and SCOPE to authenticate with, REUSE session.
98 ... (Note: If SCOPE is not empty, token-based authentication is used.)
99 ... TOOLS_SYSTEM_IP: Numeric IP address of VM to run pcc-mock and updater from by default.
100 ... TOOLS_SYSTEM_PASSWORD: Linux password to go with the username (empty means keys).
101 ... TOOLS_SYSTEM_PROMPT: Substring to identify Linux prompt on TOOLS_SYSTEM VM.
102 ... TOOLS_SYSTEM_USER: Linux username to SSH to on TOOLS_SYSTEM VM.
103 ... TOOLS_SYSTEM_WORKSPACE: Path to where files may be created on TOOLS_SYSTEM VM.
104 ... UPDATER_COLOCATED: If True, overrides UPDATERVM_* parameters to point at ODL_SYSTEM
105 ... (The purpose is to provide an option without ability to unpack ODL_SYSTEM value.)
106 ... UPDATER_ODLADDRESS: Override if public ODL_SYSTEM address is not best fit.
107 ... UPDATER_REFRESH: Main updater thread may sleep this long. Balance precision with overhead.
108 ... UPDATER_TIMEOUT: If updater stops itself if running more than this time.
109 ... (Set this limit according to your performance target.)
110 ... UPDATERVM_ENABLE_TCP_RW_REUSE: Set to false if changing Linux configuration is not desired.
111 ... UPDATERVM_IP: Override TOOLS_SYSTEM for updater.py usage.
112 ... UPDATERVM_*: Override corresponding TOOLS_SYSTEM_* for updater.py usage.
113 Suite Setup Initial Setup
114 Suite Teardown Disconnect
115 Test Setup FailFast.Fail_This_Fast_On_Previous_Error
116 Test Teardown FailFast.Start_Failing_Fast_If_This_Failed
117 Library SSHLibrary timeout=10s
118 Library RequestsLibrary
119 Library ${CURDIR}/../../../libraries/AuthStandalone.py
120 Resource ${CURDIR}/../../../variables/Variables.robot
121 Resource ${CURDIR}/../../../libraries/BGPcliKeywords.robot
122 Resource ${CURDIR}/../../../libraries/FailFast.robot
123 Resource ${CURDIR}/../../../libraries/NexusKeywords.robot # for Deploy_Artifact
124 Resource ${CURDIR}/../../../libraries/SSHKeywords.robot # for Require_* and Assure_*, Flexible_SSH_Login
127 # This Variable decides the pcc mock to run in ODL system or tools system.
128 ${USE_TOOLS_SYSTEM} False
129 # This table acts as an exhaustive list of variables users can modify on pybot invocation.
130 # It also contains commented-out lines for variables defined elswhere.
131 # Keep this list in alphabetical order.
134 ${FIRST_PCC_IP} ${PCCMOCKVM_IP}
135 # ${LOG_FILE} is reserved for location of pybot-created log.html
136 ${INIT_PCC_DEVICE_COUNT} ${100}
137 ${LOG_NAME} throughpcep.log
138 ${LOG_PATH} ${PCCMOCKVM_WORKSPACE}
140 #Reduced max pcc device count to 100 for BGPCEP-901
141 ${MAX_PCC_DEVICE_COUNT} ${100}
142 ${ODL_SYSTEM_WORKSPACE} /tmp
143 ${PARALLEL_ITERATION} 10
144 ${PCC_DEVICE_INCREMENT} ${50}
145 ${PCCDOWNLOAD_HOSTHEADER} nexus.opendaylight.org
146 ${PCCDOWNLOAD_URLBASE} http://${PCCDOWNLOAD_HOSTHEADER}/content/repositories/opendaylight.snapshot/org/opendaylight/bgpcep/pcep-pcc-mock/
147 ${PCCMOCK_COLOCATED} False
148 ${PCCMOCKVM_IP} ${TOOLS_SYSTEM_IP}
149 ${PCCMOCKVM_PASSWORD} ${TOOLS_SYSTEM_PASSWORD}
150 ${PCCMOCKVM_PROMPT} ${TOOLS_SYSTEM_PROMPT}
151 ${PCCMOCKVM_USER} ${TOOLS_SYSTEM_USER}
152 ${PCCMOCKVM_WORKSPACE} ${TOOLS_SYSTEM_WORKSPACE}
154 ${PCEP_READY_VERIFY_TIMEOUT} 300s
155 # Yes, the default timeout is 5 minutes, as this suite might be started eagerly just after ODL starts booting up.
156 ${RESTCONF_PASSWORD} ${PWD} # from Variables.robot
157 ${RESTCONF_REUSE} True
158 ${RESTCONF_SCOPE} ${EMPTY}
159 ${RESTCONF_USER} ${USER} # from Variables.robot
160 ${SEQUENTIAL_ITERATION} 15
161 ${TOOLS_SYSTEM_WORKSPACE} /tmp
162 ${UPDATER_COLOCATED} False
163 ${UPDATER_ODLADDRESS} ${ODL_SYSTEM_IP}
164 ${UPDATER_REFRESH} 0.1
165 # Updater timeout is overwritten in releng/builder
166 ${UPDATER_TIMEOUT} 300
167 ${UPDATERVM_ENABLE_TCP_RW_REUSE} True
168 ${UPDATERVM_IP} ${TOOLS_SYSTEM_IP}
169 ${UPDATERVM_PASSWORD} ${TOOLS_SYSTEM_PASSWORD}
170 ${UPDATERVM_PROMPT} ${TOOLS_SYSTEM_PROMPT}
171 ${UPDATERVM_USER} ${TOOLS_SYSTEM_USER}
172 ${UPDATERVM_WORKSPACE} ${TOOLS_SYSTEM_WORKSPACE}
176 [Documentation] Open SSH session to updater VM, copy the utility there, including dependencies, also prepare direct http session.
177 BuiltIn.Run_Keyword_If ${UPDATER_COLOCATED} Updater_From_Controller
178 SSHLibrary.Open_Connection ${UPDATERVM_IP} alias=updater
179 SSHLibrary.Set_Client_Configuration timeout=20s
180 SSHLibrary.Set_Client_Configuration prompt=${UPDATERVM_PROMPT}
181 SSHKeywords.Flexible_SSH_Login ${UPDATERVM_USER} ${UPDATERVM_PASSWORD} delay=4s
182 SSHKeywords.Require_Python
183 SSHLibrary.Put_File ${CURDIR}/../../../../tools/pcep_updater/updater.py ${UPDATERVM_WORKSPACE}/
184 SSHLibrary.Put_File ${CURDIR}/../../../libraries/AuthStandalone.py ${UPDATERVM_WORKSPACE}/
185 SSHKeywords.Assure_Library_Counter target_dir=${UPDATERVM_WORKSPACE}
186 SSHKeywords.Assure_Library_Ipaddr target_dir=${UPDATERVM_WORKSPACE}
187 # Done preparation of Updater VM, now use AuthStandalone to create session from robot VM too.
188 BuiltIn.Log_Many ${RESTCONF_USER} ${RESTCONF_PASSWORD} ${RESTCONF_SCOPE} ${ODL_SYSTEM_IP}
189 ${session} = AuthStandalone.Init_Session ${ODL_SYSTEM_IP} ${RESTCONF_USER} ${RESTCONF_PASSWORD} ${RESTCONF_SCOPE}
190 BuiltIn.Set_Suite_Variable ${rest_session} ${session}
191 # TODO: Define http timeouts.
193 Save_And_Enable_Tcp_Rw_Reuse
194 [Documentation] If requested, temporarily enable TCP port reuse on Updater VM to allow for high rate of TCP connections. Do not start failing fast.
195 BuiltIn.Pass_Execution_If not ${UPDATERVM_ENABLE_TCP_RW_REUSE} Manipulation of tcp_rw_reuse is not requested.
196 ${old_value} = SSHLibrary.Execute_Command cat /proc/sys/net/ipv4/tcp_tw_reuse
197 # The next line may be skipped if the previous line failed.
198 BuiltIn.Set_Suite_Variable ${tcp_rw_reuse} ${old_value}
199 ${out} ${rc} = SSHLibrary.Execute_Command sudo bash -c "echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse" return_rc=True
200 BuiltIn.Should_Be_Equal ${rc} ${0}
201 # Lack of sudo access should not prevent the rest of suite from trying without TCP reuse.
202 [Teardown] Do_Not_Start_Failing_If_This_Failed
204 Topology_Precondition
205 [Documentation] Verify that within timeout, PCEP topology is present, with no PCC connected.
208 Builtin.Wait_Until_Keyword_Succeeds ${PCEP_READY_VERIFY_TIMEOUT} 1s Pcep_Off
209 # Yes, timeout is 5 minutes, as this suite might be started eagerly just after ODL starts booting up.
211 Topology_Intercondition
212 [Documentation] Verify that within timeout, PCEP topology contains correct numbers of LSPs.
214 [Setup] Start_Pcc_Mock
215 ${localsize} = Evaluate int(${PCCS})*int(${LSPS})
216 Builtin.Set_Suite_Variable ${size} ${localsize}
218 Builtin.Wait_Until_Keyword_Succeeds 120s 1s Pcep_On
219 # TODO: Make timeout value scale with ${size}?
222 [Documentation] Run updater tool to change hops, using 1 blocking http thread.
225 [Teardown] Do_Not_Start_Failing_If_This_Failed
228 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
233 [Documentation] Run updater tool to change hops again, using 2 blocking http threads.
236 [Teardown] Do_Not_Start_Failing_If_This_Failed
239 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
244 [Documentation] Run updater tool to change hops again, using 4 blocking http threads.
247 [Teardown] Do_Not_Start_Failing_If_This_Failed
250 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
255 [Documentation] Run updater tool to change hops again, using 8 blocking http threads.
258 [Teardown] Do_Not_Start_Failing_If_This_Failed
261 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
266 [Documentation] Run updater tool to change hops again, using 16 blocking http threads.
269 [Teardown] Do_Not_Start_Failing_If_This_Failed
272 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
277 [Documentation] Run updater tool to change hops again, using 32 blocking http threads.
280 [Teardown] Do_Not_Start_Failing_If_This_Failed
283 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
288 [Documentation] Run updater tool to change hops again, using 64 blocking http threads.
291 [Teardown] Do_Not_Start_Failing_If_This_Failed
294 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
299 [Documentation] Run updater tool to change hops again, using 128 blocking http threads.
302 [Teardown] Do_Not_Start_Failing_If_This_Failed
305 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
310 [Documentation] Run updater tool to change hops again, using 256 blocking http threads.
313 [Teardown] Do_Not_Start_Failing_If_This_Failed
316 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
321 [Documentation] Run updater tool to change hops again, using 512 blocking http threads.
324 [Teardown] Do_Not_Start_Failing_If_This_Failed
327 [Documentation] Verify that within timeout, the correct number of new hops is in PCEP topology.
331 Updater_with delegate
332 [Documentation] Run updater tool to revoke the delegate for the given hop, using 1 blocking http thread.
334 Updater 10 1 127.1.0.0 ${PCCS} ${LSPS} True false 127.1.0.1 2
336 [Teardown] Do_Not_Start_Failing_If_This_Failed
338 PCEP_sessions_from_multiple_machines
339 [Documentation] Download and start PCC Mock in tools vm and do updation
340 Download_Pcc_Mock ${TOOLS_SYSTEM_IP} ${TOOLS_SYSTEM_USER} ${TOOLS_SYSTEM_PASSWORD} ${TOOLS_SYSTEM_PROMPT} pccmock_toolsvm
342 Start_Pcc_Mock pccmock_toolsvm ${TOOLS_SYSTEM_IP} ${PCCS} ${LSPS} throughpcep_tools.log
343 ${total_size} = BuiltIn.Evaluate int(${size})+int(${LSPS})
344 Builtin.Set_Suite_Variable ${size} ${total_size}
346 Builtin.Wait_Until_Keyword_Succeeds 120s 1s Pcep_On
347 ${second_pcc_lsp_size} = BuiltIn.Evaluate int(${LSPS})
348 Builtin.Set_Suite_Variable ${size} ${second_pcc_lsp_size}
349 Updater 10 1 ${TOOLS_SYSTEM_IP}
350 Builtin.Set_Suite_Variable ${size} ${total_size}
352 SSHLibrary.Switch_Connection pccmock_toolsvm
353 SSHLibrary.Get_File ${LOG_PATH}/throughpcep_tools.log throughpcep_tools.log
354 BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
355 Kill all pcc mock simulator processes pccmock_toolsvm
356 [Teardown] Run Keywords BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt AND Kill all pcc mock simulator processes
359 [Documentation] Send ctrl+c to pcc-mock, see prompt again within timeout.
360 [Setup] Run_Even_When_Failing_Fast
361 SSHLibrary.Switch_Connection pccmock
362 BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
363 [Teardown] Run Keywords Kill all pcc mock simulator processes AND Builtin.Wait_Until_Keyword_Succeeds ${PCEP_READY_VERIFY_TIMEOUT} 5s Pcep_Off
365 PCEP Sessions Flapped with LSP updates
366 [Documentation] Flapping PCEP sessions and perform LSP updates within flapping
367 Run Keyword If '${USE_TOOLS_SYSTEM}' == 'True' BuiltIn.Pass Execution Pcc Mock should not run in ODL System
368 FOR ${devices} IN RANGE ${INIT_PCC_DEVICE_COUNT} ${MAX_PCC_DEVICE_COUNT+1} ${PCC_DEVICE_INCREMENT}
369 Flap Pcc Mock sessions continuously with LSP updates 127.1.0.0 ${devices} 150
371 [Teardown] Run Keywords Kill all pcc mock simulator processes AND BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
373 PCEP Sessions Flapped alongside LSP updates
374 [Documentation] Flapping PCEP sessions and perform LSP updates alongside flapping
375 Run Keyword If '${USE_TOOLS_SYSTEM}' == 'True' BuiltIn.Pass Execution Pcc Mock should not run in ODL System
376 FOR ${devices} IN RANGE ${INIT_PCC_DEVICE_COUNT} ${MAX_PCC_DEVICE_COUNT+1} ${PCC_DEVICE_INCREMENT}
377 Flap Pcc Mock sessions parallelly with LSP updates 127.1.0.0 ${devices} 150
378 BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
380 [Teardown] Run Keywords Kill all pcc mock simulator processes AND BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
383 [Documentation] Transfer pcc-mock output from pcc-mock VM to robot VM.
384 [Setup] Run_Even_When_Failing_Fast
385 SSHLibrary.Get_File ${LOG_PATH}/${LOG_NAME} ${LOG_NAME}
386 SSHLibrary.Get_File ${LOG_PATH}/serial_execution.log serial_execution.log
387 SSHLibrary.Get_File ${LOG_PATH}/throughpcep_parallel_Execution.log
389 Topology_Postcondition
390 [Documentation] Verify that within timeout, PCEP topology contains no PCCs again.
392 [Setup] Run_Even_When_Failing_Fast
393 Builtin.Wait_Until_Keyword_Succeeds 90s 5s Pcep_Off_Again
396 [Documentation] If requested, restore the old value if enabling TCP reuse was successful on Updater VM.
397 [Setup] Run_Even_When_Failing_Fast
398 BuiltIn.Pass_Execution_If not ${UPDATERVM_ENABLE_TCP_RW_REUSE} Manipulation of tcp_rw_reuse is not requested.
399 SSHLibrary.Switch_Connection updater
400 BuiltIn.Variable_Should_Exist ${tcp_rw_reuse}
401 ${out} ${rc} = SSHLibrary.Execute_Command sudo bash -c "echo ${tcp_rw_reuse} > /proc/sys/net/ipv4/tcp_tw_reuse" return_rc=True
402 BuiltIn.Should_Be_Equal ${rc} ${0}
406 FailFast.Do_Not_Fail_Fast_From_Now_On
407 Select_mock_machine_and_Download_PCC_Mock
409 Select_mock_machine_and_Download_PCC_Mock
414 [Documentation] Check the tools system variable and assigns the PCC Mock
415 Run Keyword If '${USE_TOOLS_SYSTEM}' == 'False' Run Keywords Pccmock_From_Odl_System AND Updater_From_Odl_System
416 BuiltIn.Set_Suite_Variable ${FIRST_PCC_IP} ${PCCMOCKVM_IP}
419 [Arguments] ${pcc_ip}=${PCCMOCKVM_IP} ${pcc_user}=${PCCMOCKVM_USER} ${pcc_password}=${PCCMOCKVM_PASSWORD} ${pcc_prompt}=${PCCMOCKVM_PROMPT} ${ssh_alias}=pccmock
420 [Documentation] SSH login to pcc-mock VM, download latest pcc-mock executable from Nexus.
421 BuiltIn.Run_Keyword_If ${PCCMOCK_COLOCATED} Pccmock_From_Controller
422 NexusKeywords.Initialize_Artifact_Deployment_And_Usage tools_system_connect=False
423 SSHLibrary.Open_Connection ${pcc_ip} alias=${ssh_alias}
424 SSHLibrary.Set_Client_Configuration timeout=10s
425 SSHLibrary.Set_Client_Configuration prompt=${pcc_prompt}
426 SSHKeywords.Flexible_SSH_Login ${pcc_user} ${pcc_password} delay=4s
427 ${file_name} = NexusKeywords.Deploy_Test_Tool bgpcep pcep-pcc-mock
428 BuiltIn.Set_Suite_Variable ${mock_location} ${file_name}
431 [Arguments] ${ssh_alias}=pccmock ${mock-ip}=${FIRST_PCC_IP} ${pccs}=${PCCS} ${lsps}=${LSPS} ${log_name}=${LOG_NAME}
432 [Documentation] Launch pcc-mock on background so simulated PCCs start connecting to controller.
433 SSHLibrary.Switch_Connection ${ssh_alias}
434 ${command} = NexusKeywords.Compose_Full_Java_Command -jar ${mock_location} --local-address ${mock-ip} --remote-address ${ODL_SYSTEM_IP} --pcc ${pccs} --lsp ${lsps} &> ${LOG_PATH}/${log_name}
435 BuiltIn.Log ${command}
436 SSHLibrary.Write ${command}
437 # The pccmock SSH session is left alive, but no data will be exchanged for a while.
438 # We need this connection to stay alive to send ctrl+c later.
439 # SSHLibrary.Start_Command would not do that for us.
441 Pccmock_From_Odl_System
442 [Documentation] Copy Odl_System values to Pccmock VM variables.
443 BuiltIn.Set_Suite_Variable ${PCCMOCKVM_IP} ${ODL_SYSTEM_IP}
444 BuiltIn.Set_Suite_Variable ${PCCMOCKVM_PASSWORD} ${ODL_SYSTEM_PASSWORD}
445 BuiltIn.Set_Suite_Variable ${PCCMOCKVM_PROMPT} ${ODL_SYSTEM_PROMPT}
446 BuiltIn.Set_Suite_Variable ${PCCMOCKVM_WORKSPACE} ${ODL_SYSTEM_WORKSPACE}
447 BuiltIn.Set_Suite_Variable ${LOG_PATH} ${ODL_SYSTEM_WORKSPACE}
449 Updater_From_Odl_System
450 [Documentation] Copy Odl_System values to Uprater VM variables.
451 BuiltIn.Set_Suite_Variable ${UPDATERVM_IP} ${ODL_SYSTEM_IP}
452 BuiltIn.Set_Suite_Variable ${UPDATERVM_PASSWORD} ${ODL_SYSTEM_PASSWORD}
453 BuiltIn.Set_Suite_Variable ${UPDATERVM_PROMPT} ${ODL_SYSTEM_PROMPT}
454 BuiltIn.Set_Suite_Variable ${UPDATERVM_WORKSPACE} ${ODL_SYSTEM_WORKSPACE}
457 [Documentation] Explicitly close all SSH connections.
458 SSHLibrary.Close_All_Connections
459 # TODO: Make AuthStandalone session object closable?
461 Get_Pcep_Topology_Data
462 [Documentation] Use session object to download PCEP topology JSON. Check status and return Response object.
463 ${resp} = AuthStandalone.Get_Using_Session ${rest_session} data/${TOPOLOGY_URL}\=pcep-topology?content\=nonconfig
464 # Not Logging content, as it may be huge.
465 BuiltIn.Should_Be_Equal ${resp.status_code} ${200}
468 Get_Pcep_Topology_Count
469 [Arguments] ${pattern}
470 [Documentation] Get topology data, return number of pattern matches.
471 ${resp} = Get_Pcep_Topology_Data
472 # BuiltIn.Log ${resp.text}
473 ${count} = BuiltIn.Evaluate len(re.findall('${pattern}', '''${resp.text}''')) modules=re
478 [Documentation] Get topology data, Log content and assert the exact JSON of empty topology.
479 ${resp} = Get_Pcep_Topology_Data
480 # Used before topology had chance to grow huge. Be aware when creating a longevity suite from this.
481 BuiltIn.Log ${resp.text}
482 BuiltIn.Should_Be_Equal ${resp.text} {"network-topology:topology":[{"topology-id":"pcep-topology","topology-types":{"network-topology-pcep:topology-pcep":{}}}]}
485 [Documentation] Get topology count of current hop, assert the number of matches.
486 # Suite variables ${size} and ${hop} are set elsewhere.
487 ${resp} = Get_Pcep_Topology_Count ${hop}
488 BuiltIn.Should_Be_Equal ${resp} ${size}
491 [Documentation] Get topology count of final hop, assert there is none.
492 ... This is more log friendly than Pcep_Off keyword, as it does not Log possibly large content.
494 ${resp} = Get_Pcep_Topology_Count ${hop}
495 BuiltIn.Should_Be_Equal ${resp} ${0}
498 [Arguments] ${iteration}
499 [Documentation] Set pattern to match the currently expected hop.
500 ${i} = BuiltIn.Evaluate str(1 + int(${iteration}))
501 # Regular Expressions need a dot to be escaped to represent a dot.
502 BuiltIn.Set_Suite_Variable ${hop} ${i}\.${i}\.${i}\.${i}/32
506 [Arguments] ${iteration} ${workers} ${mock-ip}=${FIRST_PCC_IP} ${pccs}=${PCCS} ${lsps}=${LSPS} ${parallel}=False ${delegate}=true ${pccip}=${none} ${tunnel_no}=${none}
507 [Documentation] Compute number of workers, call updater.py, assert its response.
508 SSHLibrary.Switch_Connection pccmock
509 # In some systems, inactive SSH sessions get severed.
510 ${command} = BuiltIn.Set_Variable echo "still alive"
511 ${output} = SSHLibrary.Execute_Command bash -c '${command}'
512 # The previous line relies on a fact that Execute_Command spawns separate shels, so running pcc-mock is not affected.
514 SSHLibrary.Switch_Connection updater
515 ${response} = SSHLibrary.Execute_Command bash -c "cd ${UPDATERVM_WORKSPACE}; taskset 0x00000001 python3 updater.py --workers '${workers}' --odladdress '${UPDATER_ODLADDRESS}' --user '${RESTCONF_USER}' --password '${RESTCONF_PASSWORD}' --scope '${RESTCONF_SCOPE}' --pccaddress '${mock-ip}' --pccs '${pccs}' --lsps '${lsps}' --hop '${hop}' --timeout '${UPDATER_TIMEOUT}' --refresh '${UPDATER_REFRESH}' --reuse '${RESTCONF_REUSE}' --delegate '${delegate}' --pccip '${pccip}' --tunnelnumber '${tunnel_no}' 2>&1"
516 Check Updater response ${response} ${parallel}
518 Check Updater response
519 [Arguments] ${response} ${parallel}
520 BuiltIn.Log ${response}
521 ${expected_value_continuous_execution} = BuiltIn.Set_Variable Counter({'pass': ${size}})
522 ${not_expected_value_for_parallel_execution} = BuiltIn.Set_Variable Counter({'pass': 0})
523 BuiltIn.Log ${expected_value_continuous_execution}
524 Run Keyword If '${parallel}' == 'False' BuiltIn.Should_Contain ${response} ${expected_value_continuous_execution}
525 ... ELSE BuiltIn.Should_Not_Contain ${response} ${not_expected_value_for_parallel_execution}
528 [Arguments] ${iteration}
529 [Documentation] Set hop and verify that within timeout, all LSPs in topology are updated.
531 Builtin.Wait_Until_Keyword_Succeeds 30s 1s Pcep_On
533 Flap Pcc Mock sessions continuously with LSP updates
534 [Arguments] ${mock-ip}=${FIRST_PCC_IP} ${pccs}=${PCCS} ${lsps}=${LSPS}
535 ${localsize} = Evaluate int(${pccs})*int(${lsps})
536 Builtin.Set_Suite_Variable ${size} ${localsize}
537 ${workers} = Set Variable ${BLOCKING-THREAD}
538 FOR ${i} IN RANGE ${SEQUENTIAL_ITERATION}
539 ${workers} = Evaluate ${workers}*${workers}
541 Builtin.Wait_Until_Keyword_Succeeds ${PCEP_READY_VERIFY_TIMEOUT} 5s Pcep_Off
542 Start_Pcc_Mock pccmock ${mock-ip} ${pccs} ${lsps} serial_execution.log
543 Builtin.Wait_Until_Keyword_Succeeds 60s 5s Pcep_On
544 ${i} = Evaluate ${i}+1
545 Updater ${i} ${workers} 127.1.0.0 ${pccs} ${lsps}
547 SSHLibrary.Switch_Connection pccmock
548 BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
552 Flap Pcc Mock sessions parallelly with LSP updates
553 [Arguments] ${mock-ip}=${FIRST_PCC_IP} ${pccs}=${PCCS} ${lsps}=${LSPS}
554 SSHLibrary.Switch_Connection pccmock
555 SSHLibrary.Put File ${CURDIR}/../../../../tools/pcep_updater/mock.sh /tmp/mock.sh
557 Builtin.Wait_Until_Keyword_Succeeds ${PCEP_READY_VERIFY_TIMEOUT} 5s Pcep_Off
558 SSHLibrary.Start Command sh /tmp/mock.sh ${mock_location} ${mock-ip} ${ODL_SYSTEM_IP} ${pccs} ${lsps} parallel_Execution ${DELAY_TIME} &>1
559 FOR ${i} IN RANGE ${PARALLEL_ITERATION}
560 ${pid} = SSHLibrary.Execute Command ps -fu ${ODL_SYSTEM_USER} | grep "/home/${ODL_SYSTEM_USER}/${mock_location}" | grep -v "grep" | awk '{print $2}'
561 Run Keyword If '${pid}'!= "" Log ${pid}
562 ${i} = Evaluate ${i}+1
563 Run Keyword If '${pid}'!= "" Updater ${i} 1 127.1.0.0 ${pccs} ${lsps} True
565 BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
566 Kill all pcc mock simulator processes
570 [Documentation] Check PCEP topology with default pcc and lsp values
571 ${localsize} = Evaluate int(${PCCS})*int(${LSPS})
572 Builtin.Set_Suite_Variable ${size} ${localsize}
573 Builtin.Wait_Until_Keyword_Succeeds 90s 5s Pcep_Off_Again
575 Builtin.Wait_Until_Keyword_Succeeds 60s 5s Pcep_On
578 SSHLibrary.Switch_Connection pccmock
579 BGPcliKeywords.Stop_Console_Tool_And_Wait_Until_Prompt
581 Kill all pcc mock simulator processes
582 [Arguments] ${ssh_alias}=pccmock
583 SSHLibrary.Switch_Connection ${ssh_alias}
584 ${mock_pid} Get pid /home/${ODL_SYSTEM_USER}/${mock_location}
585 SSHLibrary.Execute_Command kill -9 ${mock_pid}
586 ${script_pid_1} Get pid bash -c sh /tmp/mock.sh
587 SSHLibrary.Execute_Command kill -9 ${script_pid_1}
588 ${script_pid_2} Get pid sh /tmp/mock.sh
589 SSHLibrary.Execute_Command kill -9 ${script_pid_2}
592 [Arguments] ${process_name}
593 ${pid} = SSHLibrary.Execute Command ps -fu ${ODL_SYSTEM_USER} | grep "${process_name}" | grep -v "grep" | awk '{print $2}'