*** Settings ***
Documentation PCEP performance suite, uses restconf with configurable authentication.
-...
+...
... Copyright (c) 2015 Cisco Systems, Inc. and others. All rights reserved.
-...
+...
... This program and the accompanying materials are made available under the
... terms of the Eclipse Public License v1.0 which accompanies this distribution,
... and is available at http://www.eclipse.org/legal/epl-v10.html
-...
-...
+...
+...
... General Overview:
-...
+...
... This is a suite which has both scale and performance aspects.
... Given scale target, suite reports failures if functional error
... is detected, or if various time limits expire.
... For passing test cases, their duration is the performance metric.
-...
+...
... ODL acts as a translation layer between PCEP capable devices
... and users employing RESTCONF.
... Performance measurement focuses on two different workflows.
-...
+...
... The first workflow is initial synchronization, when ODL learns
... the state of PCEP topology as devices connect to it,
... while restconf user reads the state repeatedly.
... The second workflow is mass update, when restconf users issue RPCs
... to updale Layer Switched Paths on Path Computation Clients.
-...
+...
... This suite uses pcc-mock (downloaded from Nexus) to simulate PCCs.
... It needs segment of bindable IP addresses,
... one for each simulated PCC; so running pcc-mock from remote machine
... is only viable when just single PCC is simulated.
... Testing with multiple PCCs works best when pcc-mock
... runs on the same VM as ODL, so 127.0.0.0/8 subnet can be used.
-...
+...
... Library AuthStandalone is used directly for restconf reads
... in the first workflow. That library transparently handles several
... http authentication methods, based on credentials and pybot arguments.
-...
+...
... In the second workflow, updater.py utility is used for issuing
... rapid restconf requests. It can use multiple worker threads,
... as http requests are blocking.
... being more limiting factor than CPU).
... This suite starts updater utility bound to single CPU,
... as this setup was the most performant in other tests.
-...
+...
... In case of failed test case, other tests are skipped (unless
... this is overriden by [Setup]) to finish test run sooner.
-...
+...
... Variables and test case names refer to Controller(ODL_SYSTEM) and Mininet
... (TOOLS_SYSTEM), those are assumed to be separate remote VMs, one to host ODL,
... other to host tools.
... If both updater VM and pcc-mock VM parameters are specified,
... Mininet(TOOLS_SYSTEM) parameters may be skipped.
... Variable ${USE_TOOLS_SYSTEM} decides the pcc-mock running machine.
-...
+...
... Some launch scripts put restrictions on how pybot options
... can be specified, so there are utility variables to help with
... copying Controller related value to apply fo updater of pccmock.
... Having a tool co-located with ODL reduces network latency,
... but puts more pressure on CPU and memory on Controller VM.
-...
+...
... In some environments, issues with TIME-WAIT prevent high restconf rates,
... so TCP reuse is temporarily allowed during the suite run, if possible
... (and if not disabled by UPDATERVM_ENABLE_TCP_RW_REUSE option value).
... See http://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux.html
... This suite ignores possible failures when changing reuse.
-...
+...
... Similarly, in some environments, handling of requests.Session object matters
... try changing RESTCONF_REUSE value to see if it helps.
-...
+...
... Variables to override (only if needed) in pybot command:
... (Look into Variables table to see the default values.)
-...
+...
... FIRST_PCC_IP: Set in case bind address is different from public pcc-mock VM address.
... LOG_NAME: Filename (without path) to save pcc-mock output into.
... LOG_PATH: Override if not the same as pccmock VM workspace.
Code Review / integration / test.git / blobdiff