Testplan for Bundle-Reysnc

This document is the testplan for the bundle-based Resync Feature

Change-Id: I8b60a7b804f78d94dabb0832d6d3a9b9f1ae95b4
Signed-off-by: eafatha <a.fathima.thasneem@ericsson.com>

diff --git a/docs/index.rst b/docs/index.rst
index fbe465f2f902283448ca734bfc7f01d74e0c56af..f2da95c3b62e1e17f59a71d3b8d3f4fd384279a3 100644 (file)
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -13,4 +13,5 @@ Contents:
 .. toctree::
    :maxdepth: 2
 
-   specs/index
\ No newline at end of file
+   specs/index
+   testplans/index

diff --git a/docs/testplans/Bundles-Resync.rst b/docs/testplans/Bundles-Resync.rst
new file mode 100644 (file)
index 0000000..18b1dcb
--- /dev/null
+++ b/docs/testplans/Bundles-Resync.rst
@@ -0,0 +1,237 @@
+.. contents:: Table of Contents
+      :depth: 3
+
+==============
+Bundles-Resync
+==============
+Test Suite for testing Bundles-Reconciliation functionality.
+
+Test Setup
+==========
+Test setup consists of ODL and two switches(Openflow nodes) connected
+to ODL via OpenflowPlugin Channel (6653).
+
+Testbed Topologies
+------------------
+This suit uses the default topology.
+
+Default Topology
+^^^^^^^^^^^^^^^^
+
+.. literalinclude:: topologies/default-topology.txt
+
+
+Hardware Requirements
+---------------------
+N.A.
+
+Software Requirements
+---------------------
+OVS 2.6+
+
+Test Suite Requirements
+=======================
+
+Test Suite Bringup
+------------------
+Following steps are followed at beginning of test suite:
+
+* Bring up ODL with `odl-openflowplugin-flow-services-rest` installed.
+* Add bridge br-int to openflow node
+* Connect bridge to OpenFlow using `ovs-vsctl set-controller`
+* Repeat above steps for other openflow nodes
+
+
+Test Suite Cleanup
+------------------
+Following steps are followed at the end of test suite:
+
+* Delete bridge br-int on openflow node
+* Repeat the same for other openflow nodes
+
+Debugging
+---------
+Following DataStore models are captured at end of each test case:
+
+* config/opendaylight-inventory:nodes
+* operational/opendaylight-inventory:nodes
+
+Test Cases
+==========
+Testcases covered in automation:
+
+* Verify the Bundle based reconciliation with switch(OVS) restart scenario
+* Verify the Bundle based reconciliation by pushing group dependent flow with switch(OVS) restart scenario
+* Verify the Bundle Based reconciliation by connecting a new switch(OVS)
+
+Verify the default reconciliation
+---------------------------------
+This Verifies the default reconciliation (bundle-based-reconciliation-enabled=false)
+
+Test Steps and Pass Criteria
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Bring up the Controller.
+#. Set the switch fail mode to Secure.
+#. Push flow via Rest call and add a flow in the ovs-switch via ovs-ofctl add-flow
+#. Restart the switch
+#. Check that the flows pushed via Rest call should be present and the static flow added
+#. Capture via Wireshark and check that the OFPT_EXP messages are not captured.
+#. Check in the karaf.log and confirm if EXP messages are not logged
+#. Flap the management interface of the switch.
+#. Also ensure that the static flow added would be present.
+#. Capture via Wireshark and check that the OFPT_EXP messages are not captured.
+#. Check in the karaf.log and confirm if EXP messages are not logged
+
+
+   #. Verify the Test Procedure.
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+N.A.
+
+
+Verify the Bundle based reconciliation by enabling the flag to True
+-------------------------------------------------------------------
+The Objective of this Testcase is to check the bundle based resync mechanism by enabling the flag
+
+Test Steps and Pass Criteria
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Bring up the Controller.
+#. Set the bundle-based-reconciliation-enabled=true.
+#. Check if the flag set event is logged in karaf.log.
+#. Set the bundle-based-reconciliation-enabled=false.
+#. Check if the flag set event is logged in karaf.log.
+
+
+   #. Verify the Test steps.
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+N.A.
+
+Verify the Bundle based reconciliation with switch(OVS) restart scenario
+------------------------------------------------------------------------
+The Objective of this Testcase to verify bundle based reconciliation with ovs restart scenario.
+
+Test Steps and Pass Criteria
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Bring up the Controller.
+#. Set the bundle-based-reconciliation-enabled=true.
+#. Push flow via Rest call and add a flow in the ovs-switch via ovs-ofctl add-flow
+#. Set the Switch fail-mode set to secure.
+#. Check if the flag set event is logged in karaf.log.
+#. Restart the switch
+#. Check if the Wireshark has the OFPT_EXP messages captured.
+#. Check for the same messages to be logged in the karaf.log.
+#. Repeat the same with fail-mode set to stand-alone
+#. Static flow should not be present in both stand-alone and secure mode as the switch is restarted.
+
+
+   #. Verify the Test steps.
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+N.A.
+
+Verify the Bundle based reconciliation by pushing group dependent flow with switch(OVS) restart scenario
+--------------------------------------------------------------------------------------------------------
+The Objective of this Testcase to verify bundle based reconciliation with ovs restart scenario.
+
+Test Steps and Pass Criteria
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Bring up the Controller.
+#. Set the bundle-based-reconciliation-enabled=true.
+#. Push flow via Rest call and add a flow in the ovs-switch via ovs-ofctl add-flow
+#. Set the Switch fail-mode set to secure.
+#. Check if the flag set event is logged in karaf.log.
+#. Restart the switch
+#. Check if the Wireshark has the OFPT_EXP messages captured.
+#. Check for the same messages to be logged in the karaf.log.
+#. Repeat the same with fail-mode set to stand-alone
+#. Static flow should not be present in both stand-alone and secure mode as the switch is restarted.
+
+
+   #. Verify the Test steps.
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+N.A.
+
+Verify the Bundle Based reconciliation by connecting a new switch(OVS)
+----------------------------------------------------------------------
+The Objective of this Testcase to verify the bundle based reconciliation by connecting a new switch to the controller.
+
+Test Steps and Pass Criteria
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Bring up the Controller.
+#. Set the bundle-based-reconciliation-enabled=true
+#. Push group dependent flow via Rest call and add a flow in the ovs-switch via ovs-ofctl add-flow
+#. Set the Switch fail-mode set to secure.
+#. Check if the flag set event is logged in karaf.log.
+#. Check if the pushed flows are there in the OVS.
+#. Get a new switch connected to the Controller.
+#. Push flow via Rest call and add a flow in the ovs-switch via ovs-ofctl add-flow to the newly added switch
+#. Flap the management interface of the new switch.
+#. Ensure the flows are pushed via bundles to the new switch.
+#. Flows remain intact in the switch that was already connected.
+
+   #. Verify the Test Steps
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+N.A.
+
+Verify the Bundle based reconciliation by killing the OVS Switch Process
+------------------------------------------------------------------------
+The Objective of this Testcase to verify the bundle based reconciliation by killing the ovs switch.
+
+Test Steps and Pass Criteria
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. Bring up the Controller.
+#. Set the bundle-based-reconciliation-enabled=true.
+#. Push flow via Rest call and add a flow in the ovs-switch via ovs-ofctl add-flow
+#. Set the Switch fail-mode set to secure.
+#. Check if the flag set event is logged in karaf.log.
+#. Kill the OVS Switch process
+#. Check if the Wireshark has the OFPT_EXP messages captured.
+#. Check for the same messages to be logged in the karaf.log.
+#. Repeat the same with fail-mode set to stand-alone
+#. Static flow should not be present in both stand-alone and secure mode.
+
+
+   #. Verify the Test Steps
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+N.A.
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+
+Primary assignee:
+  Fathima Thasneem  (a.fathima.thasneem@ericsson.com)
+
+Other contributors:
+  N.A.
+
+Work Items
+----------
+N.A.
+
+Links
+-----
+
+* https://git.opendaylight.org/gerrit/#/c/68364/
+* Script path test/csit/suites/openflowplugin/Bundlebased_Reconciliation/010_bundle_resync.robot
+
+References
+==========

diff --git a/docs/testplans/index.rst b/docs/testplans/index.rst
new file mode 100644 (file)
index 0000000..5874c0e
--- /dev/null
+++ b/docs/testplans/index.rst
@@ -0,0 +1,13 @@
+Openflowplugin Test Plans
+=========================
+
+Starting from Nitrogen, Openflowplugin uses RST format Test Plan document for
+all new Test Suites.
+
+Contents:
+
+.. toctree::
+   :maxdepth: 1
+
+   Bundles Resync <Bundles-Resync>
+   Testplan Template <testplan-template>

diff --git a/docs/testplans/testplan-template.rst b/docs/testplans/testplan-template.rst
new file mode 100644 (file)
index 0000000..542412b
--- /dev/null
+++ b/docs/testplans/testplan-template.rst
@@ -0,0 +1,156 @@
+..
+ Key points to consider:
+  * Use RST format. For help with syntax refer http://sphinx-doc.org/rest.html
+  * Use http://rst.ninjs.org/ a web based WYSIWYG RST editor.
+  * For diagrams, you can use http://asciiflow.com to make ascii diagrams.
+  * MUST READ http://docs.opendaylight.org/en/latest/documentation.html and follow guidelines.
+  * All sections should be retained, but can be marked None or N.A.
+  * Set depth in ToC as per your doc requirements. Should be at least 2.
+
+.. contents:: Table of Contents
+      :depth: 3
+
+===================
+Title of Test Suite
+===================
+
+Brief introduction of the Test Suite and feature it is testing. This should include
+links to relevant documents and specs.
+
+.. note::
+
+  Name of suite and test cases should map exactly to as they appear in Robot reports.
+
+
+Test Setup
+==========
+
+Brief description of test setup.
+
+Testbed Topologies
+------------------
+
+Detailed information on testbed topologies, including topology diagrams. Each
+should be numbered so it can be referenced by number in Test Cases.
+
+Default Topology
+^^^^^^^^^^^^^^^^
+
+.. literalinclude:: topologies/dummy-topology.txt
+
+
+Hardware Requirements
+---------------------
+
+Any specific hardware requirements e.g. SRIOV NICs.
+
+Software Requirements
+---------------------
+
+Any specific software and version requirements e.g. Mininet, OVS 2.8 etc. This
+should also capture specific versions of OpenDaylight this suit applies to. e.g.
+Nitrogen, Nitrogen-SR2 etc. This will be used to determine to which jobs this suite
+can/should be added.
+
+Test Suite Requirements
+=======================
+
+Test Suite Bringup
+------------------
+
+Initial steps before any tests are run. This should include any cleanup, sanity checks,
+configuration etc. that needs to be done before test cases in suite are run. This should
+also capture if this suite depends on another suite to do bringup for it.
+
+Test Suite Cleanup
+------------------
+
+Final steps after all tests in suite are done. This should include any cleanup, sanity checks,
+configuration etc. that needs to be done once all test cases in suite are done.
+
+Debugging
+---------
+
+Capture any debugging information that is captured at start of suite and end of suite.
+
+
+Test Cases
+==========
+This section should capture high level details about all the test cases part of the suite.
+Individual test cases should be subsections in the order they should be executed.
+
+Test Case 1
+-----------
+Give a brief description of the test case including topology used if multiple specified
+in `Testbed Topologies`_.
+
+Test Steps and Pass Criteria
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Step by step procedure of what is done as part of this test.
+
+#. Step 1
+
+   #. Pass Criteria 1
+   #. Pass Criteria 2
+
+#. Step 2
+
+   #. Pass Criteria 1
+   #. Pass Criteria 2
+
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+Any test specific information captured. Specifically mention if it is captured always,
+pass only or fail only.
+
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+Who is contributing test cases? In case of multiple authors, designate a
+primary assignee and other contributors. Primary assignee is also expected to
+be maintainer once test code is in.
+
+Primary assignee:
+  <developer-a>
+
+Other contributors:
+  <developer-b>
+  <developer-c>
+
+
+Work Items
+----------
+Break up work into individual items. For most cases it would be just writing tests
+but in some cases will include changes to images, infra etc.
+
+
+Links
+-----
+
+* Link to implementation patche(s) in CSIT
+
+
+References
+==========
+Add any useful references. Some examples:
+
+* Links to Summit presentation, discussion etc.
+* Links to mail list discussions
+* Links to patches in other projects
+* Links to external documentation
+
+[1] `OpenDaylight Documentation Guide <http://docs.opendaylight.org/en/latest/documentation.html>`__
+
+[2] https://specs.openstack.org/openstack/nova-specs/specs/kilo/template.html
+
+.. note::
+
+  This template was derived from [2], and has been modified to support our project.
+
+  This work is licensed under a Creative Commons Attribution 3.0 Unported License.
+  http://creativecommons.org/licenses/by/3.0/legalcode
+

diff --git a/docs/testplans/topologies/default-topology.txt b/docs/testplans/topologies/default-topology.txt
new file mode 100644 (file)
index 0000000..5e9cb0e
--- /dev/null
+++ b/docs/testplans/topologies/default-topology.txt
@@ -0,0 +1,12 @@
++--------+       +--------+
+|  BR1   | data  |  BR2   |
+|        <------->        |
++---^----+       +----^---+
+    |       mgmt      |
++---v-----------------v---+
+|                         |
+|           ODL           |
+|                         |
+|  Bundle Resync enabled  |
+|                         |
++-------------------------+

diff --git a/docs/testplans/topologies/dummy-topology.txt b/docs/testplans/topologies/dummy-topology.txt
new file mode 100644 (file)
index 0000000..6f84af7
--- /dev/null
+++ b/docs/testplans/topologies/dummy-topology.txt
@@ -0,0 +1,4 @@
++------------+
+|   Dummy    |
+|  Topology  |
++------------+