--- /dev/null
+
+Draft, outdated: Carbon SR1 test report
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Test Case Summary
+-----------------
+
+RelEng stability summary.
+
++ tba: Recent failures to be analyzed yet: 0.
++ test: Recent failures caused by wrong assumptions in test: 0.
++ akka: Recent failures related to pure UnreachableMember: 5.
++ tell: Recent failures not clearly caused by UnreachableMember: 9.
++ few: Tests passing unless low frequency failure happens: 22 (21 without duplication).
+ (Low frequency means UnreachableMemeber or "Message was not delivered, dead letters encountered",
+ both are related to Akka where Controller code has not real control.)
++ pass: Tests passing consistently: 17 (15 without duplication).
++ Total: 53 (50 without duplication).
++ Total minus akka: 48 (45 without duplication).
++ Total minus akka passing always or mostly: 39 (36 without duplication).
++ Acceptance rate: 39/48=81.25% (36/45=80.00% without duplication).
+
+Table
+-----
+
+S017 instead of 2017 means Sandbox run (includes changes not merged to stable/carbon yet).
+
+Last fail is date of last failure not caused by infra
+(or by a typo in test or by netconf/bgp failing to initialize properly).
+
+"S 17" or "2 17" in Last run means the documented run was superseded by a newer one, but not analyzed yet.
+
+"long ago" means the last real test failue happened before around 2017-05-19, or never.
+
+.. table:: Releng stability results (pre-SR1)
+ :widths: 30,10,20,20,10,10
+
+ ================== ===== ========== ========== ============================================================= ==========
+ Scenario name Type Last fail Last run Bugs Robot link
+ ================== ===== ========== ========== ============================================================= ==========
+ bgp-1n-1m-a pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-periodic-bgp-ingest-only-carbon/lastSuccessfulBuild/robot/bgpcep-bgp-ingest.txt/Singlepeer%20Prefixcount/>`__
+ bgp-1n-300k-t pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-periodic-bgp-ingest-only-carbon/lastSuccessfulBuild/robot/bgpcep-bgp-ingest.txt/Singlepeer%20Pc%20Shm%20300Kroutes_1/>`__
+ bgp-3n-300k-ll-t akka 2017-07-14 2017-07-14 `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__ `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/343/log.html.gz#s1-s2-t8-k2-k3-k7-k5-k1-k6-k1-k1-k1-k1-k1-k2-k1-k4>`__
+ bgp-3n-300k-lr-t akka 2017-07-13 2017-07-14 `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__ `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/343/log.html.gz#s1-s4-t8-k2-k3-k7-k7-k1-k6-k1-k1-k1-k1-k1-k2-k1-k2-k4>`__
+ ddb-cls-ms-ll-t few 2017-07-04 2017-07-15 `8794 <https://bugs.opendaylight.org/show_bug.cgi?id=8794>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/771/log.html.gz#s1-s20-t1-k2-k8>`__
+ ddb-cls-ms-lr-t few 2017-07-08 2017-07-15 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/776/log.html.gz#s1-s20-t3-k2-k8>`__
+ ddb-cls-ps-ll-t few 2017-07-09 2017-07-15 `8794 <https://bugs.opendaylight.org/show_bug.cgi?id=8794>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/777/log.html.gz#s1-s22-t1-k2-k8>`__
+ ddb-cls-ps-lr-t pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Clean%20Leader%20Shutdown%20Prefbasedshard/Remote_Leader_Shutdown/>`__
+ ddb-elm-ms-lr-t few 2017-06-13 2017-07-15 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/31/log.html.gz#s1-s24-t1-k2-k10>`__
+ ddb-elm-ms-rr-t few 2017-06-10 2017-07-15 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/738/log.html.gz#s1-s24-t3-k2-k10>`__
+ ddb-elm-ms-rl-t few 2017-06-27 2017-07-15 `8749 <https://bugs.opendaylight.org/show_bug.cgi?id=8749>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/763/log.html.gz#s1-s24-t5-k2-k10>`__
+ ddb-elm-ps-lr-t few 2017-06-11 2017-07-15 `8664 <https://bugs.opendaylight.org/show_bug.cgi?id=8664>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s26-t1-k2-k6-k3-k1-k4-k7-k1>`__
+ ddb-elm-ps-rr-t pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s26-t3>`__
+ ddb-elm-ps-rl-t few 2017-06-07 2017-07-15 `8403 <https://bugs.opendaylight.org/show_bug.cgi?id=8403>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/735/log.html.gz#s1-s26-t5-k2-k9>`__
+ ddb-li-ms-st-t tell 2017-07-15 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s28-t1-k2-k25-k1-k1>`__
+ ddb-li-ms-dt-t tell 2017-07-15 2017-07-15 `8619 <https://bugs.opendaylight.org/show_bug.cgi?id=8619>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s28-t3-k2-k25-k1-k8>`__
+ ddb-li-ps-st-t few 2017-06-08 2017-07-15 `8371 <https://bugs.opendaylight.org/show_bug.cgi?id=8371>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s30-t1-k2-k25-k1-k1>`__
+ ddb-li-ps-dt-t tell 2017-07-15 2017-07-15 `8845 <https://bugs.opendaylight.org/show_bug.cgi?id=8845>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s30-t3-k2-k25-k1-k8>`__
+ ddb-ci-ms-ll-ct-t few 2017-06-07 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/735/log.html.gz#s1-s32-t1-k2-k16-k1-k1>`__
+ ddb-ci-ms-ll-st-t tell 2017-07-15 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s32-t3-k2-k16-k1-k1>`__
+ ddb-ci-ms-lr-ct-t few 2017-06-08 2017-07-15 `8636 <https://bugs.opendaylight.org/show_bug.cgi?id=8636>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s32-t5-k2-k15-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1>`__
+ ddb-ci-ms-lr-st-t tell 2017-07-15 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s32-t7-k2-k16-k1-k1>`__
+ ddb-ci-ps-ll-ct-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t1-k2-k16-k1-k1>`__
+ ddb-ci-ps-ll-st-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t3-k2-k16-k1-k1>`__
+ ddb-ci-ps-lr-ct-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t5-k2-k16-k1-k1>`__
+ ddb-ci-ps-lr-st-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t7-k2-k16-k1-k1>`__
+ ddb-ls-ms-lr-t tell 2017-07-15 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s36-t1-k2-k12>`__
+ ddb-ls-ms-rr-t tell 2017-07-14 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/782/log.html.gz#s1-s36-t3-k2-k12>`__
+ ddb-ls-ms-rl-t tell 2017-07-12 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/779/log.html.gz#s1-s36-t5-k2-k12>`__
+ ddb-ls-ps-lr-t pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Listener%20Stability%20Prefbasedshard/Move_Leader_From_Listener_Local_To_Remote/>`__
+ ddb-ls-ps-rr-t few 2017-06-26 2017-07-15 `8733 <https://bugs.opendaylight.org/show_bug.cgi?id=8733>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/762/log.html.gz#s1-s38-t3-k2-k14-k2-k1-k4-k7-k1>`__
+ ddb-ls-ps-rl-t pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Listener%20Stability%20Prefbasedshard/Move_Leader_From_Listener_Remote_To_Local/>`__
+ drb-rpp-ms-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Rpc%20Provider%20Precedence/>`__
+ drb-rph-ms-a few 2017-06-28 2017-07-15 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s4-t6-k2-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1-k3-k1-k1-k1-k2-k1-k4-k7-k1>`__
+ drb-app-ms-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Action%20Provider%20Precedence/>`__
+ drb-aph-ms-a few 2017-07-02 2017-07-15 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/769/log.html.gz#s1-s8-t6-k2-k1-k1-k1-k1-k1-k1-k1-k1-k1-k1-k3-k1-k1-k1-k3-k1-k4-k7-k1>`__
+ dnb-1n-60k-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/lastSuccessfulBuild/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__
+ ss-ms-ms-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Master%20Stability/>`__
+ ss-ph-ms-a few 2017-06-29 2017-07-15 `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/765/log.html.gz#s1-s12-t5-k2-k3-k1-k2>`__
+ ss-cl-ms-a pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s14>`__
+ ss-ms-ms-t pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s40>`__
+ ss-ph-ms-t few 2017-07-15 2017-07-15 `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s42-t5-k2-k3-k1-k2>`__
+ ss-cl-ms-t pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s44>`__
+ netconf-ba-ms-a pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/netconf/job/netconf-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/netconf-clustering.txt/CRUD>`__
+ netconf-ok-ms-a few 2017-06-18 2017-07-14 `8596 <https://bugs.opendaylight.org/show_bug.cgi?id=8596>`__ `link <https://logs.opendaylight.org/releng/jenkins092/netconf-csit-3node-clustering-only-carbon/568/log.html.gz#s1-s5-t17-k2-k3-k2-k2-k1>`__
+ netconf-rr-ms-a pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/netconf/job/netconf-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/netconf-clustering.txt/Outages>`__
+ bgp-3n-300k-t-long akka 2017-07-08 2017-07-08 `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__ `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-bgpclustering-longevity-only-carbon/11/log.html.gz#s1-s2-t1-k10-k1-k1-k1-k1-k1-k1-k1-k1-k1-k2-k1-k3-k7-k5-k1-k6-k1-k1-k1-k1-k1-k2-k1-k1-k2-k2-k2-k1-k6-k2-k2-k1>`__
+ ddb-elm-mc-t-long tell 2017-07-08 2017-07-08 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-ddb-expl-lead-movement-longevity-only-carbon/14/log.html.gz#s1-s2-t1-k2-k1-k1-k1-k1-k1-k1-k2-k1-k1-k2-k10>`__
+ drb-rpp-ms-a-long few 2017-05-07 2017-07-08 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-drb-partnheal-longevity-only-carbon/13/console.log.gz>`__
+ drb-rph-ms-a-long akka 2017-07-08 2017-07-08 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-drb-partnheal-longevity-only-carbon/18/log.html.gz#s1-t1-k3-k1-k1-k1-k1-k1-k1-k2-k1-k1-k6-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1-k3-k1-k1-k1-k2-k1-k4-k7-k1>`__
+ dnb-1n-60k-a-long pass long ago 2017-07-08 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/620/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__
+ ss-ph-ms-a-long akka 2017-07-08 2017-07-08 `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-cs-partnheal-longevity-only-carbon/15/log.html.gz#s1-s2-t1-k3-k1-k1-k1-k1-k1-k1-k1-k1-k1-k7-k3-k1-k2>`__
+ ss-cl-ms-a-long pass long ago 2017-07-08 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/620/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__
+ ================== ===== ========== ========== ============================================================= ==========
+
+For descriptions of test cases, see `description page <tests.html>`_.
+Note that the link contains current description,
+the details might have been implemented differently at SR1 release.
Code Review / integration / test.git / blobdiff