docs/cluster/carbon/sr1.rst

   1
   2 Draft, outdated: Carbon SR1 test report
   3 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   4
   5 Test Case Summary
   6 -----------------
   7
   8 RelEng stability summary.
   9
  10 + tba: Recent failures to be analyzed yet: 0.
  11 + test: Recent failures caused by wrong assumptions in test: 0.
  12 + akka: Recent failures related to pure UnreachableMember: 5.
  13 + tell: Recent failures not clearly caused by UnreachableMember: 9.
  14 + few: Tests passing unless low frequency failure happens: 22 (21 without duplication).
  15   (Low frequency means UnreachableMemeber or "Message was not delivered, dead letters encountered",
  16   both are related to Akka where Controller code has not real control.)
  17 + pass: Tests passing consistently: 17 (15 without duplication).
  18 + Total: 53 (50 without duplication).
  19 + Total minus akka: 48 (45 without duplication).
  20 + Total minus akka passing always or mostly: 39 (36 without duplication).
  21 + Acceptance rate: 39/48=81.25% (36/45=80.00% without duplication).
  22
  23 Table
  24 -----
  25
  26 S017 instead of 2017 means Sandbox run (includes changes not merged to stable/carbon yet).
  27
  28 Last fail is date of last failure not caused by infra
  29 (or by a typo in test or by netconf/bgp failing to initialize properly).
  30
  31 "S 17" or "2 17" in Last run means the documented run was superseded by a newer one, but not analyzed yet.
  32
  33 "long ago" means the last real test failue happened before around 2017-05-19, or never.
  34
  35 .. table:: Releng stability results (pre-SR1)
  36    :widths: 30,10,20,20,10,10
  37
  38    ==================  =====  ==========  ==========  =============================================================  ==========
  39    Scenario name       Type   Last fail   Last run    Bugs                                                           Robot link
  40    ==================  =====  ==========  ==========  =============================================================  ==========
  41    bgp-1n-1m-a         pass   long ago    2017-07-14                                                                 `link <https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-periodic-bgp-ingest-only-carbon/lastSuccessfulBuild/robot/bgpcep-bgp-ingest.txt/Singlepeer%20Prefixcount/>`__
  42    bgp-1n-300k-t       pass   long ago    2017-07-14                                                                 `link <https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-periodic-bgp-ingest-only-carbon/lastSuccessfulBuild/robot/bgpcep-bgp-ingest.txt/Singlepeer%20Pc%20Shm%20300Kroutes_1/>`__
  43    bgp-3n-300k-ll-t    akka   2017-07-14  2017-07-14  `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__  `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/343/log.html.gz#s1-s2-t8-k2-k3-k7-k5-k1-k6-k1-k1-k1-k1-k1-k2-k1-k4>`__
  44    bgp-3n-300k-lr-t    akka   2017-07-13  2017-07-14  `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__  `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/343/log.html.gz#s1-s4-t8-k2-k3-k7-k7-k1-k6-k1-k1-k1-k1-k1-k2-k1-k2-k4>`__
  45    ddb-cls-ms-ll-t     few    2017-07-04  2017-07-15  `8794 <https://bugs.opendaylight.org/show_bug.cgi?id=8794>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/771/log.html.gz#s1-s20-t1-k2-k8>`__
  46    ddb-cls-ms-lr-t     few    2017-07-08  2017-07-15  `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/776/log.html.gz#s1-s20-t3-k2-k8>`__
  47    ddb-cls-ps-ll-t     few    2017-07-09  2017-07-15  `8794 <https://bugs.opendaylight.org/show_bug.cgi?id=8794>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/777/log.html.gz#s1-s22-t1-k2-k8>`__
  48    ddb-cls-ps-lr-t     pass   long ago    2017-07-15                                                                 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Clean%20Leader%20Shutdown%20Prefbasedshard/Remote_Leader_Shutdown/>`__
  49    ddb-elm-ms-lr-t     few    2017-06-13  2017-07-15  `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__  `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/31/log.html.gz#s1-s24-t1-k2-k10>`__
  50    ddb-elm-ms-rr-t     few    2017-06-10  2017-07-15  `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/738/log.html.gz#s1-s24-t3-k2-k10>`__
  51    ddb-elm-ms-rl-t     few    2017-06-27  2017-07-15  `8749 <https://bugs.opendaylight.org/show_bug.cgi?id=8749>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/763/log.html.gz#s1-s24-t5-k2-k10>`__
  52    ddb-elm-ps-lr-t     few    2017-06-11  2017-07-15  `8664 <https://bugs.opendaylight.org/show_bug.cgi?id=8664>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s26-t1-k2-k6-k3-k1-k4-k7-k1>`__
  53    ddb-elm-ps-rr-t     pass   long ago    2017-07-15                                                                 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s26-t3>`__
  54    ddb-elm-ps-rl-t     few    2017-06-07  2017-07-15  `8403 <https://bugs.opendaylight.org/show_bug.cgi?id=8403>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/735/log.html.gz#s1-s26-t5-k2-k9>`__
  55    ddb-li-ms-st-t      tell   2017-07-15  2017-07-15  `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s28-t1-k2-k25-k1-k1>`__
  56    ddb-li-ms-dt-t      tell   2017-07-15  2017-07-15  `8619 <https://bugs.opendaylight.org/show_bug.cgi?id=8619>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s28-t3-k2-k25-k1-k8>`__
  57    ddb-li-ps-st-t      few    2017-06-08  2017-07-15  `8371 <https://bugs.opendaylight.org/show_bug.cgi?id=8371>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s30-t1-k2-k25-k1-k1>`__
  58    ddb-li-ps-dt-t      tell   2017-07-15  2017-07-15  `8845 <https://bugs.opendaylight.org/show_bug.cgi?id=8845>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s30-t3-k2-k25-k1-k8>`__
  59    ddb-ci-ms-ll-ct-t   few    2017-06-07  2017-07-15  `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/735/log.html.gz#s1-s32-t1-k2-k16-k1-k1>`__
  60    ddb-ci-ms-ll-st-t   tell   2017-07-15  2017-07-15  `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s32-t3-k2-k16-k1-k1>`__
  61    ddb-ci-ms-lr-ct-t   few    2017-06-08  2017-07-15  `8636 <https://bugs.opendaylight.org/show_bug.cgi?id=8636>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s32-t5-k2-k15-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1>`__
  62    ddb-ci-ms-lr-st-t   tell   2017-07-15  2017-07-15  `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s32-t7-k2-k16-k1-k1>`__
  63    ddb-ci-ps-ll-ct-t   few    2017-06-28  2017-07-15  `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t1-k2-k16-k1-k1>`__
  64    ddb-ci-ps-ll-st-t   few    2017-06-28  2017-07-15  `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t3-k2-k16-k1-k1>`__
  65    ddb-ci-ps-lr-ct-t   few    2017-06-28  2017-07-15  `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t5-k2-k16-k1-k1>`__
  66    ddb-ci-ps-lr-st-t   few    2017-06-28  2017-07-15  `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t7-k2-k16-k1-k1>`__
  67    ddb-ls-ms-lr-t      tell   2017-07-15  2017-07-15  `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s36-t1-k2-k12>`__
  68    ddb-ls-ms-rr-t      tell   2017-07-14  2017-07-15  `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/782/log.html.gz#s1-s36-t3-k2-k12>`__
  69    ddb-ls-ms-rl-t      tell   2017-07-12  2017-07-15  `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/779/log.html.gz#s1-s36-t5-k2-k12>`__
  70    ddb-ls-ps-lr-t      pass   long ago    2017-07-15                                                                 `link <https://jenkins.opendaylight.org/releng/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Listener%20Stability%20Prefbasedshard/Move_Leader_From_Listener_Local_To_Remote/>`__
  71    ddb-ls-ps-rr-t      few    2017-06-26  2017-07-15  `8733 <https://bugs.opendaylight.org/show_bug.cgi?id=8733>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/762/log.html.gz#s1-s38-t3-k2-k14-k2-k1-k4-k7-k1>`__
  72    ddb-ls-ps-rl-t      pass   long ago    2017-07-15                                                                 `link <https://jenkins.opendaylight.org/releng/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Listener%20Stability%20Prefbasedshard/Move_Leader_From_Listener_Remote_To_Local/>`__
  73    drb-rpp-ms-a        pass   long ago    2017-07-15                                                                 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Rpc%20Provider%20Precedence/>`__
  74    drb-rph-ms-a        few    2017-06-28  2017-07-15  `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s4-t6-k2-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1-k3-k1-k1-k1-k2-k1-k4-k7-k1>`__
  75    drb-app-ms-a        pass   long ago    2017-07-15                                                                 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Action%20Provider%20Precedence/>`__
  76    drb-aph-ms-a        few    2017-07-02  2017-07-15  `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/769/log.html.gz#s1-s8-t6-k2-k1-k1-k1-k1-k1-k1-k1-k1-k1-k1-k3-k1-k1-k1-k3-k1-k4-k7-k1>`__
  77    dnb-1n-60k-a        pass   long ago    2017-07-15                                                                 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/lastSuccessfulBuild/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__
  78    ss-ms-ms-a          pass   long ago    2017-07-15                                                                 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Master%20Stability/>`__
  79    ss-ph-ms-a          few    2017-06-29  2017-07-15  `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/765/log.html.gz#s1-s12-t5-k2-k3-k1-k2>`__
  80    ss-cl-ms-a          pass   long ago    2017-07-15                                                                 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s14>`__
  81    ss-ms-ms-t          pass   long ago    2017-07-15                                                                 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s40>`__
  82    ss-ph-ms-t          few    2017-07-15  2017-07-15  `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s42-t5-k2-k3-k1-k2>`__
  83    ss-cl-ms-t          pass   long ago    2017-07-15                                                                 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s44>`__
  84    netconf-ba-ms-a     pass   long ago    2017-07-14                                                                 `link <https://jenkins.opendaylight.org/releng/view/netconf/job/netconf-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/netconf-clustering.txt/CRUD>`__
  85    netconf-ok-ms-a     few    2017-06-18  2017-07-14  `8596 <https://bugs.opendaylight.org/show_bug.cgi?id=8596>`__  `link <https://logs.opendaylight.org/releng/jenkins092/netconf-csit-3node-clustering-only-carbon/568/log.html.gz#s1-s5-t17-k2-k3-k2-k2-k1>`__
  86    netconf-rr-ms-a     pass   long ago    2017-07-14                                                                 `link <https://jenkins.opendaylight.org/releng/view/netconf/job/netconf-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/netconf-clustering.txt/Outages>`__
  87    bgp-3n-300k-t-long  akka   2017-07-08  2017-07-08  `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__  `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-bgpclustering-longevity-only-carbon/11/log.html.gz#s1-s2-t1-k10-k1-k1-k1-k1-k1-k1-k1-k1-k1-k2-k1-k3-k7-k5-k1-k6-k1-k1-k1-k1-k1-k2-k1-k1-k2-k2-k2-k1-k6-k2-k2-k1>`__
  88    ddb-elm-mc-t-long   tell   2017-07-08  2017-07-08  `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-ddb-expl-lead-movement-longevity-only-carbon/14/log.html.gz#s1-s2-t1-k2-k1-k1-k1-k1-k1-k1-k2-k1-k1-k2-k10>`__
  89    drb-rpp-ms-a-long   few    2017-05-07  2017-07-08  `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-drb-partnheal-longevity-only-carbon/13/console.log.gz>`__
  90    drb-rph-ms-a-long   akka   2017-07-08  2017-07-08  `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-drb-partnheal-longevity-only-carbon/18/log.html.gz#s1-t1-k3-k1-k1-k1-k1-k1-k1-k2-k1-k1-k6-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1-k3-k1-k1-k1-k2-k1-k4-k7-k1>`__
  91    dnb-1n-60k-a-long   pass   long ago    2017-07-08                                                                 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/620/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__
  92    ss-ph-ms-a-long     akka   2017-07-08  2017-07-08  `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__  `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-cs-partnheal-longevity-only-carbon/15/log.html.gz#s1-s2-t1-k3-k1-k1-k1-k1-k1-k1-k1-k1-k1-k7-k3-k1-k2>`__
  93    ss-cl-ms-a-long     pass   long ago    2017-07-08                                                                 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/620/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__
  94    ==================  =====  ==========  ==========  =============================================================  ==========
  95
  96 For descriptions of test cases, see `description page <tests.html>`_.
  97 Note that the link contains current description,
  98 the details might have been implemented differently at SR1 release.