docs/cluster/carbon/sandbox.rst

   1
   2 Permanent draft, inaccessible: Sandbox test report
   3 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   4
   5 Test Case Summary
   6 -----------------
   7
   8 RelEng stability summary.
   9
  10 + tba: Recent failures to be analyzed yet: 0.
  11 + test: Recent failures caused by wrong assumptions in test: 0.
  12 + akka: Recent failures related to pure UnreachableMember: 4.
  13 + tell: Recent failures not clearly caused by UnreachableMember: 6.
  14 + few: Tests passing unless low frequency failure happens: 2 (1 without duplication).
  15   (Low frequency means UnreachableMemeber or similar,
  16   related to Akka where Controller code has not real control.)
  17 + pass: Tests passing consistently: 41 (39 without duplication).
  18 + Total: 53 (50 without duplication).
  19 + Total minus akka: 49 (46 without duplication).
  20 + Total minus akka passing always or mostly: 43 (40 without duplication).
  21 + Acceptance rate: 43/49=87.75% (40/46=86.95% without duplication).
  22
  23 Table
  24 -----
  25
  26 S017 instead of 2017 means Sandbox run (includes changes not merged to stable/carbon yet).
  27
  28 Last fail is date of last failure not caused by infra
  29 (or by a typo in test or by netconf/bgp failing to initialize properly).
  30
  31 "S 17" or "2 17" in Last run means the documented run was superseded by a newer one, but not analyzed yet.
  32
  33 "no sr3" means this test was not run on Sandbox, SR2 result is reported instead.
  34 "few" status from SR2 is not inherited (such tests are marked as "pass").
  35 "long ago" means the last real test failue happened somewhere before SR2 release (or never).
  36
  37 TODO: Copy formatting from sr2 page.
  38
  39 .. table:: Releng stability results (pre-SR2)
  40    :widths: 30,10,20,20,10,10
  41
  42    ==================  =====  ==========  ==========  =============================================================  ==========
  43    Scenario name       Type   Last fail   Last run    Bugs                                                           Robot link
  44    ==================  =====  ==========  ==========  =============================================================  ==========
  45    bgp-1n-1m-a         pass   no sr3      no sr3                                                                     no sr3
  46    bgp-1n-300k-t       pass   no sr3      no sr3                                                                     no sr3
  47    bgp-3n-300k-ll-t    akka   no sr3      no sr3      `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__  no sr3
  48    bgp-3n-300k-lr-t    akka   no sr3      no sr3      `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__  no sr3
  49    ddb-cls-ms-ll-t     pass   long ago    S017-08-24                                                                 no fail this week
  50    ddb-cls-ms-lr-t     pass   long ago    S017-08-24                                                                 no fail this week
  51    ddb-cls-ps-ll-t     pass   long ago    S017-08-24                                                                 no fail this week
  52    ddb-cls-ps-lr-t     pass   long ago    S017-08-24                                                                 no fail this week
  53    ddb-elm-ms-lr-t     pass   long ago    S017-08-24                                                                 no fail this week
  54    ddb-elm-ms-rr-t     pass   long ago    S017-08-24                                                                 no fail this week
  55    ddb-elm-ms-rl-t     pass   long ago    S017-08-24                                                                 no fail this week
  56    ddb-elm-ps-lr-t     pass   long ago    S017-08-24                                                                 no fail this week
  57    ddb-elm-ps-rr-t     pass   long ago    S017-08-24                                                                 no fail this week
  58    ddb-elm-ps-rl-t     pass   long ago    S017-08-24                                                                 no fail this week
  59    ddb-li-ms-st-t      pass   long ago    S017-08-24                                                                 no fail this week
  60    ddb-li-ms-dt-t      pass   long ago    S017-08-24                                                                 no fail this week
  61    ddb-li-ps-st-t      pass   long ago    S017-08-24                                                                 no fail this week
  62    ddb-li-ps-dt-t      tell   S017-08-24  S017-08-24  `8845 <https://bugs.opendaylight.org/show_bug.cgi?id=8845>`__  `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/2/log.html.gz#s1-s30-t3-k2-k25-k1-k8>`__
  63    ddb-ci-ms-ll-ct-t   pass   long ago    S017-08-24                                                                 no fail this week
  64    ddb-ci-ms-ll-st-t   pass   long ago    S017-08-24                                                                 no fail this week
  65    ddb-ci-ms-lr-ct-t   pass   long ago    S017-08-24                                                                 no fail this week
  66    ddb-ci-ms-lr-st-t   pass   long ago    S017-08-24                                                                 no fail this week
  67    ddb-ci-ps-ll-ct-t   pass   long ago    S017-08-24                                                                 no fail this week
  68    ddb-ci-ps-ll-st-t   pass   long ago    S017-08-24                                                                 no fail this week
  69    ddb-ci-ps-lr-ct-t   pass   long ago    S017-08-24                                                                 no fail this week
  70    ddb-ci-ps-lr-st-t   pass   long ago    S017-08-24                                                                 no fail this week
  71    ddb-ls-ms-lr-t      pass   long ago    S017-08-24                                                                 no fail this week
  72    ddb-ls-ms-rr-t      pass   long ago    S017-08-24                                                                 no fail this week
  73    ddb-ls-ms-rl-t      pass   long ago    S017-08-24                                                                 no fail this week
  74    ddb-ls-ps-lr-t      tell   S017-08-24  S017-08-24  `8733 <https://bugs.opendaylight.org/show_bug.cgi?id=8733>`__  `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/2/log.html.gz#s1-s38-t1-k2-k14-k2-k1-k4-k7-k1>`__
  75    ddb-ls-ps-rr-t      tell   S017-08-24  S017-08-24  `8733 <https://bugs.opendaylight.org/show_bug.cgi?id=8733>`__  `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/2/log.html.gz#s1-s38-t3-k2-k14-k2-k1-k4-k7-k1>`__
  76    ddb-ls-ps-rl-t      pass   long ago    S017-08-24                                                                 no fail this week
  77    drb-rpp-ms-a        pass   long ago    S017-08-24                                                                 no fail this week
  78    drb-rph-ms-a        pass   long ago    S017-08-24                                                                 no fail this week
  79    drb-app-ms-a        pass   long ago    S017-08-24                                                                 no fail this week
  80    drb-aph-ms-a        pass   long ago    S017-08-24                                                                 no fail this week
  81    dnb-1n-60k-a        pass   no sr3      no sr3                                                                     no sr3
  82    ss-ms-ms-a          pass   long ago    S017-08-24                                                                 no fail this week
  83    ss-ph-ms-a          few    S017-08-24  S017-08-24  `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__  `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/2/log.html.gz#s1-s10-t5-k2-k1-k1-k4>`__
  84    ss-cl-ms-a          pass   long ago    S017-08-24                                                                 no fail this week
  85    ss-ms-ms-t          pass   long ago    S017-08-24                                                                 no fail this week
  86    ss-ph-ms-t          few    S017-08-24  S017-08-24  `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__  `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/2/log.html.gz#s1-s40-t5-k2-k1-k1-k4>`__
  87    ss-cl-ms-t          pass   long ago    S017-08-24                                                                 no fail this week
  88    netconf-ba-ms-a     pass   no sr3      no sr3                                                                     no fail this week
  89    netconf-ok-ms-a     tell   no sr3      no sr3      `9027 <https://bugs.opendaylight.org/show_bug.cgi?id=9027>`__  no fail this week
  90    netconf-rr-ms-a     tell   no sr3      no sr3      `9027 <https://bugs.opendaylight.org/show_bug.cgi?id=9027>`__  no fail this week
  91    bgp-3n-300k-t-long  akka   no sr3      no sr3      `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__  no sr3
  92    ddb-elm-mc-t-long   pass   no sr3      no sr3                                                                     no sr3
  93    drb-rpp-ms-a-long   pass   no sr3      no sr3                                                                     no sr3
  94    drb-rph-ms-a-long   pass   no sr3      no sr3      `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__  no sr3
  95    dnb-1n-60k-a-long   pass   no sr3      no sr3                                                                     no sr3
  96    ss-ph-ms-a-long     akka   no sr3      no sr3      `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__  no sr3
  97    ss-cl-ms-a-long     tell   S017-08-23  S017-08-23  `9054 <https://bugs.opendaylight.org/show_bug.cgi?id=9054>`__  `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-cs-chasing-leader-longevity-only-carbon/14/log.html.gz#s1-s2-t3-k3-k2-k1-k1-k2-k1-k4-k6-k1>`__
  98    ==================  =====  ==========  ==========  =============================================================  ==========
  99
 100 For descriptions of test cases, see `description page <tests.html>`_.
 101 Note that the link contains current description,
 102 the details might have been implemented differently at SR1 release.