From: Tom Pantelis Date: Tue, 25 Apr 2017 17:26:27 +0000 (-0400) Subject: Fix intermittent failure in testLeadershipTransferOnShutdown X-Git-Tag: release/carbon~62 X-Git-Url: https://git.opendaylight.org/gerrit/gitweb?p=controller.git;a=commitdiff_plain;h=refs%2Fchanges%2F17%2F56017%2F2 Fix intermittent failure in testLeadershipTransferOnShutdown 10:03:06 java.util.concurrent.ExecutionException: ReadFailedException{message=Error executeRead ReadData for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car, errorList=[RpcError [message=Error executeRead ReadData for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.md.sal.common.api.data.DataStoreUnavailableException: Shard member-2-shard-cars-testLeadershipTransferOnShutdown currently has no leader. Try again later.]]} 10:03:06 at org.opendaylight.yangtools.util.concurrent.MappingCheckedFuture.wrapInExecutionException(MappingCheckedFuture.java:64) 10:03:06 at org.opendaylight.yangtools.util.concurrent.MappingCheckedFuture.get(MappingCheckedFuture.java:92) 10:03:06 at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.verifyCars(DistributedDataStoreRemotingIntegrationTest.java:215) 10:03:06 at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testLeadershipTransferOnShutdown(DistributedDataStoreRemotingIntegrationTest.java:928) From the logs it seems member-2 hadn't gotten MemberUp for member-3 after the leader transfer and by the time it tried to read. I added calls to wait for members to be up. After the change it ran 333 times w/o failure. Change-Id: Ifbbf304230292f69429d3086867679effb8db01c Signed-off-by: Tom Pantelis --- diff --git a/opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/DistributedDataStoreRemotingIntegrationTest.java b/opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/DistributedDataStoreRemotingIntegrationTest.java index 2bd389eda3..f34dea6128 100644 --- a/opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/DistributedDataStoreRemotingIntegrationTest.java +++ b/opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/DistributedDataStoreRemotingIntegrationTest.java @@ -208,6 +208,9 @@ public class DistributedDataStoreRemotingIntegrationTest extends AbstractTest { testParameter, type, moduleShardsConfig, false, shards); leaderTestKit.waitUntilLeader(leaderDistributedDataStore.getActorContext(), shards); + + leaderTestKit.waitForMembersUp("member-2"); + followerTestKit.waitForMembersUp("member-1"); } private static void verifyCars(final DOMStoreReadTransaction readTx, final MapEntryNode... entries) @@ -883,6 +886,9 @@ public class DistributedDataStoreRemotingIntegrationTest extends AbstractTest { try (AbstractDataStore follower2DistributedDataStore = follower2TestKit.setupAbstractDataStore( testParameter, testName, MODULE_SHARDS_CARS_PEOPLE_1_2_3, false)) { + followerTestKit.waitForMembersUp("member-3"); + follower2TestKit.waitForMembersUp("member-1", "member-2"); + // Create and submit a couple tx's so they're pending. DOMStoreWriteTransaction writeTx = followerDistributedDataStore.newWriteOnlyTransaction();