controller.git
2 years agoBug 8885: Fix DistributedShardedDOMDataTree initialization 80/60780/4
Tom Pantelis [Wed, 26 Jul 2017 16:16:48 +0000 (12:16 -0400)]
Bug 8885: Fix DistributedShardedDOMDataTree initialization

DistributedShardedDOMDataTree initialization expects the prefix
configuration shard to be present and ready with leader however
the latter isn't the case when the static module-shards is
bootstrapped without the local member so it can be dynamically
joined into an existing cluster. So I modified the ConfigShardLookupTask
to elide the ConfigShardReadinessTask.

Once past that, creation of the prefix-based default shard is attempted
as there isn't a local module-based shard however this fails b/c the
local prefix configuration shard is not connected to a leader. To alleviate
this I just commented out the code to create the shard. Since the default
shard configuration is present in the out-of-box modules.conf and is
expected to be present, we can assume at this point that the local member
isn't in the replica list with the intention of dynamically joining it to
an existing cluster, at which time the shard will be created.

These changes at least fix the regression with the boostrapping scenario.
We can revisit this iniialization later w.r.t. prefix-based shards.

Change-Id: I1faf531f4c79914d45203ee132dd4e65ad2f18ba
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoDeprecate org.opendaylight.controller.md.sal.dom.spi.AbstractRegistrationTree 78/61078/2
Robert Varga [Wed, 2 Aug 2017 21:32:28 +0000 (23:32 +0200)]
Deprecate org.opendaylight.controller.md.sal.dom.spi.AbstractRegistrationTree

This is a utility class, which has a conterpart in mdsal. Deprecate it
and related classes, migrating users.

Change-Id: I8206350ddb60bb19aed93ff3840e0e68e288d55a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoDeprecated controller fine-grained sharding APIs 77/61077/1
Robert Varga [Wed, 2 Aug 2017 21:18:57 +0000 (23:18 +0200)]
Deprecated controller fine-grained sharding APIs

These are not maintained nor were they ever implemented. Deprecate
them and point to their mdsal counterparts to reduce confusion.

Change-Id: Idd1908c65b0737df0a3731e6e81a7d1c71f272d0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoRemove logback related stuff 33/60833/3
Michael Vorburger [Thu, 27 Jul 2017 18:55:27 +0000 (00:25 +0530)]
Remove logback related stuff

as discussed on https://lists.opendaylight.org/pipermail/odlparent-dev/2017-July/001262.html '

Change-Id: I09146cd363d1ab706143bc12c8b1e37aa96c8723
Signed-off-by: Michael Vorburger <vorburger@redhat.com>
2 years agoOptimize use of YangInstanceIdentifier.getPathArguments() 07/60907/6
Robert Varga [Sat, 29 Jul 2017 12:05:36 +0000 (14:05 +0200)]
Optimize use of YangInstanceIdentifier.getPathArguments()

This method returns a list, hence we can lookup the first item
without iterating and also can use Lists.transform().

Change-Id: Ie26bfcc225c74154d65ef963e3444ac5ec10bafb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBUG-8898: prioritize InternalCommand 81/60981/1
Robert Varga [Mon, 31 Jul 2017 14:08:22 +0000 (16:08 +0200)]
BUG-8898: prioritize InternalCommand

InternalCommand requests should be processed as soon as possible,
and since we are already using ControlAwareMailbox, this is as simple
as marking InternalCommand as a ControlMessage.

Change-Id: Ic6025f4254da47801676c0c474d03e18abbf8f50
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2ac32ea2c4f57993a1dc49ef8ce380cb03acc822)

2 years agoBUG-8898: do not invoke timeouts directly 80/60980/1
Robert Varga [Mon, 31 Jul 2017 13:54:12 +0000 (15:54 +0200)]
BUG-8898: do not invoke timeouts directly

Request timeouts are occuring with the connection lock held,
at which point the connection can be at the tail of a successor
chain:

oldestConnection -> olderConnection -> connection

If the callback being invoked attempts to transmit an entry,
we will end up attempting to lock the entire chain. This would not
be a problem except that if there is a concurrent attempt to lock
the entire chain it ends up holding the lock of oldestConnection
and it is waiting for the lock on connection -- which will only be
released once the callback finishes executing, but that in turn
waits for oldestConnection to be unlocked -- a classic AB/BA deadlock.

This patch alleviates the problem by deferring callback execution
via executeInActor, i.e. the timeout will be delivered at as part
of normal message processing.

Change-Id: I237908cf214bcdfd477fe0212d09b207a0c2cdbf
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 4367f456f3c7a30c8ee9c7bca738b3e120a4e1d1)

2 years agoBug 8879: Migrate controller to the new XML parser 13/60913/9
Igor Foltin [Sun, 30 Jul 2017 08:41:42 +0000 (10:41 +0200)]
Bug 8879: Migrate controller to the new XML parser

Migrate blueprint to the new XML parser from YANG tools.

Change-Id: Ib82da2f4b2b49dde7df78f425c700b9d3f473a26
Signed-off-by: Igor Foltin <igor.foltin@pantheon.tech>
2 years agoTake advantage of default methods in DOMRpcProviderService 35/51035/11
Robert Varga [Wed, 25 Jan 2017 22:05:02 +0000 (23:05 +0100)]
Take advantage of default methods in DOMRpcProviderService

We can make one of the methods default, as all implementations
are exactly the same (codifying contract).

Change-Id: I64f0b62fd3a0987ed1ed01ec14b2c4d3b77560ac
Signed-off-by: Robert Varga <rovarga@cisco.com>
2 years agoMake -it- parents use current odlparent version 76/60876/2
Vratko Polak [Mon, 31 Jul 2017 11:34:32 +0000 (13:34 +0200)]
Make -it- parents use current odlparent version

Change-Id: I4fd29a6323eb9181f9629126a51ab578db1f5df2
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2 years agoconfig-persister-impl: use lambdas 70/57170/4
Stephen Kitt [Tue, 16 May 2017 15:48:58 +0000 (17:48 +0200)]
config-persister-impl: use lambdas

This series of patches uses lambdas instead of anonymous classes for
functional interfaces when possible. Lambdas are replaced with method
references when appropriate.

Change-Id: I20e8b07b839c168d0944c44a57602c3b9a96ce6a
Signed-off-by: Stephen Kitt <skitt@redhat.com>
2 years agoSlice front-end request messages 74/60574/7
Tom Pantelis [Wed, 19 Jul 2017 20:47:28 +0000 (16:47 -0400)]
Slice front-end request messages

Added infrastructure to use the MessageSlicer to slice SliceableMessages in
the TransmitQueue.Transmitting class on the front-end. A MessageAssembler is
used on the Shard side to re-assemble. Currently only the
front-end ModifyTransactionRequest is a SliceableMessage as it contains a
NormalizedNode which can be arbitrarily large - the others are small and
don't require slicing.

Change-Id: I7b09e4864e19d3fdb215c2b9dbcb64c14b6a143c
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoblueprint: final parameters 08/56908/4
Stephen Kitt [Thu, 11 May 2017 15:42:46 +0000 (17:42 +0200)]
blueprint: final parameters

This automatically-generated patch flags all appropriate parameters as
final (including caught exceptions).

Change-Id: I565047abcceb31a3da2ef8b2ebdee857e6623196
Signed-off-by: Stephen Kitt <skitt@redhat.com>
2 years agoBug 8494: Separate writing and completion threads 03/60703/1
Vratko Polak [Fri, 21 Jul 2017 10:24:49 +0000 (12:24 +0200)]
Bug 8494: Separate writing and completion threads

If AbstractTransactionHandler uses only one executor thread,
future completion callbacks are delayed by throttling on writes.
CSIT aims to detect RequestTimeoutException within a narrow window,
so a separate executor for callbacks is used now.

The delay would not be that critical, but the problem is the timing
between a scheduled execution which exceeds scheduling gaps. These
seem to hold up normally-submitted tasks, leading to futures never
completing.

Therefore we use two Executors and synchronize state modification
call sites. Hence the two tasks (throttled producer) and future
completions can run concurrently (aside from state synchronization).

Change-Id: I642c5295ab6188b2d7e1b5feae62ab7ef52d41eb
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 8744119235b90d89021567e5f12361d98b823b8f)

2 years agoBUG-8618: refresh transaction access when isolated 02/60702/1
Robert Varga [Mon, 17 Jul 2017 15:11:48 +0000 (17:11 +0200)]
BUG-8618: refresh transaction access when isolated

When we are isolated leader we stop accepting messages from
the frontend. If we remain in this state for more than 15 seconds
this can result in a timeout -- which is obvious, but it really
is our fault.

Since we cannot make forward progress anyway, there is no point
in purging the transaction. Update its access time with whatever
the last mark for that frontend was.

Change-Id: I9ff56c91e4fda4b68cd34c05609dc88d6d65fd32
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 1529bb8bdd4c30a782cf1574b0127833da5831b7)

2 years agoBUG-8792: allow transactions to not time out after reconnect 01/60701/1
Robert Varga [Mon, 17 Jul 2017 12:41:47 +0000 (14:41 +0200)]
BUG-8792: allow transactions to not time out after reconnect

During reconnect churn, the frontend may be catching up with previous
transactions, hence we should hold off timing it out until it does.

When we arrive at a timed out transaction, we allow the access time to
be updated to connect time -- effectively saying the transaction was
touched at the time of reconnect.

Change-Id: I3930b5782579f50931b204d8579c2aee51e2bc55
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 55661ed801e178812f16ac990c93b51a3d68c00e)

2 years agoBUG-8619: do not touch forward path during purge enqueue 00/60700/1
Robert Varga [Thu, 20 Jul 2017 15:51:52 +0000 (17:51 +0200)]
BUG-8619: do not touch forward path during purge enqueue

In case of a purge request, the request is sent from the head
of a connection chain (i.e. the original connection which created
the transaction) and propagated via forwarders. This path needs
to make sure it does not go via throttling, as it is an internal
detail.

Separate the transmit paths a bit more, so that TransmitQueue
can push messages to forwarders' replay path.

Change-Id: I5e146b8d11e8654b4beae3959207efb9c2f18315
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b83c7f5e5cdaee5f250988182dccb749ac7432c2)

2 years agoBUG-8618: record LeaderFrontendState time 99/60699/2
Robert Varga [Sat, 15 Jul 2017 21:33:25 +0000 (23:33 +0200)]
BUG-8618: record LeaderFrontendState time

In order to deal with IsolatedLeader state and transaction timeouts,
we need to maintain an accurate view of when we have seen the frontend
even if we are not accepting messages from it.

Add correspoding field and maintain it whenever we interact with
LeaderFrontend state. Also record last connect ticks for the same use.

Change-Id: I8e49037507fcd01470a03be8c0d611efca55dabf
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 7633a2a50144dad7cf987b29959dc06509575c05)

2 years agoBug 3401: Remove/cleanup Import-Package in maven-bundle-plugin config 55/59955/5
Tom Pantelis [Wed, 5 Jul 2017 03:41:06 +0000 (23:41 -0400)]
Bug 3401: Remove/cleanup Import-Package in maven-bundle-plugin config

Some of the pom files don't need to explicitly specify Import-Package
in the maven-bundle-plugin configuration. Others were cleaned up to remove
unnecessary entries.

Change-Id: I6b9a741d1a110f17d371497e04e2ab2187aff6b6
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBUG-7464: use yangtools.triemap 61/60561/1
Robert Varga [Wed, 19 Jul 2017 12:08:04 +0000 (14:08 +0200)]
BUG-7464: use yangtools.triemap

Yangtools is moving away from using upstreap Triemap to its
internal fork of that codebase. Switch this code, too.

Change-Id: I0d60ccc8927505a83a35631333203817484da9e0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBug 8619: Introduce inheritance of progress trackers 38/60538/1
Vratko Polak [Fri, 14 Jul 2017 15:50:32 +0000 (17:50 +0200)]
Bug 8619: Introduce inheritance of progress trackers

+ Introduce cancelDebt method.
+ Use the newly introduced functionality in client code.
+ Delete unused copy constructors (including unit test).

Change-Id: Ib976343ed5f50c649ea08206c897cb70dead8b86
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 12b4928ef66a82f4a128a11701663ac23143c1d7)

2 years agoSimplify QuarantinedMonitorActor 84/60484/3
Robert Varga [Mon, 17 Jul 2017 12:57:08 +0000 (14:57 +0200)]
Simplify QuarantinedMonitorActor

As per the comments, upstream has provided a dedicated event, hence
use that instead of digging inside akka internals.

Change-Id: I4731dfbbdd228d562ddd32ec5fd3d0e9af0855d0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBUG-8143: issue a JVM restart 85/60485/1
Robert Varga [Mon, 17 Jul 2017 13:03:50 +0000 (15:03 +0200)]
BUG-8143: issue a JVM restart

Instead of just restarting the OSGi framework, instruct karaf to
re-execute the JVM.

Change-Id: I10709f61b71d578e4677a5948c23e38f9871c6a1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoProgressTracker: Decrease delay due nearestAllowed 38/60438/2
Vratko Polak [Fri, 7 Jul 2017 11:14:57 +0000 (13:14 +0200)]
ProgressTracker: Decrease delay due nearestAllowed

If nearestAllowed is in past, that means we have
a temporary interval of relatively small demand for tasks.
We can reduce delay, as if the time since nearestAllowed
was a "delay in advance".

This way the queue stays closer to the intended capacity.

Change-Id: I40f95ea9cb25ea62d8c65ee78cafc79e9b56cc11
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit 80e6514d56cd4dc6aa40997dea2b460723148341)

2 years agoBUG-8618: fix test driver 37/60437/1
Robert Varga [Mon, 10 Jul 2017 14:08:57 +0000 (16:08 +0200)]
BUG-8618: fix test driver

Since the test can produce bursts of completions, which in turn can
get slowed down by writout of new messages, offload future completion
to the executor we have internally. This in turn simplifies things,
as we can rely on state being manipulated (mostly) from a single thread.

Also change ArrayDeque to a HashSet to ensure removal of tasks completes
quickly even in face of misordered responses.

Change-Id: Ia5341633af2dbe3e26e7208436405daf7632a876
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2be77b3bcef31ad8b6dbdce073471561d2cf76d6)

2 years agoBUG-8618: add pause/unpause mechanics for tell-based protocol 36/60436/1
Robert Varga [Thu, 6 Jul 2017 16:26:44 +0000 (18:26 +0200)]
BUG-8618: add pause/unpause mechanics for tell-based protocol

When we are transitioning to/from paused state, we need to remove
all frontend-related state, including pending transactions, to ensure
ShardDataTree does not track them.

When we change to unpaused leader, we can reconstruct the state
from the journal -- the rest will be forwarded from the frontend anyway.

Change-Id: I28d486d1a6695e21dd7e6518609680d54e5a15eb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 40d27d44d6f0b0358505b2e8ac5abbad25f47d4b)

2 years agoBUG-8618: introduce RaftActor.unpauseLeader() 19/60419/2
Robert Varga [Thu, 6 Jul 2017 15:15:21 +0000 (17:15 +0200)]
BUG-8618: introduce RaftActor.unpauseLeader()

This is a preparatory patch, which notifies RaftActor when
the operation hooked to pauseLeader() fails to complete and the
leader should resume its normal operation.

This is needed to correctly resume operations of tell-based protocol
after a pauseLeader() completes without actually changing the leader.

Change-Id: Ia00e52ebb327575a484af62bf0c31131a33303b3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 3a10a45e0f78337435c8bc84015c4724a9fa7741)

2 years agoBUG-8618: eliminate SimpleShardDataTreeCohort subclasses 18/60418/2
Robert Varga [Mon, 3 Jul 2017 17:57:38 +0000 (19:57 +0200)]
BUG-8618: eliminate SimpleShardDataTreeCohort subclasses

Now that we handle pre-cancommit failures useing reportFailure(),
there is no need to have specialized subclasses for cohorts, as
the initial failure can cleanly be handled via nextFailure.

This also places a guard in reportFailure() so we do not override
a failure once it is set -- which should only happen in the case
of a dead-on-arrival transaction and it timing out in READY state.

Change-Id: I057c5b36006843f51d60034d30af83bac4e02cd7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2783c9dffdd91dae87d3351f4ebffbd8679e3133)

2 years agoBUG-8618: rework AbstractProxyTransaction.flushState() 17/60417/2
Robert Varga [Thu, 6 Jul 2017 07:04:10 +0000 (09:04 +0200)]
BUG-8618: rework AbstractProxyTransaction.flushState()

Instead of directly forwarding state use ModifyTransactionRequest
to encapsulate state and forward it separately to the successor.

This eliminates sendRequest() from replay path, ensuring the replay
thread is not blocked.

Change-Id: Ice86791d417b7487b9d3b1df06341dd028cde7f8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit c525e5f25b951daa28d0cbde237ba3040b68f99f)

2 years agoBUG-8618: reconnect connections more aggressively 16/60416/2
Robert Varga [Mon, 3 Jul 2017 17:10:22 +0000 (19:10 +0200)]
BUG-8618: reconnect connections more aggressively

Given that the timeout period on backend for an existing transaction
is 15 seconds, sleeping for 5 seconds between reconnect attempts seems
excessive. Lower the timer to 1 second, which should give us a slightly
better chance to avoid timeouts.

Change-Id: Ib74480f5630865cb7a11ca7027e0495443d1d14e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 70f287502823bab284555b52b91043c3204b829b)

2 years agoBUG-8618: turn timeouts in READY state into canCommit failures 15/60415/2
Robert Varga [Mon, 3 Jul 2017 16:55:18 +0000 (18:55 +0200)]
BUG-8618: turn timeouts in READY state into canCommit failures

This patch adds more details to the TimeoutException reported when
we prune a transaction while it is in the queue. It also peels the
READY case from the defaults and makes sure we send an authoritative
reply back to the frontend when it requests the transaction to be
committed.

Change-Id: I21364ff7e7103af8be6988b8483adc112c3c1d25
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 0d5408c4babc902d270d9f81ed53c6af93bb2867)

2 years agoBUG-8618: improve logging 14/60414/2
Robert Varga [Mon, 3 Jul 2017 15:30:43 +0000 (17:30 +0200)]
BUG-8618: improve logging

While target sequence is important, we also need to log transmit
sequence, too.

Since this issue involves a state mismatch on the backend, improve
ShardDataTreeCohort logging to include transaction identifier
and state.

Change-Id: I21735870a9ae7983dc14a8f8f4d7464d3448ca60
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d2d9179e52a0d87aae2b9014b4c36384e24692e3)

2 years agoBug 7449: Slice ReadTransactionSuccess response 07/59407/11
Tom Pantelis [Thu, 22 Jun 2017 14:20:11 +0000 (10:20 -0400)]
Bug 7449: Slice ReadTransactionSuccess response

Added slicing of the ReadTransactionSuccess message. The slicing is
initiated by the Shard usung a MessageSlicer and re-assembly is done
by the ClientActorBehavior on the FE. Introduced a SliceableMessage
interface implemented by ReadTransactionSuccess which Shard checks for
to determine if the response message should be sliced.

Change-Id: Ie55e35aa82a9d2bc21f7a8f24396cb4df467252e
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBug 8163: getDataTreeChangeListenerExecutor() & DataBrokerTestModule 12/60312/1
Michael Vorburger [Thu, 13 Jul 2017 21:17:24 +0000 (02:47 +0530)]
Bug 8163: getDataTreeChangeListenerExecutor() & DataBrokerTestModule

Adjust AbstractDataBrokerTestCustomizer with a
getDataTreeChangeListenerExecutor() instead of a
setDataTreeChangeListenerExecutor(), just for more consistency with the
existing getCommitCoordinatorExecutor() method.  Also less confusing (to
me) than seeing the private Executor set by default which may get
changed by the setter later.

Adjust ConcurrentDataBrokerTestCustomizer with the
useMTDataTreeChangeListenerExecutor as a constructor argument, instead
of an useMTDataTreeChangeListenerExecutor() method, just for consistency
for how you already have it in AbstractConcurrentDataBrokerTest.

Extend ConstantSchemaAbstractDataBrokerTest and DataBrokerTestModule to
allow passing through this new opt-in tweak flag, so that tests in
downstream projects such as genius and netvirt can staring exploring
enabling this.

Change-Id: I4ad85ac48163d2f4bac865f46a3b047d5b7d333a
Signed-off-by: Michael Vorburger <vorburger@redhat.com>
2 years agoBug 8163: Use MT DTCL executor in AbstractConcurrentDataBrokerTest 22/60022/5
Tom Pantelis [Thu, 6 Jul 2017 12:30:28 +0000 (08:30 -0400)]
Bug 8163: Use MT DTCL executor in AbstractConcurrentDataBrokerTest

Using a direct executor can cause deadlocks so the DTCL executor was
made configurable to use a threadpool as an opt-in. Direct executor
is still the default as many existing tests would break.

Change-Id: I41e14f1e6d3b77a44e61dfc75abff29d11a777dc
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoRefactor Follower#handleAppendEntries 85/59085/7
Tom Pantelis [Fri, 16 Jun 2017 14:51:19 +0000 (10:51 -0400)]
Refactor Follower#handleAppendEntries

This method is large - refactor it a bit.

Change-Id: Idae1883accdd7c73b57471501e66398306cf6e91
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoRemove <version> from org.apache.aries.blueprint.core dep. 66/58366/2
Michael Vorburger [Tue, 6 Jun 2017 18:24:02 +0000 (20:24 +0200)]
Remove <version> from org.apache.aries.blueprint.core dep.

Because in c/58365a the latest version was added to oldparent
dependencyManagement, and it seems wiser to declare this in only a
single place, to avoid possible version discrepancy problems re. this
artifact in the future.

Change-Id: If4370f3c7e80123fe6c225a11c1224145b3ad2b9
Signed-off-by: Michael Vorburger <vorburger@redhat.com>
2 years agoBUG-8676: add UnsignedLongRangeSet 03/59703/6
Robert Varga [Thu, 29 Jun 2017 16:47:14 +0000 (18:47 +0200)]
BUG-8676: add UnsignedLongRangeSet

This patch adds the wrapper class and updates users to use it directly.
The implementation itself is not changed, that will be done in a follow-up
patch.

Change-Id: Ie240ca5c3c9fc1448629bb5db6ecfa1029f66b8f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoCleanup warnings 89/59589/11
Robert Varga [Tue, 27 Jun 2017 21:43:40 +0000 (23:43 +0200)]
Cleanup warnings

- pom.xml groupId duplicate
- Futures.addCallback()
- Throwables.propagate*()
- pontentially-static methods
- remove 'throws Exception' where it is not really needed

Change-Id: Ib47e6255e0f510ab7dd0dcd08f71f2dd124df7b7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoFix Verify/Preconditions string format 66/59866/3
Robert Varga [Mon, 3 Jul 2017 08:17:54 +0000 (10:17 +0200)]
Fix Verify/Preconditions string format

These methods take a String.format() string, not a logging one, hence
we are not getting the information we want.

Change-Id: I46de0d64c85594e3d7b8be97951f1cf5249bca8f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 3e9ac68fea1aef0c7fedec346e50882efdde8acc)

2 years agoBug 6794: Remove threadpool config modules 85/59985/5
Tom Pantelis [Wed, 5 Jul 2017 18:35:41 +0000 (14:35 -0400)]
Bug 6794: Remove threadpool config modules

With the removal of the CSS netconf connector, the threadpool config modules
are no longer used so remove them. The *Wrapper classes are used via blueprint
so they remain. Also removed the eventbus config modules which were deprecated
in Carbon.

Change-Id: Ic528e5817a9f5ccdb67ef41987128ead4db51cbd
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoUpgrade to odlparent 2.0.2 90/60090/2
Stephen Kitt [Fri, 7 Jul 2017 15:47:55 +0000 (17:47 +0200)]
Upgrade to odlparent 2.0.2

Change-Id: I748830e39c108056ecd81809a0556e8c43d251f4
Signed-off-by: Stephen Kitt <skitt@redhat.com>
2 years agoSpeed up slow tell-based Distributed*IntegrationTest cases 01/60101/4
Tom Pantelis [Fri, 7 Jul 2017 21:13:59 +0000 (17:13 -0400)]
Speed up slow tell-based Distributed*IntegrationTest cases

Some test cases for tell-based take up to 2 minutes due to the default
2 minute request timeout and 30 sec backend aliveness timer. Speed up
the tests by setting much lower values.

Change-Id: If8dba80d625ded8753178e937f14b435675ef0e4
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoMake AbstractClientConnection timeouts configurable 87/60087/4
Tom Pantelis [Fri, 7 Jul 2017 14:59:40 +0000 (10:59 -0400)]
Make AbstractClientConnection timeouts configurable

So we can tweak them in production and unit tests.

Change-Id: I39ce8cdf3cd5397a71f52c42357943dfe5eccb7c
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoFix incorrect spelling of fileBackedStreamFactory 64/60064/1
Tom Pantelis [Fri, 7 Jul 2017 08:53:14 +0000 (04:53 -0400)]
Fix incorrect spelling of fileBackedStreamFactory

filedBackedStreamFactory should be fileBackedStreamFactory.

Change-Id: Ib0b65d68d37c5b0ded4f1739d4ddc578973fe6ec
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBUG-4513 UT for Change event is empty when homogenious composite key is used 79/28679/5
Valentin Mayamsin [Wed, 21 Oct 2015 22:55:51 +0000 (15:55 -0700)]
BUG-4513 UT for Change event is empty when homogenious composite key is used

The original DCL test passes now so whatever the issue was before has been
fixed. I also added a DTCL test.

Change-Id: I5c5037f49a77835dbfea1ce9db8b22d03b6191ec
Signed-off-by: Valentin Mayamsin <vmayamsi@cisco.com>
2 years agoTransfer leadership in PreLeader state 24/60024/1
Robert Varga [Thu, 6 Jul 2017 13:21:16 +0000 (15:21 +0200)]
Transfer leadership in PreLeader state

Rather than dying when requested to shutdown in pre-leader state,
follow the same code path we perform in normal leader mode, i.e.
transfer leadership.

Change-Id: I2ca30d44626df05c5f8b5ff6984eea20c7bf0949
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoExplicitly load the real DataBroker with component-name 25/59925/1
Josh [Tue, 4 Jul 2017 10:02:32 +0000 (13:02 +0300)]
Explicitly load the real DataBroker with component-name

It seems that karaf4 has "better" wiring so the
TracingBroker was being wired to itself, resulting
in stack overflows.

Change-Id: Iedb2e9dcfd53acf384ed3130cfcd78f313d76e1e
Signed-off-by: Josh <jhershbe@redhat.com>
2 years agoRe-enable karaf distribution 15/59315/7
Robert Varga [Wed, 21 Jun 2017 20:41:20 +0000 (22:41 +0200)]
Re-enable karaf distribution

This re-enables opendaylight distribution build to get us back
on par.

Change-Id: I11e5ee4d1f9f9de716f5636ac9afbad0137c93fc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBump odlparent dependency to 2.0.1 54/59854/2
Robert Varga [Mon, 3 Jul 2017 08:54:34 +0000 (10:54 +0200)]
Bump odlparent dependency to 2.0.1

Bumps odlparent to latest release.

Change-Id: Ifaf36c6539206ec5c35663717b691a0d962d1744
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBug 7449: Add slicer Id to MessageSliceIdentifier 97/59897/2
Tom Pantelis [Mon, 3 Jul 2017 17:00:07 +0000 (13:00 -0400)]
Bug 7449: Add slicer Id to MessageSliceIdentifier

Both Shard and RaftActor (via AbstractLeader) (will) have separate
MessageSlicer instances and we need to determine to which instance
MessageSliceReply messages should be forwarded otherwise the first
MessageSlicer will drop messages destined for the second MessageSlicer.
Therefore add a slicerId field to MessageSliceIdentifier which is
checked by MessageSlicer#handleMessage.

Change-Id: Ib39ede29789d5bfaf1fdaea66a8d2994fe6ebcd6
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBUG-8704: rework seal mechanics to not wait during replay 51/59851/2
Robert Varga [Wed, 28 Jun 2017 14:47:13 +0000 (16:47 +0200)]
BUG-8704: rework seal mechanics to not wait during replay

AbstractProxyTransaction.seal() and most notably internalSeal()
can end up pushing down messages down the connection hence they
can end up slowing down the replay process.

The replay paths end up enqueing subsequent requests anyway, so
rework the structure to split the 'seal only' and 'seal and flush'
codepaths.

Change-Id: Ie75c1ef8aa0d3d5d7ca482d383fd516077ca50b4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 1e07329c0d800b8fea43ae0c4060aded5fd18739)

2 years agoBug 8768: Close itemProducer for every code path 50/59850/2
Vratko Polak [Thu, 29 Jun 2017 11:50:07 +0000 (13:50 +0200)]
Bug 8768: Close itemProducer for every code path

Change-Id: Ib87de13e2a0e6f128f74a05b80ffb4331e345d2c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit 35b7e595945a1386047c1af73c94b70fbdaf9a59)

2 years agoBUG-8494: rework AbstractTransactionHandler 49/59849/2
Robert Varga [Wed, 28 Jun 2017 09:34:34 +0000 (11:34 +0200)]
BUG-8494: rework AbstractTransactionHandler

If we have a transaction failure while we are producing transactions,
we could end up adding a delay until the failure is detected as we
would continue jamming in transactions.

Rework internal logic to halt processing as soon as a failure is seen,
speeding up detection and simplifying code.

Change-Id: I19d13c78d94bb39481abde477ec4e3df03a6aa57
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b7657c3ac7b4697372674b75e820581a6d59e2ba)

2 years agoBUG-8494: fix failure path thinko 48/59848/2
Robert Varga [Tue, 27 Jun 2017 14:53:09 +0000 (16:53 +0200)]
BUG-8494: fix failure path thinko

The check should be to see if the failure has *not* been set,
hence invert the check.

Change-Id: I2c3893924f1c985687beedbfae0889388fad15c7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 5e986f5320c561953759a7beffb11db7e296817c)

2 years agoBUG-8445: check sessionId before propagating failures 47/59847/2
Robert Varga [Mon, 26 Jun 2017 14:31:30 +0000 (16:31 +0200)]
BUG-8445: check sessionId before propagating failures

When we have leader movement ocurring, based on timing details we
can re-establish a connection to the new leader and then start
receiving responses from the old leader telling us it no longer
is the leader.

To stop this from happening we need to check connection session ID
against the incoming failure.

Change-Id: If9a891016c7f213f2552283e3ec13485e598f5a4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 1c495bceb8d9c203f5ce53ea1ab9d907efb4d7b3)

2 years agoBUG-8494: Cleanup clustering-it-provider 46/59846/2
Robert Varga [Fri, 23 Jun 2017 11:02:40 +0000 (13:02 +0200)]
BUG-8494: Cleanup clustering-it-provider

Fixes various warnings and refactors MdsalLowLevelTestProvider
to be slightly cleaner in terms of number of classes.

It also eliminates synchronous thread blocking on future collection
and instead schedules task which performs the cleanup if the system
gets stuck.

Change-Id: I657f3df60c620284538bdf39ab1536eac8448801
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d97061af6814ad7b085af10797a252aa4aa5cda6)

2 years agoCleanup ProduceTransactionsHandler 45/59845/2
Robert Varga [Thu, 15 Jun 2017 08:39:55 +0000 (10:39 +0200)]
Cleanup ProduceTransactionsHandler

Shuffle invariants around to reduce overheads. Also adds better debugs
around futures completing.

Change-Id: I01f940de08e9e0b7fc0e95b48b2d5fecdfd78f86
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 9797fc8e587a51395342586bc44de9750fb67af3)

2 years agoBUG 8604 set proper tag when producer creation times out 44/59844/2
Tomas Cere [Mon, 12 Jun 2017 13:20:29 +0000 (15:20 +0200)]
BUG 8604 set proper tag when producer creation times out

Change-Id: I405f4d546a32b2d0f5b56fb03907a63334fabd6c
Signed-off-by: Tomas Cere <tcere@cisco.com>
(cherry picked from commit ec734245413c94cdd758f4c22ad3f3b63cfae5e6)

2 years agoBUG 8494 log possibly hanged futures in tx handlers 43/59843/2
Tomas Cere [Mon, 12 Jun 2017 13:29:46 +0000 (15:29 +0200)]
BUG 8494 log possibly hanged futures in tx handlers

Change-Id: Iccc90e575033c6770a3a499853f31e0684a712e4
Signed-off-by: Tomas Cere <tcere@cisco.com>
(cherry picked from commit 0723037074588cb901212e9b3ad9bf437e754f89)

2 years agoCatch all exceptions when submitting in tx handlers 42/59842/2
Tomas Cere [Thu, 8 Jun 2017 09:48:00 +0000 (11:48 +0200)]
Catch all exceptions when submitting in tx handlers

Change-Id: I5b9a2ec26b1b6001423f2cf5cf57285ce6c7e340
Signed-off-by: Tomas Cere <tcere@cisco.com>
(cherry picked from commit 31a52c56cb4e8398403f299d0c3d3830084e260e)

2 years agoBUG-8620: handle direct commit and disconnect correctly 41/59841/2
Robert Varga [Wed, 7 Jun 2017 16:08:28 +0000 (18:08 +0200)]
BUG-8620: handle direct commit and disconnect correctly

Transactions committed directly can complete in a disconnected
fashion as we are skipping the back-and-forth communication of the
three-phase commit. This period may involve shard leadership changes
and so we may end up in a situation where we are replaying a direct
commit request to a transaction which already completed -- which
raises a RequestFailure to make sure we do not do anything untoward.

In the specific case of direct commit, though, this is perfectly fine
and so update the callback to account for this case happening.

Change-Id: Ic60e69f0f58cc7c5a3ac869386dc12f856aa1f74
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit da42d2ffc8904b8dd24596cf6d918a0d30c8c521)

2 years agoBUG 8602: Skip initial fill of idints 40/59840/2
Tomas Cere [Tue, 6 Jun 2017 16:28:10 +0000 (18:28 +0200)]
BUG 8602: Skip initial fill of idints

Change-Id: If197c9b2318a52b3608f6065bea44af860a09849
Signed-off-by: Tomas Cere <tcere@cisco.com>
(cherry picked from commit 09630b9ae171a976301a795e745044ae58812df7)

2 years agoBug 2890: Chunk AppendEntries when single payload size exceeds threshold 01/57301/12
Tom Pantelis [Thu, 18 May 2017 13:23:04 +0000 (09:23 -0400)]
Bug 2890: Chunk AppendEntries when single payload size exceeds threshold

Utilizes the MessageSlicer in AbstractLeader to slice/chunk AppendEntries
messages whose single log entry payloas exceeds the max size threshold.
The MessageAssembler is used in the Follower to re-assemble.

For efficiency, with multiple followers, the AbstractLeader reuses the
FileBackedOutputStream containing the serialized AppendEntries data.
However, since the MessageSlicer takes ownership of the FileBackedOutputStream
and cleans it up when slicing is complete, I added a
SharedFileBackedOutputStream class that maintains a usage count and
performs cleanup when the usage count reaches 0. The AbstractLeader maintains
a Map of SharedFileBackedOutputStream instances keyed by log index.

The FollowerLogInformation keeps track of whether or not slicing is in
progress for the follower. Same as with install snapshot, we only want to send
empty AppendEntries as heartbeats.

Change-Id: Id163944b9989f6cb39a6aaaa98d1f3c4b0026bbe
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoImprove ShardBackendInfo.toString() 17/59617/2
Robert Varga [Wed, 28 Jun 2017 08:24:17 +0000 (10:24 +0200)]
Improve ShardBackendInfo.toString()

Slight update to eliminate a space from the property name and
an explicit present/absent string.

Change-Id: I9cb3a57049737c8ea25d22263140ff9974e23502
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 741013a2d48a4d08f83082c4e3cff79f59d17dde)

2 years agoBUG-8445: ignore responses from mismatched sessions 16/59616/1
Robert Varga [Wed, 28 Jun 2017 08:11:47 +0000 (10:11 +0200)]
BUG-8445: ignore responses from mismatched sessions

We have to check the session ID of the response in order not to
wreck transmit consistency if face of leader changes and reconnects.

If we reconnect the connection to the new leader before we saw all
responses from the old leader, we end up in a situation where the
old leader completes some of the replayed messages before we either
send them to the new leader or receive (the correct) reply.

Guard against this by checking the session ID before attempting to
pair a response to a request.

Change-Id: I28fa98b89c679715c3a0c546962d00533e76aa5d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 0ea09c71a5902f1ebf27ad683be634ded773e2c7)

2 years agoRemove EmptyQueue 25/59525/2
Robert Varga [Mon, 26 Jun 2017 14:17:21 +0000 (16:17 +0200)]
Remove EmptyQueue

This class is not used anywhere, remove it. If this functionality
is needed somewhere, use yangtools.util.EmptyDeque instead.

Change-Id: I12414fd2a2a5b4e7ac8b73fe70e8aa3dc929d025
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBug 7449: Move Dispatchers to sal-clustering-commons 63/59463/3
Tom Pantelis [Fri, 23 Jun 2017 11:49:48 +0000 (07:49 -0400)]
Bug 7449: Move Dispatchers to sal-clustering-commons

Moved Dispatchers to a common package so it can be used in
cds-access-client.

Change-Id: I4cea4c586dded9e413c1feee698b04d981b19ea2
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBug 7449: Introduce ClientActorConfig in cds-access-client 06/59406/5
Tom Pantelis [Thu, 22 Jun 2017 13:28:03 +0000 (09:28 -0400)]
Bug 7449: Introduce ClientActorConfig in cds-access-client

Upcoming changes in cds-access-client will need access to some config
params in DatastoreContext. However DatastoreContext is in
sal-distributed-datastore and thus can't be referenced in cds-access-client.
So refactor a ClientActorConfig interface with the necessary accessors.

Change-Id: I55e7291340e711c585f4fb1236a27396503d1914
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBug 7449: Add custom dispatcher for message serialization/slicing 97/59297/4
Tom Pantelis [Wed, 21 Jun 2017 15:11:49 +0000 (11:11 -0400)]
Bug 7449: Add custom dispatcher for message serialization/slicing

This will be used in subsequent patches to parallelize message
message serialization/slicing.

Change-Id: I6b89e1a61e10d743b24d834807d5a27bfc6e5c2b
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBUG 8629: Try to allow notification processing to finish in unsubscribe of listeners. 12/59512/1
Tomas Cere [Wed, 14 Jun 2017 13:42:07 +0000 (15:42 +0200)]
BUG 8629: Try to allow notification processing to finish in unsubscribe of listeners.

Change-Id: I8638c6066b86b101484d3d80cd0fed146a478778
Signed-off-by: Tomas Cere <tcere@cisco.com>
(cherry picked from commit bc5486e6d9fab8f550be8b72874ce96a9eb52651)

2 years agoBug 8740: startup archetype should default to Yang 1.1 85/59485/2
Ryan Goulding [Fri, 23 Jun 2017 19:09:13 +0000 (15:09 -0400)]
Bug 8740: startup archetype should default to Yang 1.1

Default to yang 1.1 and showcase the awesome work our yangtools
developers put in for 1.1 support.

Change-Id: I5f8320baca375f4e57f036840b27d7270191f530
Signed-off-by: Ryan Goulding <ryandgoulding@gmail.com>
2 years agoBug 8739: Autogenerate date for yang in archetype 83/59483/2
Ryan Goulding [Fri, 23 Jun 2017 19:02:16 +0000 (15:02 -0400)]
Bug 8739: Autogenerate date for yang in archetype

A couple of lines to autogenerate the date based on the day that
the archetype is run.  The date is then formatted appropriately
and used as the yang revision.  This helps keep the yang data
model writer honest.

Change-Id: Ic885b71a777119702b3ce78a21623298c44ad9c1
Signed-off-by: Ryan Goulding <ryandgoulding@gmail.com>
2 years agoBug 8735: Remove dlux deps from startup archetype 76/59476/1
Ryan Goulding [Fri, 23 Jun 2017 16:50:07 +0000 (12:50 -0400)]
Bug 8735:  Remove dlux deps from startup archetype

Since dlux is in maintenance mode, remove the dlux dependency from
the opendaylight-startup-archetype and remove the corresponding
-ui feature.

Change-Id: I78f43b90d718a10e47de99d54cd4952acc699470
Signed-off-by: Ryan Goulding <ryandgoulding@gmail.com>
2 years agoBug 7449: Add maximum-message-slice-size config param 94/59294/3
Tom Pantelis [Wed, 21 Jun 2017 14:53:53 +0000 (10:53 -0400)]
Bug 7449: Add maximum-message-slice-size config param

Added a new maximum-message-slice-size config param that will be used
when fragmenting messages thru the akka remoting framework. This is
a generalized version of the shard-snapshot-chunk-size param and
replaces it.

Change-Id: I4dc4cc0de92d6f876e5587cd8cb3ade2abb59285
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBug 8621 - Add shutdown-prefix-shard-replica rpc to MdsalLowLevelTestProvider 78/58578/5
Jakub Morvay [Fri, 9 Jun 2017 07:45:14 +0000 (09:45 +0200)]
Bug 8621 - Add shutdown-prefix-shard-replica rpc to MdsalLowLevelTestProvider

csit testing scenarios require clean shutdown of shard's local replica
funcionality. This introduces shutdown-prefix-shard-replica rpc to
MdsalLowLevelTestProvider. Upon invoking this rpc, local replica of
specified prefix-based shard is gracefully stopped.

Change-Id: I620b7ae2dbc9978dd155c64f703d421d46108e3d
Signed-off-by: Jakub Morvay <jmorvay@cisco.com>
(cherry picked from commit bdf02e09c13b6c8d170202054d44877707642cd9)

2 years agoBug 8621 - Add shutdown-shard-replica rpc to MdsalLowLevelTestProvider 76/58576/4
Jakub Morvay [Fri, 9 Jun 2017 07:12:20 +0000 (09:12 +0200)]
Bug 8621 - Add shutdown-shard-replica rpc to MdsalLowLevelTestProvider

csit testing scenarios require clean shutdown of shard's local replica
funcionality. This introduces shutdown-shard-replica rpc to
MdsalLowLevelTestProvider. Upon invoking this rpc, local replica of
specified module-based shard is gracefully stopped.

Change-Id: Ia8e0be65ecc99f9e208ff4ffd737b210437a9f51
Signed-off-by: Jakub Morvay <jmorvay@cisco.com>
(cherry picked from commit d5fcf5d66568519595b533cc20651634d66d34fb)

2 years agoBump to odlparent 2.0.0 76/50176/13
Robert Varga [Wed, 21 Jun 2017 13:40:28 +0000 (15:40 +0200)]
Bump to odlparent 2.0.0

This takes odlparent 2.0.0, adjusts for guava update and feature
movement. Since jenkins is failing on the distro run, that is
disabled and a follow-up patch will re-enable it to get us going
again.

Change-Id: If3e1289ed7f73a79a5a47428c634bda9702e824d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBUG-8494: propagate submit failure immediately 80/59280/2
Robert Varga [Tue, 20 Jun 2017 14:06:01 +0000 (16:06 +0200)]
BUG-8494: propagate submit failure immediately

Rather than waiting for abort to complete, which cannot happen
during isolation for example, propagate timeout immediately.

Change-Id: I90333938cb951f3b478320c682c65be219660fdf
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b6a43d9e31f6300fe35a27ecd1830a044b7cceb9)

2 years agoRetrofit DOMSchemaService into SchemaService 92/57592/15
Robert Varga [Sun, 21 May 2017 14:34:21 +0000 (16:34 +0200)]
Retrofit DOMSchemaService into SchemaService

For migration purposes we need to retrofit SchemaService so that
in extends MDSAL's DOMSchemaService. Also allow datastores to
be instantiated with DOMSchemaService.

Change-Id: Ie71732fb09f4da6dbc2d0819931d5ade2356d6f2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBug 4290 related: Remove <version> from org.osgi.service.event dep. 64/58364/4
Michael Vorburger [Tue, 6 Jun 2017 18:16:38 +0000 (20:16 +0200)]
Bug 4290 related: Remove <version> from org.osgi.service.event dep.

Because in c/58362 that was added to oldparent dependencyManagement, and
it seems wiser to declare this in only a single place, to avoid possible
version discrepancy problems re. this artifact in the future.

Change-Id: Ic1062efa211c6c5be4ece11a16c0191489d0bf42
Signed-off-by: Michael Vorburger <vorburger@redhat.com>
2 years agoFix (and suppress) some static code analysis warnings in blueprint 98/56098/4
Michael Vorburger [Wed, 26 Apr 2017 16:26:18 +0000 (18:26 +0200)]
Fix (and suppress) some static code analysis warnings in blueprint

Change-Id: I39a2c07176406d469f18949b30abb3deb3c21f6c
Signed-off-by: Michael Vorburger <vorburger@redhat.com>
2 years agoBUG-6709: Remove odl-karaf-empty 55/58655/5
Thanh Ha [Sat, 10 Jun 2017 03:04:48 +0000 (23:04 -0400)]
BUG-6709: Remove odl-karaf-empty

Projects should be using the odlparent version instead.

Change-Id: Ief85bf7e35aea5c9e85a07bdae8d47c29219ccdc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoBug 7449: Add message slicing/re-assembly classes 67/55767/26
Tom Pantelis [Thu, 20 Apr 2017 13:10:05 +0000 (09:10 -0400)]
Bug 7449: Add message slicing/re-assembly classes

Added re-usable classes for slicing messages into smaller chunks and
re-assembling them. The process is similar to the current raft
install snapshot chunking.

The 2 main classes are MessageSlicer and MessageAssembler. The basic workflow
is:

 - MessageSlicer#slice method is called which serializes the message,
   creates and caches a SlicedMessageState instance and determines the number
   of slices based on the serialized size and the maximum message slice size,
   then sends the first MessageSlice message.
 - The MessageAssembler on the other end receives the MessageSlice, creates an
   AssembledMessageState instance if necessary and appends the byte[] chunk to
   the assembled stream. A MessageSliceReply is returned.
 - The MessageSlicer receives the MessageSliceReply and sends the next
   MessageSlice chunk, if necessary.
 - Once the last MessageSlice chunk is received by the MessageAssembler, it
   re-assembles the original message by de-serializing the assembled stream
   and notifies the user-supplied callback (of type Consumer<Object>) to handle
   the message.

Both MessageSlicer and MessageAssembler use a guava Cache and can be configured
to evict state that has been inactive for a period of time, ie if a message hasn't
been received by the other end.

The MessageSliceReply can propagate a MessageSliceException. If the
MessageSliceException indicates it's re-triable, the MessageSlicer will restart
slicing from the beginning. Otherwise slicing is aborted and the user-supplied
failure callback is notified.

Change-Id: Iceea212b12f49c3944bade50afded92244e4b31a
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoBUG-8445: Guard against NPE 94/59194/2
Robert Varga [Mon, 19 Jun 2017 12:15:10 +0000 (14:15 +0200)]
BUG-8445: Guard against NPE

We have observed this NPE:

[...]
Caused by: java.lang.NullPointerException
        at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:810)
        at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:105)
        at org.opendaylight.controller.cluster.datastore.ChainedCommitCohort.canCommit(ChainedCommitCohort.java:58)
        at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:384)
        at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:527)
        at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:174)
        at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:141)

Which is quite weird, as the FrontendReadWriteTransaction state seems
to indicate the transaction is ready to be committed, yet ShardDataTree
does not seem to have a record of it.

While we are investigating the root cause, this patch adds an explicit
warning when this happens.

Change-Id: I2ddff76357c33d7df2b3f25a2703c69715fbd871
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit da09aa70325ef2c6f34de73b7cf10fc63410ace2)

2 years agoLower UnboundedDequeBasedControlAwareMailbox logging 93/59193/1
Robert Varga [Mon, 19 Jun 2017 11:53:26 +0000 (13:53 +0200)]
Lower UnboundedDequeBasedControlAwareMailbox logging

Using debug logging seems excessive, leading to a lot of messages
at debug level. I think we can downgrade to trace instead.

Change-Id: I2a7f87760a1eefe9794eac3b4025b6a3891c30a3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 27193873ccddbdc8126a24cdb9b0536c5e98ae5f)

2 years agoEnforce non-null entries field in AppendEntries 52/59052/2
Tom Pantelis [Fri, 16 Jun 2017 03:13:06 +0000 (23:13 -0400)]
Enforce non-null entries field in AppendEntries

The List<ReplicatedLogEntry> entries field has to be non-null
so enforce it with PreCondition. Same with leaderId. Callers
of getEntries do not need to check for null.

Change-Id: I5cd404d54e453e43456952ea2e11ea7f8f1c626c
Signed-off-by: Tcm Pantelis <tompantelis@gmail.com>
2 years agoConvert message-bus to blueprint 09/59109/2
Tom Pantelis [Fri, 16 Jun 2017 20:40:53 +0000 (16:40 -0400)]
Convert message-bus to blueprint

Change-Id: I2dcabedf8a5fa05ca7433573c4d1957884154161
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
2 years agoOptimize Follower.isOutOfSync() 49/59049/1
Robert Varga [Thu, 15 Jun 2017 16:10:22 +0000 (18:10 +0200)]
Optimize Follower.isOutOfSync()

This is a fast-path method which does a few duplicate checks
and calculations that may end up being unnecessary.

Restructure it so we check each partial condition just once
and compute required inputs only when we are going to need them.

Change-Id: I67a0089693a2ba1cd8c06c43504266534090545b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d3c5dc3b0f6bea3fa1c2f964353b87d1a9fcaef8)

2 years agoBUG-8618: update sync status only after processing 41/59041/2
Robert Varga [Thu, 15 Jun 2017 15:45:50 +0000 (17:45 +0200)]
BUG-8618: update sync status only after processing

Since the commitIndex may move in chunks we really want to update
our sync status after we have gone through the AppendEntries message
so our commitIndex reflects the state after processing.

Change-Id: I49c72a21f8d9c3efb7ae9cc1b64276220057f2e2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit bdb818fbfc5f015ab14883348f170cca8ce79128)

2 years agoBUG-8618: make sync threshold tuneable 40/59040/1
Robert Varga [Thu, 15 Jun 2017 01:13:47 +0000 (03:13 +0200)]
BUG-8618: make sync threshold tuneable

We are observing quite a few of these transitions, which may be coming
from batching scenarios. Introduce sync-index-threshold config knob
to expose control over it.

Change-Id: Ief4c89c2fe5b95cebaf3fb83cbcdda37cac126b6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 890e4bbf40aee318a2174bd4130cf34437e5617b)

2 years agoBUG-8618: improve debug logs 25/59025/2
Robert Varga [Thu, 15 Jun 2017 01:12:56 +0000 (03:12 +0200)]
BUG-8618: improve debug logs

We can have a reasonable ID prepended, add that. Also improve range
of threshold parameter, as we are addressing journal entries here.

Change-Id: I86aac1be04df8b72bfa6ffaa2b7a7e3b4cbfad6e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 22e817f688bb73420ddcac1f20cf71379ff3a508)

2 years agoLog data after in IdIntsDOMDataTreeLIstener 29/58929/2
Vratko Polak [Tue, 13 Jun 2017 09:49:54 +0000 (11:49 +0200)]
Log data after in IdIntsDOMDataTreeLIstener

At TRACE level.

Change-Id: Ic71aeec4c121d5cfb53a09762c9845e3e94f4f04
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit 8e1dd830bcc8f0b5d34192ebf9a8d45a165a90b1)

2 years agoBUG-8618: refactor SyncStatusTracker state 17/59017/1
Robert Varga [Wed, 14 Jun 2017 23:29:01 +0000 (01:29 +0200)]
BUG-8618: refactor SyncStatusTracker state

Introducing a leader target encapsulation allows us to
enfore state transitions (i.e. state is guaranteed to be
non-null when we need its bits).

This enables us to eliminate the need for a magic constant.

Change-Id: Iab7178694edc3c62032e32c4386c371630f67b6f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2c42c1d35a45fb7eced5a3ffd07b52bdc26f7e40)

2 years agoBUG-8618: make sure we refresh backend info 89/58989/2
Robert Varga [Wed, 14 Jun 2017 10:38:20 +0000 (12:38 +0200)]
BUG-8618: make sure we refresh backend info

When we are performing a reconnection attempt we must never use
previous backend info, but rather have to refresh it.

Fix this by removing state when resolution fails.

Change-Id: I65592f2101547a606a15d9c8030c7d8c58afe8a5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a816f13940862759cefcd6e69330c6a70b512ee2)

2 years agoBUG-8618: add threshold crossing debugs 88/58988/1
Robert Varga [Wed, 14 Jun 2017 23:00:43 +0000 (01:00 +0200)]
BUG-8618: add threshold crossing debugs

We are observing messages about sync status changing on the order
of 10s a second (14ms between messages). This looks awfully like
inter-node latency, hence it needs to be tuneable.

We do not have an understading of what sort of jumps are we talking
about, so add logging to the source of this events at debug, so these
can be diagnosed.

Change-Id: I9e2d78629f8808914cdb664cb28afcd47a55ee80
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 97875ef2635a536c0663750503ea6e64486d2fe6)

2 years agoImprove timeout message 36/58836/2
Robert Varga [Tue, 6 Jun 2017 09:19:07 +0000 (11:19 +0200)]
Improve timeout message

Rather than reporing nanoseconds, convert them to fraction seconds.

Change-Id: I9052462990f8c6b99349ed123f682ce3f0e23461
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 715bf60ac1899a3c01690d244d26b12c9212ecc7)

2 years agoPull in pax-exam-features 31/58931/1
Robert Varga [Wed, 14 Jun 2017 09:45:34 +0000 (11:45 +0200)]
Pull in pax-exam-features

BGPCEP downstream is using mdsal-it-base and it is failing to bring
up the container, as the features are not present in local repo.

Add the appropriate declaration, which fixes the breakage downstream.

Change-Id: Id3c21addb628ff37fb50e705773b9e4db83d17bc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
2 years agoMigrate to odlparent 1.9.0 45/58845/2
Thanh Ha [Tue, 13 Jun 2017 18:40:56 +0000 (14:40 -0400)]
Migrate to odlparent 1.9.0

Change-Id: I7d4af74e7713d48dd6ad8431229c4963423abbf6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
2 years agoRemove filter-valve 43/58843/3
Tom Pantelis [Tue, 13 Jun 2017 18:34:16 +0000 (14:34 -0400)]
Remove filter-valve

Change-Id: I9d6449a1e6337626373896e4766ea5a71ccee8ff
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>

©2013 OpenDaylight, A Linux Foundation Collaborative Project. All Rights Reserved.
OpenDaylight is a registered trademark of The OpenDaylight Project, Inc.
Linux Foundation and OpenDaylight are registered trademarks of the Linux Foundation.
Linux is a registered trademark of Linus Torvalds.