22 months agoSeparate lazy-versioned NormalizedNodeDataInput 78/82278/6
Robert Varga [Tue, 28 May 2019 13:28:55 +0000 (15:28 +0200)]
Separate lazy-versioned NormalizedNodeDataInput

NormalizedNodeInputStreamReader is really only one implementation
of the streaming format, which happens to be bound to LITHIUM_VERSION.

This effectively means NormalizedNodeInputOutput can give out either
a version-bound reader or an unbound reader -- which we facilitate by
creating VersionedNormalizedNodeDataInput which is a simple forwarder
with lazily-initialized delegate.

Since this increases the number of (NormalizedNode)DataInput
implementations, we share as much code as possible through the use
of forwarding implementations.

Change-Id: I28713346730ed5bf7f912bf8afa5d8722b202035
Signed-off-by: Robert Varga <>
22 months agoDo not format QNames to string on input 76/82276/4
Robert Varga [Tue, 28 May 2019 11:14:34 +0000 (13:14 +0200)]
Do not format QNames to string on input

Profiling has revelead the creation of String for QNameFactory
lookup is taking around 19% of overall cost of

Rather than using concatenation, use a dedicated Key object, which
holds the separated-out localname/namespace/revision strings,
thus allocating a small object instead of a full-blown String.

This improves snapshot deserialization perfomance on a 350MiB
snapshot by about 16%, also resulting in lower memory pressure.

Change-Id: If3e9e4b332b0a5acbe33aa10fca189bbc7da8c81
Signed-off-by: Robert Varga <>
22 months agoUpdate QNameFactory caching 75/82275/3
Robert Varga [Tue, 28 May 2019 10:42:02 +0000 (12:42 +0200)]
Update QNameFactory caching

This eliminates the use of soft references in favor of weak ones,
so that we do not rely on GC heuristics for retention.

We also make the maximum cache size runtime-configurable via

Finally we use QName.intern() to defer to the global QName cache,
resulting in better QName instance sharing.

Change-Id: Ice3596d23b96c4ecfc5775cb21968233b042a984
Signed-off-by: Robert Varga <>
22 months agoFix write transaction shard stats 89/82189/3
Tomas Cere [Tue, 21 May 2019 11:42:47 +0000 (13:42 +0200)]
Fix write transaction shard stats

With the changes in ask protocol due to to tell based
integration, shards stats had incorrectly reported
write transaction counts. Since backend doesnt care
about the difference between write-only and read-write
transactions and all are read-write from backend point of view
we can safely remove writeOnly counter.

Change-Id: I18046ad3b229755bbd0daa788a3fe62f59e4140d
Signed-off-by: Tomas Cere <>
22 months agoUse Files.delete() in LocalSnapshotStore 13/82213/2
Tomas Cere [Thu, 23 May 2019 08:04:16 +0000 (10:04 +0200)]
Use Files.delete() in LocalSnapshotStore

If delete fails, it needs to be logged so change this up.

Change-Id: If4051cb3e5b83dee51f919fc1713cc17f032597e
Signed-off-by: Tomas Cere <>
22 months agoDo not enque purges for ask based protocol 37/82137/16
Tomas Cere [Thu, 16 May 2019 12:12:01 +0000 (14:12 +0200)]
Do not enque purges for ask based protocol

With metadata disabled for ask based protocol, there is no need
to purge transaction on the backend and pollute the journal.

Tell based protocol has purges handled by the fronted so it
should be safe for us to remove them on the backend for ask

Change-Id: Ia5c9470782bb4d59b78bbaa605ee5d36c7fb1644
Signed-off-by: Tomas Cere <>
23 months agoRemove PersistAbortTransactionPayload 27/82027/14
Tomas Cere [Tue, 14 May 2019 10:56:04 +0000 (12:56 +0200)]
Remove PersistAbortTransactionPayload

With the metadata tracking disabled for ask based protocol
there is no need to track aborts for read only transactions
on the backend.

Change-Id: I189ae3231bb2f3c0eaa0bbe21a14342446708c5f
Signed-off-by: Tomas Cere <>
23 months agoDo not leak DataTree from backend actor 82/82182/1
Robert Varga [Tue, 21 May 2019 08:47:34 +0000 (10:47 +0200)]
Do not leak DataTree from backend actor

YANG tools is now exposing the read-only part of DataTree as
ReadOnlyDataTree. Use this interface to restrict the set of
methods accessible to frontend.

Change-Id: I9426d4cee1e68e3cc21f7a39322a0df09739e770
Signed-off-by: Robert Varga <>
23 months agoDisable transaction tracking for ask-based protocol 40/82140/8
Robert Varga [Thu, 16 May 2019 13:54:29 +0000 (15:54 +0200)]
Disable transaction tracking for ask-based protocol

When we encounter an ask-based protocol message on a leader,
we persist a payload to make sure no state tracking occurs
for that client.

Change-Id: I3d12a06ce9e5b65ada5b59bde9f9b5486e5e4ef7
Signed-off-by: Robert Varga <>
Signed-off-by: Tomas Cere <>
23 months agoAllow transaction tracking to be disabled 83/81983/13
Robert Varga [Fri, 10 May 2019 10:25:31 +0000 (12:25 +0200)]
Allow transaction tracking to be disabled

Ask-based protocol does not need tracking of transactions and
histories, as it is not retransmitting requests. It also does
not inform backend about purely-local aborted transactions
(read-write and read-only), which leads to transaction tracking
rangesets having holes where those IDs are used.

This adds the prerequisite handling of disabling from the leader
without adding the actual mechanics.

Change-Id: I133e7688b492336937f394f0f6c3f080a05a820f
Signed-off-by: Robert Varga <>
Signed-off-by: Tomas Cere <>
23 months agoRemove QueuedNotificationManagerMXBean 71/82171/1
Robert Varga [Mon, 20 May 2019 10:43:49 +0000 (12:43 +0200)]
Remove QueuedNotificationManagerMXBean

This interface is available from yangtools directly, remove
it from controller.

Change-Id: Ic894c0e9fc5d114ebff07bb11f67f1ec67611031
Signed-off-by: Robert Varga <>
23 months agoDerive config-artifacts from odlparent-lite 71/82071/1
Robert Varga [Wed, 15 May 2019 07:58:52 +0000 (09:58 +0200)]
Derive config-artifacts from odlparent-lite

This is to fix failures seen on promoting to Central.

Change-Id: I6204d68c5144437e5af50bb5f2cb29431841d394
Signed-off-by: Robert Varga <>
23 months agoBump mdsal to 4.0.1 23/81823/8
Robert Varga [Tue, 30 Apr 2019 10:55:09 +0000 (12:55 +0200)]
Bump mdsal to 4.0.1

Upgrade mdsal to 4.0.1 and remove most references to Javassist
and its associated streamer implementation.

Change-Id: Ia9dc2c886217f1c353db6523b654e993ed9a397a
Signed-off-by: Robert Varga <>
23 months agoRemove sal-dom-broker dependency on in-memory datastore 34/81934/5
Robert Varga [Tue, 7 May 2019 10:02:55 +0000 (12:02 +0200)]
Remove sal-dom-broker dependency on in-memory datastore

This dependency should be purely scope=test, so that we do not
depend on a particular datastore.

Change-Id: I24962d09b80d9f1a68ba777980d129387bc41380
Signed-off-by: Robert Varga <>
23 months agoSplit odl-mdsal-broker-local and odl-mdsal-broker features 33/81933/4
Robert Varga [Tue, 7 May 2019 09:50:00 +0000 (11:50 +0200)]
Split odl-mdsal-broker-local and odl-mdsal-broker features

These two features are supposed to be exclusive -- either we want
local-only services (-local) or we want clustered services.

Change-Id: I11c261b6c97471356645826c00ea052ffbe7b82b
Signed-off-by: Robert Varga <>
23 months agoCentralize read{Augmentation,Node}Identifier() methods 08/82008/3
Robert Varga [Mon, 13 May 2019 05:53:27 +0000 (07:53 +0200)]
Centralize read{Augmentation,Node}Identifier() methods

We are using the same construct in multiple places, make sure we
centralize it and mark a FIXME for some efficiency increase.

Change-Id: Ie65089a1a7bb6a2752887498d354a61cb32f0e40
Signed-off-by: Robert Varga <>
23 months agoMigrate deprecated yangtools method 10/82010/2
Robert Varga [Mon, 13 May 2019 07:26:34 +0000 (09:26 +0200)]
Migrate deprecated yangtools method

DataTreeCandidateNodes.fromNormalizedNode() has been deprecated
in favor of written() -- migrate this user.

Change-Id: I355058568c2c1512fbca083142b17a3a515e9329
Signed-off-by: Robert Varga <>
23 months agoPoison entries outside of main lock 49/81949/4
Robert Varga [Tue, 7 May 2019 17:42:26 +0000 (19:42 +0200)]
Poison entries outside of main lock

Poisoning entries may involve reaction from their callbacks, which
can attempt to circle back through connections.

Make sure we poison them outside of lock context, so that any
callbacks end up seeing a poisoned connection, but without the lock
being held -- hence the locks can be acquired in-order.

Change-Id: I26551d052307812e76f3e45024a77dbb83312b17
Signed-off-by: Robert Varga <>
23 months agoAllow SnapshotBackedReadTransaction customization 55/81955/1
Robert Varga [Wed, 8 May 2019 10:28:24 +0000 (12:28 +0200)]
Allow SnapshotBackedReadTransaction customization

In some specific cases we need to customize abort-like handling,
which is already implemented for write-like transactions, but is
not present for read-only transaction.

This patch adds the capability to attach a close() handler and
makes sure AbstractSnapshotBackedTransactionChain takes advantage
of it.

Change-Id: Ic7027956556b5dd25120ee81613a6151e5dbc501
Signed-off-by: Robert Varga <>
23 months agoEnqueue purge snapshot directly 45/81945/1
Robert Varga [Tue, 7 May 2019 14:23:19 +0000 (16:23 +0200)]
Enqueue purge snapshot directly

When we are sending abort+purge, make sure we propagate it out
as soon as possible.

Change-Id: I2b63aa20867a8b99b779115f0bb448c0e8ac9a22
Signed-off-by: Robert Varga <>
23 months agoMake sure we purge transaction on abort 39/81939/2
Robert Varga [Tue, 7 May 2019 11:22:10 +0000 (13:22 +0200)]
Make sure we purge transaction on abort

In case of read-only transactions it is not sufficient to just
persist the abort payload, we need to also purge the transaction.

Change-Id: I8761040d074743b5a1c5eebbecc6449c30abe47e
Signed-off-by: Robert Varga <>
23 months agoRefactor odl-mdsal-remoterpc-connector 17/81917/4
Robert Varga [Mon, 6 May 2019 17:48:51 +0000 (19:48 +0200)]
Refactor odl-mdsal-remoterpc-connector

RPC connector has a logical dependency on components present
in the distributed datastore. This refactors it to take that
into account.

Change-Id: Icd0145b35310133822ee02cdf4a3d4cffbc6e257
Signed-off-by: Robert Varga <>
23 months agoRefactor odl-mdsal-distributed-datastore 16/81916/3
Robert Varga [Mon, 6 May 2019 18:04:10 +0000 (20:04 +0200)]
Refactor odl-mdsal-distributed-datastore

odl-mdsal-distributed-datastore forms the baseline for supporting
akka-based cluster. This refactors it to pull its dependencies in
so it can be properly tested.

This also moves CommitStatsMXBean, so we do not depend on controller's
broker at all.

Finally we co-locate all configuration files into this feature, so
it can boot up.

Change-Id: I9319de78d61d63e3c49825e940af5c8c70657185
Signed-off-by: Robert Varga <>
23 months agoSplit out odl-controller-blueprint 14/81914/2
Robert Varga [Mon, 6 May 2019 17:39:00 +0000 (19:39 +0200)]
Split out odl-controller-blueprint

Blueprint only needs controller APIs, it does not need implementation
and it should be packaged in odl-config-netty.

This patch splits it out in a separately-tested feature

Change-Id: I4b12a3880b32fc21515dbb1e69f7e21b7c4d5c57
Signed-off-by: Robert Varga <>
23 months agoRefactor odl-mdsal-clustering-commons 13/81913/2
Robert Varga [Mon, 6 May 2019 16:41:46 +0000 (18:41 +0200)]
Refactor odl-mdsal-clustering-commons

odl-mdsal-clustering-commons does not have a dependency on
odl-mdsal-broker-local, hence we can factor it out to pull in
its upstream dependencies, thus fixing its tests.

Change-Id: Ie1df487329df8ea1295e0e2dc3b0e0f07e2e4037
Signed-off-by: Robert Varga <>
23 months agoSplit out odl-controller-binding-api 12/81912/2
Robert Varga [Mon, 6 May 2019 16:32:43 +0000 (18:32 +0200)]
Split out odl-controller-binding-api

This refactors packaging so that we package Binding-level APIs
in a corresponding feature -- mirroring MD-SAL's setup.

Change-Id: I92adc8f14462633a253136d6c37e708a30cc4a37
Signed-off-by: Robert Varga <>
23 months agoReset snapshot progress after timeout has been hit 16/81816/7
Tomas Cere [Tue, 30 Apr 2019 08:49:02 +0000 (10:49 +0200)]
Reset snapshot progress after timeout has been hit

Add a stopwatch to LeaderInstallSnapshotState that allows
us to keep track whether a follower took too much time to respond
to InstallSnapshot for whatever reason allowing us to retry
chunks that time out.

Change-Id: Id443a4cc7a069ad4d9982d537d9a8d82b845ac35
Signed-off-by: Tomas Cere <>
23 months agoSplit out odl-controller-dom-api 11/81911/1
Robert Varga [Mon, 6 May 2019 16:12:41 +0000 (18:12 +0200)]
Split out odl-controller-dom-api

This refactors packaging so that we package DOM-level APIs
in a corresponding feature -- mirroring MD-SAL's setup. We also fix
dependencies on MD-SAL features to use ranges instead of fixed

Change-Id: Ib5e0649552b34662a91b4fb32da9af69f589bb6b
Signed-off-by: Robert Varga <>
23 months agoFix memory leak in BA mount service 91/81891/5
Maros Marsalek [Fri, 3 May 2019 12:04:56 +0000 (14:04 +0200)]
Fix memory leak in BA mount service

The cache used in BA mount service uses weakKeys() so the keys (DOM mountpoint)
could be GCed when no ono, except cache, referenced them).
However the values in the cache (BA mountpoint) also kept a reference to
the key, so the key was never weakly reachable and thus never GCed
resulting in a memory leak.

Which is especially visible in case of frequent reconnects of netconf
mountpoints where after every reconnect a new DOM mountpoint is created,
requiring a new BA mountpoint creating new entry in the cache of BA mount service...

Added also a simple test verifying proper cache cleanup

Change-Id: I9c109dd6d499d6185842d6a1ad5a59d75565af5c
Signed-off-by: Maros Marsalek <>
23 months agoImprove segmented journal JMX beans 90/81890/1
Robert Varga [Fri, 3 May 2019 11:01:14 +0000 (13:01 +0200)]
Improve segmented journal JMX beans

This patch updates metric naming to properly tie them to individual
journals -- so that the statistics are actually useful.

Change-Id: I6f4f398894860566e234562d731462535b62801e
Signed-off-by: Robert Varga <>
23 months agoChange segmented journal naming 89/81889/1
Robert Varga [Fri, 3 May 2019 10:48:34 +0000 (12:48 +0200)]
Change segmented journal naming

Using Base64 encoding is not quite nice for debugging purposes,
use URLEncoder, so that we are consistent with snapshot store

Change-Id: I55f7bef79b7d5c14ab10180444770fedcc23eab9
Signed-off-by: Robert Varga <>
23 months agoHarden ShardCommitCoordinator to survive EOS write retries 90/81790/1
Tomas Cere [Fri, 26 Apr 2019 10:37:38 +0000 (12:37 +0200)]
Harden ShardCommitCoordinator to survive EOS write retries

EntityOwnershipShard can break the usual shard rule of only one 3PC
running at one time, causing the retrying write attempting to modify
an already sealed modification.

Change-Id: Iaa56b68a30f9191f7aa14428f39d4d7524aeb09d
Signed-off-by: Tomas Cere <>
2 years agoDo not allow persistence callbacks to throw Exception 40/81640/6
Robert Varga [Sat, 13 Apr 2019 12:55:13 +0000 (14:55 +0200)]
Do not allow persistence callbacks to throw Exception

Since we are indirecting through an executor, we are forced
to wrap any exception -- just do not bother, as the callbacks
are executed in the context of an actor anyway (dealing with
RuntimeExceptions) and users are not throwing checked exceptions.

Change-Id: I6cea19ab7192fa42ad3c346d554411cd0d558a64
Signed-off-by: Robert Varga <>
2 years agoBump mdsal to 4.0.0 30/81130/38
Robert Varga [Fri, 22 Mar 2019 18:33:21 +0000 (19:33 +0100)]
Bump mdsal to 4.0.0

This bumps mdsal to 4.0.0, adjusting for changes it brings, notably:
- FluentFuture removed from DOM RPC invocation services
- ping-pong functionality directly part of DOMDataBroker API and not
  via a dedicated DOMDataBroker instance
- DataContainer.getImplementedInterface() renamed to
- DOMBrokerReadWriteTransaction.close() method being removed
- BindingCodecTreeNode being refactored

Change-Id: I5af13aa82036373826291a54661f7fd20ff387c3
Signed-off-by: Robert Varga <>
2 years agoBump yangtools to 3.0.0 29/81129/31
Robert Varga [Fri, 22 Mar 2019 18:32:29 +0000 (19:32 +0100)]
Bump yangtools to 3.0.0

This is bumps yangtools to 3.0.0, adjusting for changed APIs. The
changes specifically include:

- use ListenerRegistry.getRegistrations() instead of it implementing
- adjust for NormalizedNodeStreamWriter interface change
- remove use of *AttrBuilder, as NormalizedNodes no longer hold
- DataTreeCandidateNodes.empty() instead of home-grown implementation
- adjust for DataTreeCandidateNode.getModifiedChild() returning Optional
- use QueuedNotificationManager.BatchedInvoker

Change-Id: Ia30d73ccf6380e3759414b3026b822df0950a481
Signed-off-by: Robert Varga <>
2 years agoBump odlparent to 5.0.0 28/81128/25
Robert Varga [Fri, 22 Mar 2019 18:31:50 +0000 (19:31 +0100)]
Bump odlparent to 5.0.0

This bumps odlparent references to 5.0.0, removing use of the JSR305
remnants. Also a useless javadoc plugin override is removed.

Change-Id: Iae5355e49115afdd73b60bd26ac85f7f542bbff8
Signed-off-by: Robert Varga <>
2 years agoDo not break actor containment 82/81482/7
Tomas Cere [Tue, 9 Apr 2019 08:46:18 +0000 (10:46 +0200)]
Do not break actor containment

During async persistence the actor containment is broken which
results in applyState being called directly from the persistence actor.
This means that the ClientRequestTracker might be missing an entry
for the transaction and the transaction being applied as a foreign candidate
in ShardDataTree.
If this happens there will be a transaction stuck in COMMIT_PENDING state,
blocking any further progress in the shard until its restarted.

Change-Id: I944b233d13103df08b68baeaf3907c064d1d526e
Signed-off-by: Tomas Cere <>
Signed-off-by: Robert Varga <>
2 years agoIntegrate SegmentedJournalActor with metrics/JMX 53/81353/1
Robert Varga [Tue, 2 Apr 2019 21:40:20 +0000 (23:40 +0200)]
Integrate SegmentedJournalActor with metrics/JMX

This adds the wiring needed to expose segmented journal metrics
through JMX.

Change-Id: I42f0b29985410a34ec91639ce8be2903bbe7dd5b
Signed-off-by: Robert Varga <>
2 years agoFixup HTML5 javadoc compatibility 39/81339/1
Robert Varga [Mon, 1 Apr 2019 18:05:15 +0000 (20:05 +0200)]
Fixup HTML5 javadoc compatibility

This fixes use of 'summary' attribute.

Change-Id: I872afdef8b678a68f4e1781330c3f11f0e010925
Signed-off-by: Robert Varga <>
2 years agoMigrate nullness annotations 18/81318/2
Robert Varga [Fri, 29 Mar 2019 01:53:09 +0000 (02:53 +0100)]
Migrate nullness annotations

This mass-migrates sal-distributed-datastore to use JDT nullness
annotations instead of JSR305.

Change-Id: I2cd739e3db8c074a55e335e17297a23065775d6a
Signed-off-by: Robert Varga <>
2 years agoMigrate to AbstractActor 96/80396/8
Robert Varga [Tue, 19 Feb 2019 14:58:28 +0000 (15:58 +0100)]
Migrate to AbstractActor

UntypedActor has been deprecated, this patch migrates to using
AbstractActor so we eliminate a slew of deprecatation warnings.

Change-Id: I4e3274ce92d5f68df1937f91f823eed905da4d6c
Signed-off-by: Robert Varga <>
Signed-off-by: Ajay Lele <>
2 years agoFix checkstyle in toaster-it 69/81269/1
Robert Varga [Thu, 28 Mar 2019 18:54:43 +0000 (19:54 +0100)]
Fix checkstyle in toaster-it

Simple whitespace cleanup to keep checkstyle happy.

Change-Id: Id51f7b87b632cce92c29cbae8794662fe6feb250
Signed-off-by: Robert Varga <>
2 years agoFix transaction mocking 62/81262/1
Robert Varga [Thu, 28 Mar 2019 16:55:17 +0000 (17:55 +0100)]
Fix transaction mocking

We should not confuse FluentFutures and CheckedFutures, as the latter
is deprecated and with Guava 27 they are not compatible anyway.

Change-Id: I7489b4470df6fa435aab190b6e38eaf154522fe7
Signed-off-by: Robert Varga <>
2 years agoMigrate to use openCursor() 61/81261/1
Robert Varga [Thu, 28 Mar 2019 16:26:24 +0000 (17:26 +0100)]
Migrate to use openCursor()

createCursor() has been deprecated long time ago, use its replacement

Change-Id: Id8862470ca4ad8c9fdec5c05cbac044c6715e153
Signed-off-by: Robert Varga <>
2 years agoFix logging messages 60/81260/5
Robert Varga [Thu, 28 Mar 2019 13:32:44 +0000 (14:32 +0100)]
Fix logging messages

Upgraded spotbugs is finding more violations, fix them up.

Change-Id: I8956c5b7a6eef38286f4c2b6fb07ca6a775e6965
Signed-off-by: Robert Varga <>
2 years agoUse ReusableNormalizedNodePruner in PruningDataTreeModification 58/81258/5
Robert Varga [Thu, 28 Mar 2019 12:01:48 +0000 (13:01 +0100)]
Use ReusableNormalizedNodePruner in PruningDataTreeModification

Rather than instantiating a pruner for each invocation, keep an
instance around for reuse. In order to facilitate free-standing
modifications, we also keep a reference to the DataSchemaContextTree
bound our current SchemaContext.

Change-Id: I5d310b12feb1ff46dc2af0e67294a925d48a4957
Signed-off-by: Robert Varga <>
2 years agoAdd ReusableNormalizedNodePruner 57/81257/4
Robert Varga [Thu, 28 Mar 2019 11:46:34 +0000 (12:46 +0100)]
Add ReusableNormalizedNodePruner

This version of NormalizedNodePruner is reusable across invocations,
and improving CPU and memory efficiency in bulk operations due to
not needing to allocate stack nor lookup DataSchemaContextTree.

Change-Id: I1798f5f6255cd23702bf3792c1f4f5149f92d208
Signed-off-by: Robert Varga <>
2 years agoMove checkNotSealed() to addBuilder() 51/81251/2
Robert Varga [Thu, 28 Mar 2019 10:22:55 +0000 (11:22 +0100)]
Move checkNotSealed() to addBuilder()

addBuilder() is called after previous check for seal from all callers,
who do not have any side-effects aside allocating arguments to

Move the call to checkNotSealed() to addBuilder(), making the code
more straightforward and possibly preventing bugs from creeping in.

Change-Id: I127385378957deb7f38f1203870a0e52cd630644
Signed-off-by: Robert Varga <>
2 years agoRemove NormalizedNodePruner.findSchemaNodeForNodePath() 50/81250/2
Robert Varga [Thu, 28 Mar 2019 10:16:27 +0000 (11:16 +0100)]
Remove NormalizedNodePruner.findSchemaNodeForNodePath()

DataSchemaContextTree.findChild() provides equivalent functionality,
so use that instead of brewing our own.

Change-Id: Ia55ac99ce8fa37daf06ce0e7e1615f1bd0a07123
Signed-off-by: Robert Varga <>
2 years agoDo not use Optional in NormalizedNodeWrapper 49/81249/2
Robert Varga [Thu, 28 Mar 2019 09:58:49 +0000 (10:58 +0100)]
Do not use Optional in NormalizedNodeWrapper

Use nullable schema instead, as the caller is package-private and
ends up unpacking the return anyway. This simplifies
NormalizedNodePruner a bit.

Change-Id: I395dd8266bd7198f66890c4ae3633ecda9281726
Signed-off-by: Robert Varga <>
2 years agoHide NormalizedNodeBuilderWrapper 48/81248/2
Robert Varga [Thu, 28 Mar 2019 07:48:02 +0000 (08:48 +0100)]
Hide NormalizedNodeBuilderWrapper

This class is not instantiable outside of the package is it does
not leak from NormalizedNodePruner -- therefore it cannot be used
by anyone else. Make sure it's package-private and clean it up
a bit.

Change-Id: I17911508273d68c2795d38e2156c9407522f55fe
Signed-off-by: Robert Varga <>
2 years agoRemove NormalizedNodePruner.SimpleStack 34/81234/2
Robert Varga [Wed, 27 Mar 2019 22:51:57 +0000 (23:51 +0100)]
Remove NormalizedNodePruner.SimpleStack

This custom implementation is highly inefficient, because:
1) peek/pop operations perform get(int) and remove(int) operations
   on ArrayList, which are highly inefficient O(N)
2) it allocates an object for each entry

Replace this class with a stock Deque, which takes care of the first
problem, as push/pop/peek are amortized O(1).

For implementation use an ArrayDeque, which takes care of the second
problem, as stack entries are reused.

Change-Id: Ie59e2fb47f841e4e5fc08c3ab7e164ad0cb368ad
Signed-off-by: Robert Varga <>
2 years agoClean up NormalizedNodePruner 33/81233/1
Robert Varga [Wed, 27 Mar 2019 22:45:15 +0000 (23:45 +0100)]
Clean up NormalizedNodePruner

- shorten YangInstanceIdentifier.PathArgument references
- do not declare runtime exceptions as thrown
- use a space after comma

Change-Id: Id0f09d2cd72bc619331c2d16e8f184db784c9976
Signed-off-by: Robert Varga <>
2 years agoDeleting a non-existent node fails 84/28684/8
Valentin Mayamsin [Fri, 16 Oct 2015 22:08:34 +0000 (15:08 -0700)]
Deleting a non-existent node fails

This adds a unit test for deletion of a non-existent node.

Change-Id: I32d537c7e58a6161147e7e51b80e71cb8aaa0d79
Signed-off-by: Valentin Mayamsin <>
Signed-off-by: Robert Varga <>
2 years agoFix test mocking 27/81127/1
Robert Varga [Fri, 22 Mar 2019 19:40:03 +0000 (20:40 +0100)]
Fix test mocking

MD-SAL store APIs require FluentFuture, not CheckedFuture, fix that

Change-Id: I2222c6a56f8b7a1bcc6569ba84e01ce3b4904a51
Signed-off-by: Robert Varga <>
2 years agoUpdate javadoc links 26/81126/1
Robert Varga [Fri, 22 Mar 2019 19:32:32 +0000 (20:32 +0100)]
Update javadoc links

Our javadoc links are pointing to deprecated Guava methods, update
them to point to their equivalents, as they are going away in our
next upgrade.

Change-Id: Ia82471897ba860f38be45219676b44f3d55d977c
Signed-off-by: Robert Varga <>
2 years agoFix logging arguments 25/81125/1
Robert Varga [Fri, 22 Mar 2019 19:32:08 +0000 (20:32 +0100)]
Fix logging arguments

Upgraded spotbugs is finding these violations, fix them up.

Change-Id: I6f7490cd1d48c6e22d048842265bf71d0ff24afc
Signed-off-by: Robert Varga <>
2 years agoAdd missing license headers 64/68764/3
Stephen Kitt [Mon, 26 Feb 2018 12:02:44 +0000 (13:02 +0100)]
Add missing license headers

Change-Id: Ie8405e6f1de698c958d347cfb53380a111d6a9ef
Signed-off-by: Stephen Kitt <>
2 years agoUse a simple list for string code table 89/80889/1
Robert Varga [Fri, 15 Mar 2019 11:33:05 +0000 (12:33 +0100)]
Use a simple list for string code table

Since the codes are assigned in a linear fashion, we can use
a simple ArrayList to track assignes codes, preventing the need
for boxing integer during population and lookups. Lookups also
end up requiring an array offset lookup rather than needing
to go through multiple indirections.

While this can in theory lead to a large array being allocated,
the set of strings we keep in the table is limited by the
SchemaContext and thus will be typically capped ad couple of

The cost of growing the lookup should not be significantly
higher, even if ArrayList grows slower:
- 10, 16, 25, 38,  58,  88
versus HashMap (with .75 load factor)
- 12, 24, 48, 96, 192, 384

each resize operation is significantly faster.

Change-Id: I00c7a2e0985dbf901b6857a16424bf157e5af39d
Signed-off-by: Robert Varga <>
2 years agoImprove NormalizedNodeInputStreamReader defensiveness 88/80888/2
Robert Varga [Fri, 15 Mar 2019 11:20:24 +0000 (12:20 +0100)]
Improve NormalizedNodeInputStreamReader defensiveness

If face of corrupted streams, for example due to CONTROLLER-1752,
we can end up interpreting unknown bytes as null strings, potentially
corrupting data.

This patch uses explicit type dispatch and detects wrong string
value types, as well as rejecting unknown string value codes.

Change-Id: Ic2e7887bc1d32a1dc7a364d1bff195752edd762d
Signed-off-by: Robert Varga <>
2 years agoAdd integration test for segmented journal 23/80823/3
Tomas Cere [Wed, 13 Mar 2019 11:50:55 +0000 (12:50 +0100)]
Add integration test for segmented journal

Extracts test cases for DistributedDataStore that do not need latching
in InMemoryJournal/Snapshot so we can have these run with segmented journal.

Change-Id: I0ea4e58bff0f0e8e1a40bd4f26b827623139d6c9
Signed-off-by: Tomas Cere <>
2 years agoFix segmented journal replay 22/80822/1
Tomas Cere [Tue, 12 Mar 2019 09:54:47 +0000 (10:54 +0100)]
Fix segmented journal replay

lastDelete was getting set with the incorrect value from the delete journal.
The actual written value from the delete journal needs to be used
instead of the index.
dataJournal compact also needs to be compacted upto lastDelete + 1 since
deleteUpTo is inclusive and compact keeps this value around.

Change-Id: I4a678be67fd1ad09c57273ef0fc3b7398a7d714f
Signed-off-by: Tomas Cere <>
2 years agoIntroduce candidate election timeout divisor 63/80663/2
Robert Varga [Fri, 1 Mar 2019 20:45:48 +0000 (21:45 +0100)]
Introduce candidate election timeout divisor

When a candidate is waiting for an election to occur, it is sometimes
preferable to shorten the timeout interval down, so that we do not
wait the full election timeout.

This patch introduces the knob which serves to divide the normal
timeout -- i.e. it can be halved.

Change-Id: Idbe81c39bf7aecf56b9f13a242bea89a1a4ac4f1
Signed-off-by: Robert Varga <>
2 years agoPrevent a CME during actor instantiation 65/80665/3
Robert Varga [Mon, 4 Mar 2019 12:02:36 +0000 (13:02 +0100)]
Prevent a CME during actor instantiation

This issue was missed in review: createHandler() should not be
touching handlers, as it is already called from computeIfAbsent()
and hence would result in a CME with JDK11:

Change-Id: I8d4940b8bec2cf727d97d68796f1c9cdaaaf89d3
Signed-off-by: Robert Varga <>
2 years agoAdd SegmentedFileJournal 68/80368/10
Robert Varga [Mon, 4 Feb 2019 09:09:57 +0000 (10:09 +0100)]
Add SegmentedFileJournal

This adds an alternative Akka persistence journal implementation
based on Atomix's SegmentedJournal.

SegmentedJournal is a linear append-only log of entries, each of
which has a incrementing 63bit index. This logical structure is
stored in rolling segments -- i.e. entries are appended to current
segment until it is full, at which point a new segment is allocated
and writeout continues there. Old segments can be freed as long as
they are not needed.

This layout makes it good match to how Akka's persistence works,
with the one exception that the SegmentedJournal cannot explicitly
delete entries from the head of the journal.

SegmentedFileJournal allocates one SegmentedJournal for each
persistenceId it encounters and uses a dedicated actor for it. This
provides a major simplification in the implementation as well as
allows for concurrent persistence of multiple PersistentActors.

Change-Id: I78140b154bab44a3e17d5ffb76b040c62add3204
Signed-off-by: Robert Varga <>
2 years agoRename ActorContext to ActorUtils 94/80394/4
Robert Varga [Tue, 19 Feb 2019 15:43:22 +0000 (16:43 +0100)]
Rename ActorContext to ActorUtils

ActorContext is overloaded name even within Akka, without us adding
to it. Furthermore Akka's AbstractActor defines ActorContext as its
nested class, which thoroughly breaks resolution priorities.

Rename ActorContext to ActorUtils, reducing confusion around class

Change-Id: I140239a8f74ee7deecf9ee848df0cfbbb72f3c4d
Signed-off-by: Robert Varga <>
2 years agoAdd controller aggregate javadocs 68/80568/2
Robert Varga [Sun, 24 Feb 2019 23:20:54 +0000 (00:20 +0100)]
Add controller aggregate javadocs

This adds aggregated javadocs for the controller projects.

Change-Id: If5efaa7d5dda94a7a6416230e6c2e61dd963bd90
Signed-off-by: Robert Varga <>
2 years agoDo not assert seal transition on forward path 11/80411/2
Robert Varga [Wed, 20 Feb 2019 12:56:30 +0000 (13:56 +0100)]
Do not assert seal transition on forward path

Unlike the replay path, where we expect no successor, the forward
path may very well encounter a successor being present.

Move the assertion to replay path only, as that is where it catches
issues. The forward path automatically propagates the seal via its
use of sendRequest(), which is forwarded as needed and therefore
we only note the race via LOG.debug() and not assert it.

Change-Id: I6f95acc49800d456c049d19982c801015fd69420
Signed-off-by: Robert Varga <>
2 years agoImprove aborted transaction logging 15/80415/2
Robert Varga [Wed, 20 Feb 2019 13:47:12 +0000 (14:47 +0100)]
Improve aborted transaction logging

The fact that a snapshot is recorded does not mean it has not been
closed -- in fact write-aspect transactions do not actively remove

When we prune recorded snapshots on transaction chain close, we need
to pay attention to the return value of abort(), as that is
indicating whether we should in fact warn about an unclosed

This means that the actual set of transactions we warn about cannot
be determined until we have traversed all recorded snapshots, hence
modify the logic to record the identifiers that were in fact aborted
and report all of them in a single message along with a stack trace
so the offender may be identifier.

Change-Id: I8b176f1990c9aa9c056cd03203ea50fb9b9549a0
Signed-off-by: Robert Varga <>
2 years agoCleanup java.util.Optional references 10/80410/2
Robert Varga [Wed, 20 Feb 2019 10:46:03 +0000 (11:46 +0100)]
Cleanup java.util.Optional references

As a byproduct of gradual migration, we have a few places where
we use FQCN to reference java.util.Optional eventhough it is among
imports. Fix that.

Change-Id: Ibede76ffc614c02dbc5e29e78e6c421abcd81ec9
Signed-off-by: Robert Varga <>
2 years agoMigrate to ListenerRegistry.create() 93/80393/2
Robert Varga [Tue, 19 Feb 2019 14:24:29 +0000 (15:24 +0100)]
Migrate to ListenerRegistry.create()

As per deprecation, the default constructor will be going away.

Change-Id: Id3216acc23fd56554b30b4225275f5bd2cbf4893
Signed-off-by: Robert Varga <>
2 years agoFix free-standing transaction lookup with module-based shards 92/80392/3
Robert Varga [Tue, 19 Feb 2019 12:50:04 +0000 (13:50 +0100)]
Fix free-standing transaction lookup with module-based shards

When we have a combination of tell-based protocol with module-based
shards, the frontend will use the cookie within LocalHistoryIdentifier,
making it potentially non-zero.

The backend tracks free-standing transactions under a local history,
which has cookie set to zero, hence it will not match when we attempt
to look it up for the purposes commit/abort/purge -- leading to
mismatched leader state for these transactions.

Change-Id: Ic4a493d0728b8de288a779d8c97a157584dc76bb
Signed-off-by: Robert Varga <>
2 years agoRemove use of CheckedFuture 71/80371/1
Robert Varga [Mon, 18 Feb 2019 17:59:03 +0000 (18:59 +0100)]
Remove use of CheckedFuture

YangTextSchemaSource uses ListenableFuture only, hence we can
remove use of deprecated Futures.immediate(Failed)CheckedFuture().

Change-Id: I40e571441894c49bbea7a8c11a3854505147af3d
Signed-off-by: Robert Varga <>
2 years agoReduce JSR305 proliferation 11/80211/6
Robert Varga [Thu, 7 Feb 2019 16:53:09 +0000 (17:53 +0100)]
Reduce JSR305 proliferation

retention=runtime annotations are mixing really badly with Java EE
injection and Java 11.

Make sure we do not use javax.annotation package in APIs and reduce
overall proliferation inside implementations.

Change-Id: I569815f0336efdc0de662c3b80f0fa6e5dd47d8a
Signed-off-by: Robert Varga <>
2 years agoAccount for reported UNMODIFIED nodes 67/80267/4
Robert Varga [Mon, 11 Feb 2019 16:00:14 +0000 (17:00 +0100)]
Account for reported UNMODIFIED nodes

DataObjectModification has no way of expressing UNMODIFIED nodes,
but they can (and are) reported from DataTreeCandidateNode, hence
we end up reporting IllegalStateException when the user tries to
look at those nodes.

Make sure we take the node under consideration and do not report
unmodified children, hence preventing the ISE from happening.

Change-Id: I34d36ae083c5ce3ad793eb584236f175f7a3a906
Signed-off-by: Robert Varga <>
2 years agoBump versions by x.y.(z+1) 84/80084/1
jenkins-releng [Fri, 1 Feb 2019 13:06:29 +0000 (13:06 +0000)]
Bump versions by x.y.(z+1)

Change-Id: I6a04c82981ff3244cf20416216d230d77f58f5dc
Signed-off-by: jenkins-releng <>
2 years agoMove BP xml files to standard OSGI-INF/blueprint 49/75549/7
Tom Pantelis [Tue, 22 Jan 2019 15:46:56 +0000 (10:46 -0500)]
Move BP xml files to standard OSGI-INF/blueprint

We originally used org/opendaylight/blueprint in case we needed
to customize how BP bundles are processed but this wasn't needed.
So let's move to the standard BP dir. We'll keep the
BlueprintBundleTracker as is for backwards compatibility - the only
change was to modify it to look in both dirs for BP xml files
when restarting the container for a bundle.

Change-Id: I777e831ef4d8293b49d7aa6c8cdecec96b97937f
Signed-off-by: Tom Pantelis <>
2 years agoImplement DOMDataTreeChangeListener.onInitialData 02/79802/3
Tom Pantelis [Tue, 22 Jan 2019 00:50:01 +0000 (19:50 -0500)]
Implement DOMDataTreeChangeListener.onInitialData

When there's no initial data on DTCL registration, we need to
invoke onInitialData().

Change-Id: Ib6e8a822b0a6cdfa54f523cacce9ceb699463585
Signed-off-by: Tom Pantelis <>
2 years agoBump to mdsal 3.0.6 06/79806/1
Stephen Kitt [Tue, 22 Jan 2019 08:50:11 +0000 (09:50 +0100)]
Bump to mdsal 3.0.6

Change-Id: I127f882c8dc93e5841da128575096c00794c4fd3
Signed-off-by: Stephen Kitt <>
2 years agoBump mdsal to 3.0.5 43/79743/4
Robert Varga [Sat, 19 Jan 2019 20:19:54 +0000 (21:19 +0100)]
Bump mdsal to 3.0.5

This aligns with odlparent-4.0.9 and yangtools-2.1.8.

Change-Id: If27257ddec66b02d6b7d3def493645ef1cb69c33
Signed-off-by: Robert Varga <>
2 years agoBump yangtools to 2.1.8 42/79742/2
Robert Varga [Sat, 19 Jan 2019 20:18:59 +0000 (21:18 +0100)]
Bump yangtools to 2.1.8

This align with odlparent-4.0.9.

Change-Id: I0bfa26a120ade132192c17da4f52968008d59950
Signed-off-by: Robert Varga <>
2 years agoBump to odlparent 4.0.9 64/79664/1
Stephen Kitt [Fri, 18 Jan 2019 15:32:20 +0000 (16:32 +0100)]
Bump to odlparent 4.0.9

Change-Id: I841d9cc7c6b0af646c2387492d7b799f3310d5f8
Signed-off-by: Stephen Kitt <>
2 years agoImprove error reporting for tell-based reads 10/78310/5
Tom Pantelis [Fri, 30 Nov 2018 03:00:34 +0000 (22:00 -0500)]
Improve error reporting for tell-based reads

Added contextual info similar to ask-based, including the
yang path of the requested read and the backend shard name.
Also wrapped RequestTimeoutException with

Change-Id: I5487e5531034cc1abbda27a4953897da7212eba8
Signed-off-by: Tom Pantelis <>
2 years agoBump mdsal to 3.0.4 09/79409/3
Robert Varga [Thu, 10 Jan 2019 10:32:17 +0000 (11:32 +0100)]
Bump mdsal to 3.0.4

This realigns odlparent/yangtools/mdsal and brings in the latest

Also upgrade dropwizard-metrics to odlparent-propagated version.

Change-Id: Ic106db810ef170f836fcbfad24032ab348a6bd8c
Signed-off-by: Robert Varga <>
2 years agoBump yangtools to 2.1.7 08/79408/1
Robert Varga [Thu, 10 Jan 2019 10:30:30 +0000 (11:30 +0100)]
Bump yangtools to 2.1.7

This realigns odlparent/yangtools and brings in a few minor

Change-Id: I0d79d12f37a8f213bccebd4107d57f8683e22d2c
Signed-off-by: Robert Varga <>
2 years agoBump to odlparent 4.0.8 58/79358/3
Stephen Kitt [Wed, 9 Jan 2019 14:54:52 +0000 (15:54 +0100)]
Bump to odlparent 4.0.8

Change-Id: I9a04273a846b08e5e5ccc5953d70221b4835b80c
Signed-off-by: Stephen Kitt <>
2 years agoOutput last log term on recovery complete 75/79375/2
Tom Pantelis [Wed, 9 Jan 2019 16:18:37 +0000 (11:18 -0500)]
Output last log term on recovery complete

Change-Id: Idea699d1620cf0f3d1f8173c3b08954d4d41a8d4
Signed-off-by: Tom Pantelis <>
2 years agoImprove follower term conflict resolution 34/79334/1
Robert Varga [Tue, 23 Jan 2018 16:08:03 +0000 (17:08 +0100)]
Improve follower term conflict resolution

Rather than performing a linear search downwards for a matching
log entry, take into account follower's last log index. This
eliminates the need for a lot of round-trips if the follower is
far behind the leader, but does not have a complete common ancestry.

Change-Id: I63c815f108d322de5d438a6eda43aaa7982d820a
Signed-off-by: Robert Varga <>
2 years agoAdd info logging and improve error reporting in clustering-it 20/79320/2
Tom Pantelis [Tue, 8 Jan 2019 18:44:50 +0000 (13:44 -0500)]
Add info logging and improve error reporting in clustering-it

Change-Id: I3579a6e296a8eedfd4bc032c159eee983e792ea7
Signed-off-by: Tom Pantelis <>
2 years agoinfo should be debug in CommitTransactionPayload 85/79285/2
Tom Pantelis [Mon, 7 Jan 2019 17:53:48 +0000 (12:53 -0500)]
info should be debug in CommitTransactionPayload

The log statement was recently added but should be debug.

Change-Id: I04fe1e22f9ca47826a634d9ea8748f271ac57353
Signed-off-by: Tom Pantelis <>
2 years agoOutput Request toString in RequestTimeoutException 84/79284/1
Tom Pantelis [Mon, 7 Jan 2019 16:43:51 +0000 (11:43 -0500)]
Output Request toString in RequestTimeoutException

It will help for debugging to see the transaction ID -
might as well output the entire Request which includes

Change-Id: I63eb20dc6b883c95d4b2059a231a5848d5a7b67c
Signed-off-by: Tom Pantelis <>
2 years agoSend leader's full address via AppendEntries 07/78507/5
Tom Pantelis [Thu, 6 Dec 2018 16:43:26 +0000 (11:43 -0500)]
Send leader's full address via AppendEntries

Added the leader's full actor address to AppendEntries. The
address can be rather long and we don't need to normally send
it so it's an Optional. The follower indicates via AppendEntriesReply
whether it needs the leader to send its address (new
needsLeaderAddress field). On receipt of the leader's address,
the follower sets it in its local RaftActorContext and notifies
the PeerAddressResolver. The ShardPeerAddressResolver impl
updates its local cache thus enabling transactions to resolve
the remote leader actor.

Since we're changing the serialized footprint of AppendEntries
and AppendEntriesReply, I preserved backwards compatibility by
versioning a new externalizable Proxy for each.

Change-Id: I1c0870a596b1782015eb973153b74dfcd48694e7
Signed-off-by: Tom Pantelis <>
2 years agoBump odlparent to 4.0.7 99/79099/7
Robert Varga [Tue, 1 Jan 2019 17:55:01 +0000 (18:55 +0100)]
Bump odlparent to 4.0.7

This fixes SFT/Java11 failures.

Change-Id: I9ee4895c7a63df4e0713ca5899a4f7ffc9f787b1
Signed-off-by: Robert Varga <>
2 years agoMigrate messagebus-impl to MDSAL APIs 06/79206/1
Robert Varga [Thu, 3 Jan 2019 20:31:11 +0000 (21:31 +0100)]
Migrate messagebus-impl to MDSAL APIs

This removes yet another user of long-deprecated APIs, migrating
it over to MD-SAL.

Change-Id: I51f2ce0e01ed3078dbd2fa4ad6eb91081d692fa3
Signed-off-by: Robert Varga <>
2 years agoFix mdsal-it-base with JDK9+ 04/79104/4
Robert Varga [Tue, 1 Jan 2019 22:09:51 +0000 (23:09 +0100)]
Fix mdsal-it-base with JDK9+

With JDK9+ karaf-4.2.2+ requires additional VMOptions, make sure
we pass them down so the container can boot up when run with JDK9+.

Change-Id: I77760b6e48b3f4335c382a6da1e9770a912b4467
Signed-off-by: Robert Varga <>
2 years agoSwitch to spotbugs 73/79073/3
Robert Varga [Thu, 27 Dec 2018 03:46:10 +0000 (04:46 +0100)]
Switch to spotbugs

Rather than using findbugs, use spotbugs, which works with Java 9.

Change-Id: I8807b2dc34fdb4ddfd476a2a0ed9cf0db2e044cd
Signed-off-by: Robert Varga <>
2 years agoFix CSIT failure in unsubscribeDtcl RPC 40/79140/2
Tom Pantelis [Wed, 2 Jan 2019 14:34:55 +0000 (09:34 -0500)]
Fix CSIT failure in unsubscribeDtcl RPC modified
the error message returned from unsubscribeDtcl if no DTCN's
are received but the CSIT test is expecting specific text content
to change it back.

Change-Id: I1791175ac1a6c5b6b9936a820646b147adf4c497
Signed-off-by: Tom Pantelis <>
2 years agoFix unit test with Java 9+ 01/79101/2
Robert Varga [Tue, 1 Jan 2019 21:32:01 +0000 (22:32 +0100)]
Fix unit test with Java 9+

has microsecond precision, which leads to a comparison failure.

Fix this by truncating the result of to milliseconds.

Change-Id: Ida8172e5b02cf352325e6f516e19592be4a83d67
Signed-off-by: Robert Varga <>
2 years agocommons-lang dependency should be scope=test 07/79007/5
Robert Varga [Sun, 23 Dec 2018 07:31:29 +0000 (08:31 +0100)]
commons-lang dependency should be scope=test

The classes are used in tests only, the dependency should be
scoped appropriately.

Change-Id: Ifb88ce765c63c0f57779dab986d6d2511060cfcd
Signed-off-by: Robert Varga <>
2 years agocommons-io dependency should be scope=test 08/79008/4
Robert Varga [Sun, 23 Dec 2018 07:42:00 +0000 (08:42 +0100)]
commons-io dependency should be scope=test

The classes are used in tests only, the dependency should be
scoped appropriately.

Change-Id: Icc0f2eca5296de62586f547363843b8e98cbac4e
Signed-off-by: Robert Varga <>

©2013 OpenDaylight, A Linux Foundation Collaborative Project. All Rights Reserved.
OpenDaylight is a registered trademark of The OpenDaylight Project, Inc.
Linux Foundation and OpenDaylight are registered trademarks of the Linux Foundation.
Linux is a registered trademark of Linus Torvalds.