14 months agoFixup chunk offset movement on resend 17/83317/7
Tomas Cere [Tue, 30 Jul 2019 09:48:06 +0000 (11:48 +0200)]
Fixup chunk offset movement on resend

Since we are moving the offset backwards on failures, it doesnt
matter whether we have seen a failure or success when incrementing
offset. We always need to move it to indicate a chunk is in flight,
otherwise canSendNextChunk() will allow us to send the same chunk
again and again even when it is already inflight.

Change-Id: If5dfd1913bda4328c5dfab0869f2df697bbd3cfc
Signed-off-by: Tomas Cere <>
14 months agoClean up opendaylight-inventory model 10/84210/2
Robert Varga [Thu, 5 Sep 2019 06:52:10 +0000 (08:52 +0200)]
Clean up opendaylight-inventory model

This is just a formatting change, cleaning up descriptions and

Change-Id: I2757280371851e2586221b01de143e8b987ed1ad
Signed-off-by: Robert Varga <>
14 months agoRevert "Bug 3871: Deprecate opendaylight-inventory model." 09/84209/2
Robert Varga [Thu, 5 Sep 2019 06:34:00 +0000 (08:34 +0200)]
Revert "Bug 3871: Deprecate opendaylight-inventory model."

This reverts commit 973bde38d5f02b4e65f20363ef03d17587433a21,
as the migration is not coming along and deprecated constructs
are causing massive number of warnings, hiding more important

Change-Id: I1e3d27374ffba0e584f194d468cebcfa9cecfe81
Signed-off-by: Robert Varga <>
14 months agoFix checkstyle/spotbugs violations 64/84164/1
Robert Varga [Tue, 3 Sep 2019 07:34:01 +0000 (09:34 +0200)]
Fix checkstyle/spotbugs violations

Enabling checkstyle/spotbugs by default finds these, fix them up.

Change-Id: Ib407830f0a15cb4c04492f3c694fb463cdb30bad
Signed-off-by: Robert Varga <>
14 months agoCleanup ValueTypes lookup 36/84036/1
Robert Varga [Wed, 28 Aug 2019 12:56:04 +0000 (14:56 +0200)]
Cleanup ValueTypes lookup

Change type of TYPES to ImmutableMap, so we are binding to a concrete
class when looking up. Also clean up initialization to follow fluent
builder pattern.

Finally inline requireNonNull() call into the invocation, so that its
return value is not wasted.

Change-Id: I4487bd43232131ca548441f4c8901a8c5dc60efa
Signed-off-by: Robert Varga <>
15 months agoDeduplicate MapNode key leaf values 98/83998/9
Robert Varga [Mon, 26 Aug 2019 20:44:30 +0000 (22:44 +0200)]
Deduplicate MapNode key leaf values

When we are encountering a leaf node inside a MapEntryNode, it can
be a leaf corresponding to a key -- in which case its value is
already present in NodeIdentifierWithPredicates and we do want to
de-duplicate those objects.

Change-Id: I2ed65c311f9921aa77c9f23bd1f7681d7f11355a
Signed-off-by: Robert Varga <>
15 months agoMove common code to startNode() 08/84008/4
Robert Varga [Tue, 27 Aug 2019 09:34:34 +0000 (11:34 +0200)]
Move common code to startNode()

We are duplicating null checks in all entrypoints which lead to
startNode(). We can centralize these in startNode(), leading to
denser code and fixing Sonar warnings.

Change-Id: I3f2a0c53216179e9d25624a3e3139340eff551fa
Signed-off-by: Robert Varga <>
15 months agoRename SodiumNormalizedNode* classes 01/84001/2
Robert Varga [Mon, 26 Aug 2019 22:08:05 +0000 (00:08 +0200)]
Rename SodiumNormalizedNode* classes

These are implementing Neon SR2 serialization format, hence they
should be named as such.

Change-Id: Iaa509e3490b420b8005a2724105bf99b53d95cf2
Signed-off-by: Robert Varga <>
15 months agoDrop public modifier from NodeTypes 99/83999/2
Robert Varga [Mon, 26 Aug 2019 21:17:51 +0000 (23:17 +0200)]
Drop public modifier from NodeTypes

The class itself is not visible outside of the package, there is no
point to declare the constants as public.

Change-Id: I4136a71c538452aba33a5abb3337e0f8fc7e8da8
Signed-off-by: Robert Varga <>
15 months agoRestore javassist dependency for tests 08/83908/1
Stephen Kitt [Thu, 22 Aug 2019 14:02:54 +0000 (16:02 +0200)]
Restore javassist dependency for tests

It’s still used in BindingBrokerTestFactory.

Change-Id: Id162dcc1c45b776644888a7bd4d0c7f603f1216f
Signed-off-by: Stephen Kitt <>
15 months agoReduce reliance on Guava Fuction/Supplier 40/83840/4
Robert Varga [Wed, 21 Aug 2019 08:55:57 +0000 (10:55 +0200)]
Reduce reliance on Guava Fuction/Supplier

We have JDK8+ equivalents, use those instead.

Change-Id: I3796c993cc7c13d3a278e1ffdffce03801be21e7
Signed-off-by: Robert Varga <>
15 months agoFix modernization issues 32/83832/8
Robert Varga [Tue, 20 Aug 2019 19:06:59 +0000 (21:06 +0200)]
Fix modernization issues

This fixes issues pointed out by Modernizer, which mostly boil
down to using Objects.requireNonNull() and direct java.util

Change-Id: Id32530a6722cd101f96c23f6a745f91b2f09e2f9
Signed-off-by: Robert Varga <>
15 months agoFinal bits of NodeIdentifier migration 47/83847/1
Robert Varga [Wed, 21 Aug 2019 10:33:36 +0000 (12:33 +0200)]
Final bits of NodeIdentifier migration

This picks up the last two callers of getKeyValues() and migrates
them to using asMap().

Change-Id: Ibb8ca4cd2ba0d97ff938d34b7e88afea213b0596
Signed-off-by: Robert Varga <>
15 months agoAnother round of checkstyle fixes 46/83846/1
Robert Varga [Wed, 21 Aug 2019 09:40:15 +0000 (11:40 +0200)]
Another round of checkstyle fixes

This fixes failures reported by upgraded checkstyle, as well as
some non-enforced violations.

Change-Id: I8b6a9667c6586bef528587d455f7a8597b4f483f
Signed-off-by: Robert Varga <>
15 months agoMigrate away from Throwables.propagate() 35/83835/2
Robert Varga [Tue, 20 Aug 2019 22:31:55 +0000 (00:31 +0200)]
Migrate away from Throwables.propagate()

This method has been deprecated, move off of it.

Change-Id: Ida34fb165dad1c0cf1786c773325e077de53a4c2
Signed-off-by: Robert Varga <>
15 months agoMove createProxy() 34/83834/2
Robert Varga [Tue, 20 Aug 2019 20:56:26 +0000 (22:56 +0200)]
Move createProxy()

This is a simple internal method, inline it.

Change-Id: I5568f95f31c2487441fb19dbf3d33e02de5ada2a
Signed-off-by: Robert Varga <>
15 months agoMove DataNormalizationOperation methods 30/83830/3
Robert Varga [Tue, 20 Aug 2019 19:20:55 +0000 (21:20 +0200)]
Move DataNormalizationOperation methods

Some these are not use in the base class, move them closer to their

Change-Id: I09a09946fb0b96ba0d2b291f74613b7f82566f41
Signed-off-by: Robert Varga <>
15 months agoMigrate NodeIdentifierWithPredicates.getKeyValues() 29/83829/3
Robert Varga [Tue, 20 Aug 2019 19:17:57 +0000 (21:17 +0200)]
Migrate NodeIdentifierWithPredicates.getKeyValues()

Change-Id: Iea47db72483111eea40374c23d7be0da33f67728
Signed-off-by: Robert Varga <>
15 months agoRefactor AbstractBrokerAwareActivator 31/83831/3
Robert Varga [Tue, 20 Aug 2019 20:19:57 +0000 (22:19 +0200)]
Refactor AbstractBrokerAwareActivator

This simplifies the implementation by encapsulating state.

Change-Id: I673e71f431026302974057af5933dba14e4c19c3
Signed-off-by: Robert Varga <>
15 months agoFixup checkstyle 28/83828/2
Robert Varga [Tue, 20 Aug 2019 19:04:19 +0000 (21:04 +0200)]
Fixup checkstyle

Updated checkstyle does not like some our docs, fix them up.

Change-Id: I2d4f608fe9c44a56cdcac511cc1e2ae75c03b413
Signed-off-by: Robert Varga <>
15 months agoPropagate action logical datastore type 54/83754/2
Robert Varga [Mon, 19 Aug 2019 12:01:27 +0000 (14:01 +0200)]
Propagate action logical datastore type

Unlike RFC7950 and related specifications, actions are generalized
in ODL to work on any datastore type, as otherwise internal cooperation
could not work on onything but operational datastore.

This patch correctly propagates the invocation datastore type, so that
invoking actions on top of config datastore becomes possible.

Change-Id: I20f340bee0321a446a07d701650d6dac57057ee6
Signed-off-by: Robert Varga <>
15 months agoChange in the version name 86/83786/1
Robert Varga [Tue, 20 Aug 2019 10:39:34 +0000 (12:39 +0200)]
Change in the version name

This catches up documentation to Magnesium.

Change-Id: I579b891baeba329d3ded6816f59e1b2da18379e1
Signed-off-by: Anushka Bhandari <>
Signed-off-by: Robert Varga <>
15 months agoRemove triemap-bom import 82/83782/2
Robert Varga [Tue, 20 Aug 2019 06:33:28 +0000 (08:33 +0200)]
Remove triemap-bom import

Triemap is provided by odlparent, remove the reference to it.

Change-Id: Ib37bc9957742a7e9f4e094248b6b8129a32ccd73
Signed-off-by: Robert Varga <>
15 months agoBump mdsal to 4.0.4 23/83723/2
Robert Varga [Thu, 15 Aug 2019 08:26:16 +0000 (10:26 +0200)]
Bump mdsal to 4.0.4

This brings in latest fixes.

Change-Id: I59f89ef4fe76ef16fcdc9b59f8d9b50aa0849847
Signed-off-by: Robert Varga <>
15 months agoRemove BindingToNormalizedNodeCodec service export 86/83686/1
Robert Varga [Fri, 16 Aug 2019 10:23:43 +0000 (12:23 +0200)]
Remove BindingToNormalizedNodeCodec service export

This service is not used anywhere (not should it be), remove it.

Change-Id: I841065ff4932c4d56ca20bd3c465da4fe814216c
Signed-off-by: Robert Varga <>
15 months agoMake sure registrations are closed 30/83530/4
EmmettCox [Thu, 8 Aug 2019 11:01:42 +0000 (12:01 +0100)]
Make sure registrations are closed

This ensures registrations are closed as soon as they are not

Change-Id: I3a391f202963852f47486b78748c8e2d7e97162a
Signed-off-by: EmmettCox <>
Signed-off-by: Robert Varga <>
15 months agoBump versions by x.y.(z+1) 44/83544/1
jenkins-releng [Mon, 12 Aug 2019 16:14:12 +0000 (16:14 +0000)]
Bump versions by x.y.(z+1)

Signed-off-by: jenkins-releng <>
Change-Id: I7108fdde06e4c0d16aa2577c8a050d3567bdfbfe

15 months agoBump mdsal to 4.0.3 87/83087/9
Robert Varga [Thu, 18 Jul 2019 06:41:02 +0000 (08:41 +0200)]
Bump mdsal to 4.0.3

This bumps mdsal to 4.0.3, bringing in latest fixes and improvements.

Change-Id: Ie8e03aab1f31edf80bc9f1d7896c334c9d83be8d
Signed-off-by: Robert Varga <>
15 months agoBump yangtools to 3.0.4 86/83086/7
Robert Varga [Thu, 18 Jul 2019 06:40:17 +0000 (08:40 +0200)]
Bump yangtools to 3.0.4

This picks up latest fixes and improvements.

Change-Id: I03201927d23762acf641c1ab0ff3419ba8b94969
Signed-off-by: Robert Varga <>
15 months agoBump odlparent to 5.0.1 85/83085/5
Robert Varga [Thu, 18 Jul 2019 06:39:32 +0000 (08:39 +0200)]
Bump odlparent to 5.0.1

This picks up the latest upgrades.

Change-Id: If6006a94197839a9718cd5eae16b0a025913af93
Signed-off-by: Robert Varga <>
15 months agoAdd unit test for FrontedMetadata memory leaks 71/81971/30
Tomas Cere [Thu, 9 May 2019 09:12:34 +0000 (11:12 +0200)]
Add unit test for FrontedMetadata memory leaks

This memory leak occurs when write and read-only transactions
intertwine, leading to sparse range set.
We need to be able to request frontend metadata from shards,
so we can see whether it converges to a single range.

Also rework testing datastores, so we can inject custom testing
implementations allowing us to retrieve frontend metadata.

Change-Id: Ia8f350be7831b0c8f6846ee6fa41a665d1e191f4
Signed-off-by: Tomas Cere <>
Signed-off-by: Robert Varga <>
16 months agoDo not log the whole DataTree at debug level 32/83232/2
Tomas Cere [Wed, 24 Jul 2019 10:40:29 +0000 (12:40 +0200)]
Do not log the whole DataTree at debug level

This needs to be at trace, as its polluting logs otherwise.

Change-Id: I6141a1793402e87ab49defe40007114b40d16335
Signed-off-by: Tomas Cere <>
Signed-off-by: Robert Varga <>
16 months agoForce AbstractRaftRPC to use Externalizable proxy pattern 65/83165/2
Robert Varga [Fri, 19 Jul 2019 07:15:15 +0000 (09:15 +0200)]
Force AbstractRaftRPC to use Externalizable proxy pattern

All implementations are already using the pattern, this change
makes that official and removes the no-arg constructor, which
is not used anyway.

Change-Id: Iaea4a2f39972537907cce19551d2ca6e4b33fad6
Signed-off-by: Robert Varga <>
16 months agoUse merging transaction chains in dsbenchmark 58/83158/2
Robert Varga [Thu, 18 Jul 2019 17:21:29 +0000 (19:21 +0200)]
Use merging transaction chains in dsbenchmark

This functionality was broken, as the injection conversion done
for 4.0.0 did not take into account the fact we are dropping

Change-Id: I41244d6732216554e197ffe9b7ef6d5be41ab520
Signed-off-by: Robert Varga <>
16 months agoMake RaftVersions a final class 64/83164/1
Robert Varga [Fri, 19 Jul 2019 07:22:53 +0000 (09:22 +0200)]
Make RaftVersions a final class

RaftVersions is a simple holder of constants, not an interfaces
which would be implemented. Change it to a utility class.

Change-Id: Ie7406cf579de2e4f17e03063f492eeb73d5ecb6c
Signed-off-by: Robert Varga <>
16 months agoRaise EOS unsuccessful request reporting to error 90/82990/3
Robert Varga [Wed, 10 Jul 2019 09:54:38 +0000 (11:54 +0200)]
Raise EOS unsuccessful request reporting to error

When we fail to send a message to the backend we are entering an
inconsistent state, where our users think the candidate is registered,
but the backend knows nothing about it -- which warrants an ERROR
level output.

Also place a FIXME on operations which should be retried.

Change-Id: I0799afe2e0786468cdef5f80c1064cb08d3c1af0
Signed-off-by: Robert Varga <>
16 months agoEnforce RegisterListenerLocal arguments 89/82989/3
Robert Varga [Wed, 10 Jul 2019 09:59:42 +0000 (11:59 +0200)]
Enforce RegisterListenerLocal arguments

The fields need to be initialized to non-null, make sure the object
does that (instead of its users).

Change-Id: I545421167bd33e06ea7221a0fbcc136083bfc80d
Signed-off-by: Robert Varga <>
16 months agoChange remaining SnapshotSelectionCriteria to prioritize timestamp 17/83017/2
Tomas Cere [Thu, 11 Jul 2019 10:19:47 +0000 (12:19 +0200)]
Change remaining SnapshotSelectionCriteria to prioritize timestamp

These are also needed and were missed in the original run-through.

Change-Id: Icede0673c580f793ffe81ccecf728c15dc6e83ca
Signed-off-by: Tomas Cere <>
16 months agoFurther Guava Optional cleanups 09/83009/6
Robert Varga [Wed, 10 Jul 2019 14:37:12 +0000 (16:37 +0200)]
Further Guava Optional cleanups

This migrates most of sal-akka-raft to use java.util.Optional.

Change-Id: Iaccc760101762dc9d4d647ded80de9a76f1f067b
Signed-off-by: Robert Varga <>
16 months agoMigrate most of CDS to use java.util.Optional 07/83007/6
Robert Varga [Wed, 10 Jul 2019 14:25:37 +0000 (16:25 +0200)]
Migrate most of CDS to use java.util.Optional

This picks up some stragglers, updating them to not use Guava's

Change-Id: Id6e4c27e5bd55f19058e87f04473b859922bd441
Signed-off-by: Robert Varga <>
16 months agoUse OptionalLong to track enqueue time 06/83006/5
Robert Varga [Wed, 10 Jul 2019 14:07:59 +0000 (16:07 +0200)]
Use OptionalLong to track enqueue time

This prevents the need to box/unbox primitive longs and makes things
generally more convenient.

Change-Id: I6ca394526b09fdd469a9440c9313371e6185c135
Signed-off-by: Robert Varga <>
16 months agoUse OptionalLong in AbstractClientConnection 05/83005/5
Robert Varga [Wed, 10 Jul 2019 14:02:30 +0000 (16:02 +0200)]
Use OptionalLong in AbstractClientConnection

This removes the need for boxing longs.

Change-Id: I73a6c0be7f9a662f9e1df884f46adbc0fc121fb1
Signed-off-by: Robert Varga <>
16 months agoMigrate ShardDataTree to use OptionalLong 04/83004/5
Robert Varga [Wed, 10 Jul 2019 13:54:51 +0000 (15:54 +0200)]
Migrate ShardDataTree to use OptionalLong

This prevents boxing and is more modern than Guava's Optional<Long>.

Change-Id: Iaef3b7f18527f532ed2d775641cd855b14087fc1
Signed-off-by: Robert Varga <>
16 months agoMigrate InstallSnapshot/SnapshotTracker use of Optional 03/83003/5
Robert Varga [Wed, 10 Jul 2019 13:38:51 +0000 (15:38 +0200)]
Migrate InstallSnapshot/SnapshotTracker use of Optional

This patch migrates both classes to use Java 8 Optional and

Change-Id: Ib8e14f963b4e0db821fd8d3ab37f65b5bf5d5b54
Signed-off-by: Robert Varga <>
16 months agoConvert MessageTracker to use java.util.Optional 02/83002/3
Robert Varga [Wed, 10 Jul 2019 13:30:22 +0000 (15:30 +0200)]
Convert MessageTracker to use java.util.Optional

This migrates from Guava, using Java 8 equivalent.

Change-Id: I71d01dae05e4f426406989d240c12745149d6964
Signed-off-by: Robert Varga <>
16 months agoMigrate ActorUtils to java.util.Optional 01/83001/3
Robert Varga [Wed, 10 Jul 2019 13:20:33 +0000 (15:20 +0200)]
Migrate ActorUtils to java.util.Optional

This migrates ActorUtils to Java 8 Optional, updating its users.

Change-Id: I0ec900d16bf44b2ab8ae48d8a72f2b63fa633312
Signed-off-by: Robert Varga <>
16 months agoUse OptionalInt in FlexibleThreadPoolWrapper 00/83000/1
Robert Varga [Wed, 10 Jul 2019 13:11:09 +0000 (15:11 +0200)]
Use OptionalInt in FlexibleThreadPoolWrapper

This gets rid of Guava Optional and fixes an unboxing warning
through use of Java 8.

Change-Id: I9c22bbedfcdd08f18c5ff14283dd8f130d6020c6
Signed-off-by: Robert Varga <>
16 months agoUpgrade jmh-core 62/82962/4
Robert Varga [Mon, 8 Jul 2019 13:53:29 +0000 (15:53 +0200)]
Upgrade jmh-core

This upgrade jmh-core to the latest release.

Change-Id: I6317ad6689fb5ac6d5156c9e6ddbfb1a6a051321
Signed-off-by: Robert Varga <>
16 months agoMigrate to MD-SAL APIs 63/82963/3
Robert Varga [Mon, 8 Jul 2019 14:02:51 +0000 (16:02 +0200)]
Migrate to MD-SAL APIs

This removes controller dependencies.

Change-Id: Ic3fa0748d4e92f86899af2b8dcf73b7f1ae2560a
Signed-off-by: Robert Varga <>
16 months agoDo not depend on odl-mdsal-broker-local 59/82959/2
Robert Varga [Mon, 8 Jul 2019 13:38:03 +0000 (15:38 +0200)]
Do not depend on odl-mdsal-broker-local

odl-mdsal-trace depends on both odl-mdsal-broker (clustered) and
odl-mdsal-broker-local (non-clustered), which is not appropriate.
Remove the latter.

Change-Id: Ifdeb1c242004b6e365c133360c96af00c17c67c7
Signed-off-by: Robert Varga <>
16 months agoRemove odl-mdsal-all 58/82958/2
Robert Varga [Mon, 8 Jul 2019 13:35:29 +0000 (15:35 +0200)]
Remove odl-mdsal-all

This set of features was used for testing and does not really
contain 'all' features. Remove it in favor of explicitly listing

Change-Id: I4db4932594c17621ab2fb17f1436531ce3b7dbe5
Signed-off-by: Robert Varga <>
16 months agoTeach sal-remoterpc-connector to route actions 83/82283/13
EmmettCox [Thu, 4 Jul 2019 16:31:38 +0000 (16:31 +0000)]
Teach sal-remoterpc-connector to route actions

sal-remoterpc-connector already handles routing of RPC registrations
and invocations across a cluster. Actions are very similar to RPCs,
hence it is natural to keep both in the same component.

This patch refactors common bits that go into tracking both, so that
we share common actors and concepts.

Change-Id: I0b9005bc3560b4dd5977a280d83eceebe132bec9
Signed-off-by: EmmettCox <>
16 months agoAdd support for reusable streaming 17/82417/8
Robert Varga [Thu, 6 Jun 2019 10:26:37 +0000 (12:26 +0200)]
Add support for reusable streaming

With the actual implementations working on top of
NormalizedNodeStreamWriter, we gained the ability to flexibly
receive stream events.

This patch takes advantage of that flexibility by allowing
a ReusableImmutableNormalizedNodeStreamWriter to be the receiver
of the events -- thus allowing parts of the state involved in
building a NormalizedNode tree to be reused -- lowering GC

A number of call sites, which can safely reuse such state are
converted to use the newly-introduced facility.

Change-Id: Iaf1b3ab2b2996e7004c036fc93a80a8ca8792314
Signed-off-by: Robert Varga <>
16 months agoAdd streaming interface to NormalizedNodeDataInput 09/82009/24
Robert Varga [Mon, 13 May 2019 05:44:05 +0000 (07:44 +0200)]
Add streaming interface to NormalizedNodeDataInput

NormalizedNodeDataInput allows reading NormalizedNodes, which is
wired directly to immutable node builders, providing little

This inflexibility requires us to re-stream the data set through
NormalizedNodePruner, effectively doubling memory requirements
during recovery.

Extend NormalizedNodeDataInput to have a NormalizedNodeStreamWriter
to be plugged in, so that the byte stream becomes a source of
NormalizedNodeStreamWriter events, which can be routed through
a custom-built pipeline.

readNormalizedNode() becomes a simple default wrapper, which
pipes the stream into an ImmutableNormalizedNodeStreamWriter.

Change-Id: Ic732ba9105dd7e27d5612853b931aba66bdd83a5
Signed-off-by: Robert Varga <>
16 months agoFix multiple snapshots present after journal removal 80/82380/8
Tomas Cere [Mon, 3 Jun 2019 13:02:08 +0000 (15:02 +0200)]
Fix multiple snapshots present after journal removal

After journal removal the sequence number starts counting from 0
once again. Therefore we can have multiple snapshots present and
snapshots with higher sequence number always take priority for loading
even when a newer snapshot is present.

Change this up in 2 ways:
1. disregard sequence number while deleting snapshots(delete all
snapshots with an older timestamp)
2. while loading snapshots prioritize timestamp rather than seqNr.

Also simplify

Change-Id: I205ea0ddf48d73b0a09297a1ce4e9fd514531993
Signed-off-by: Tomas Cere <>
Signed-off-by: Robert Varga <>
16 months agoAdd transaction debugs 58/82858/2
Robert Varga [Mon, 1 Jul 2019 06:50:24 +0000 (08:50 +0200)]
Add transaction debugs

This adds a few debug messages allowing logs to be introspected
to see the timing of readying/applying transactions.

Change-Id: I1dde184023a034c6a0fa30e9a3898ab65db2decc
Signed-off-by: Robert Varga <>
16 months agoRename SODIUM versions to NEON_SR2 67/82867/2
Robert Varga [Fri, 28 Jun 2019 11:37:50 +0000 (13:37 +0200)]
Rename SODIUM versions to NEON_SR2

We are shipping these improvements in Neon SR2, which means we
should reserve this version not as Sodium.

Change-Id: If5caa0dd7510c6e1ffe3c483c74c456798ca6bb3
Signed-off-by: Robert Varga <>
16 months agoDeprecate all MD-SAL APIs 63/82863/1
Robert Varga [Mon, 1 Jul 2019 11:00:52 +0000 (13:00 +0200)]
Deprecate all MD-SAL APIs

These APIs have either been completely deprecated or have their
replacements in the mdsal project. Mark all of them as deprecated,
including their implementations.

Change-Id: I903afcef57c70c1013ab479cfcf31a42f475dc0f
Signed-off-by: Robert Varga <>
16 months agoMake private methods static 62/82862/1
Robert Varga [Mon, 1 Jul 2019 12:12:48 +0000 (14:12 +0200)]
Make private methods static

Eclipse is flagging a number of methods as potentially-static,
make sure we mark them as such.

Change-Id: I4254dd29716f4365d71f81c3f3a0e7a9590d9801
Signed-off-by: Robert Varga <>
16 months agoRemove ShardManager.configListenerReg 61/82861/1
Robert Varga [Mon, 1 Jul 2019 12:06:28 +0000 (14:06 +0200)]
Remove ShardManager.configListenerReg

This field is not used anywhere, remove it.

Change-Id: Id66143aefea072ce6d529d39e86a961aa1d551ba
Signed-off-by: Robert Varga <>
16 months agoMigrate use of AbstractDOMStoreTransaction 60/82860/1
Robert Varga [Mon, 1 Jul 2019 12:09:55 +0000 (14:09 +0200)]
Migrate use of AbstractDOMStoreTransaction

This utility class has been migrated to mdsal, use the instance
from there.

Change-Id: I9aad875f631903d4ce9dfb2779dfce6f8883a50c
Signed-off-by: Robert Varga <>
17 months agoReset replyReceivedForOffset correctly 26/82726/1
Tomas Cere [Tue, 25 Jun 2019 12:16:50 +0000 (14:16 +0200)]
Reset replyReceivedForOffset correctly

replyReceivedForOffset needs to be set to the initial -1
otherwise we can have multiple sendSnapshotChunk()'s called
before receiving any replies. Which would fail due to the already
running stopwatch.

Change-Id: Ie708f0c2454a87f77cba97e4fed8a9ffc866b624
Signed-off-by: Tomas Cere <>
17 months agoFailed chunks should move offset backwards 25/82725/1
Tomas Cere [Tue, 25 Jun 2019 08:45:15 +0000 (10:45 +0200)]
Failed chunks should move offset backwards

If a chunk has failed it needs to move the offset backwards,
otherwise this would misalign the stream, or make it possible
to call sendNextChunk() right after resend which could be
seen as the chunkTimer attempted to be started while already running.

Change-Id: I6d44195ae9bcb23075ffea9c4f1d1ab1d0540c93
Signed-off-by: Tomas Cere <>
17 months agoFix IMDS test synchronization 53/82553/1
Robert Varga [Tue, 18 Jun 2019 15:39:37 +0000 (17:39 +0200)]
Fix IMDS test synchronization

Before we close the DTCL we must make sure we flush all changes,
otherwise we could end up losing changes simply because we are no
longer interested in them.

Change-Id: I6df384c375ea1692a10c9f460d3dceb626468061
Signed-off-by: Robert Varga <>
17 months agoReset snapshot progress on IOExceptions 94/82394/6
Tomas Cere [Tue, 4 Jun 2019 09:17:10 +0000 (11:17 +0200)]
Reset snapshot progress on IOExceptions

Also log the whole LeaderInstallSnapshotState of failures.

Change-Id: I33b1c38f2277f08acbe77bcb0c16d2dfe41d8851
Signed-off-by: Tomas Cere <>
17 months agoSend commitIndex updates to followers as soon as possible 98/82498/1
Robert Varga [Tue, 11 Jun 2019 13:42:34 +0000 (15:42 +0200)]
Send commitIndex updates to followers as soon as possible

When the commitIndex gets updated when we reach consensus, we should
propagate this fact to the sender peer. Otherwise data tree change
listeners on peers may be unnecessarily delayed, as they end up
waiting for heartbeat interval (or non-batched entries) to be propagated

The results in observably-better behaviour between leader and follower,
as the leader is sending out notifications of consensus, without
delayling for heartbeat -- which is reflected in the changes to

Furthermore, DelayedMessagesElectionScenarioTest is detecting the change,
as two members may achieve asynchronous consensus -- hence the leader
would be sending two messages (request to persist, commitIndex) to push
the consensus out. This exposes a race in the test, where we do not know
the order between TimeoutNow and AppendEntries. If AppendEntries is
processed by member 2 after TimeoutNow, leader will know about the new
term due to AppendEntriesReply we send out. The cluster will still
converge, but the assert for leader will be different if it manages to
process the message. Fix this by isolating member 2.

Change-Id: I695ef25c7a4cf8799c9c5e04c2c33fbf3e2f21df
Signed-off-by: Robert Varga <>
(cherry picked from commit c2b0e92d2c3ba05abf0fefb24413c68fd1c56bea)

17 months agoSimplify recoveryTime computation 58/82458/3
Robert Varga [Fri, 7 Jun 2019 22:44:58 +0000 (00:44 +0200)]
Simplify recoveryTime computation

Stopwatch.stop() returns self, take advantage of that to shorted
the code to compute the formatting string. Also drop toString()
call, as it is implied by string concat.

Change-Id: I4fa645e42fc38b5f9f0775d986fd17aa693725b1
Signed-off-by: Robert Varga <>
17 months agoSwitch default output stream version 13/82313/27
Robert Varga [Wed, 29 May 2019 16:17:13 +0000 (18:17 +0200)]
Switch default output stream version

All versioned callers have been converted to use appropriate
stream versions, this patch bumps the default unversioned stream
to Sodium version.

Since we're changing the serialization format for ExecuteRpc,
make sure we fully embrace writer/reader, so that we get better

Change-Id: I17deeda1a49394600a5b0f93bd46d3b2c812123c
Signed-off-by: Robert Varga <>
17 months agoBump persisted PayloadVersion 14/82314/25
Robert Varga [Wed, 29 May 2019 16:23:04 +0000 (18:23 +0200)]
Bump persisted PayloadVersion

Since the NormalizedNodeStream format has changed, shard persisted
state is affected.

This patch bumps PayloadVersion and applies that bump to snapshots
and CommitTransactionPayload.

On recovery, a snapshot's need to migrate is reflected in its state,
and it is examined just as MigratedSerializables are.
CommitTransactionPayload is not examined, as understanding its
stream version would require deserializing at least its header.

Change-Id: I678527be4487ee1729123ba8b9dcd2269e6cf262
Signed-off-by: Robert Varga <>
17 months agoBump datastore version 11/82311/21
Robert Varga [Wed, 29 May 2019 14:59:01 +0000 (16:59 +0200)]
Bump datastore version

This bumps the ask-based protocol datastore version, so that
SODIUM_VERSION is using Sodium NormalizedNode{Input,Output}.

Change-Id: Ib4ad1c27566179db958ad7776f81417ab99a78cf
Signed-off-by: Robert Varga <>
17 months agoAdd QNameModule coding 26/82426/5
Robert Varga [Thu, 6 Jun 2019 17:36:06 +0000 (19:36 +0200)]
Add QNameModule coding

Coding on QNames is effective when there are a lot of same QNames,
but there are cases where QNames themeselves are not reused --
for example YangInstanceIdentifiers and small read subtrees.

These typically end up using different QNames from a small set
of modules, hence coding modules actually helps them quite a bit.

This patch adds a namespace/revision cache, so that encoding such
QNames is also efficient.

Change-Id: I6cd0885ef605e4a1bf7b768ffe215e9eb3e2fbc5
Signed-off-by: Robert Varga <>
17 months agoAdd buffering to LocalSnapshotStore 39/82439/3
Tomas Cere [Tue, 4 Jun 2019 13:32:20 +0000 (15:32 +0200)]
Add buffering to LocalSnapshotStore

These improve snapshot save/load load times by around 15-20%
on my environment.

Change-Id: I67236f7e97f519993031d462c947588004857c33
Signed-off-by: Tomas Cere <>
17 months agoRemove use of thread-local input 91/82391/9
Robert Varga [Mon, 3 Jun 2019 19:37:43 +0000 (21:37 +0200)]
Remove use of thread-local input

All callers are only called from top-level constructs, thus
not having a relationship with the context in which they are

Change-Id: I856f9109480fb83475a20ce4430d02a1dd055522
Signed-off-by: Robert Varga <>
17 months agoTake advantage of AugmentationIdentifier.create() 99/82399/2
Robert Varga [Tue, 4 Jun 2019 17:51:43 +0000 (19:51 +0200)]
Take advantage of AugmentationIdentifier.create()

When we are reading an augmentation identifier for the first
time we can optimize the number of identifiers by consulting
the global weak cache.

Change-Id: I5cd2d87e0a7c3f01357beb1b097d4a16a6eda3e0
Signed-off-by: Robert Varga <>
17 months agoDo not use NodeIdentifierWithPredicates constructors 97/82397/3
Robert Varga [Tue, 4 Jun 2019 17:05:55 +0000 (19:05 +0200)]
Do not use NodeIdentifierWithPredicates constructors

This class will transition into a value-based object for memory
efficiency reasons. Switch to using its static factory methods.

Change-Id: I7e21a6ccf1ad999d2d0427769b26dffeac166274
Signed-off-by: Robert Varga <>
17 months agoOptimize readQNameSet() 98/82398/2
Robert Varga [Tue, 4 Jun 2019 17:54:22 +0000 (19:54 +0200)]
Optimize readQNameSet()

We are providing a bad hint to HashSet, as we are not taking
into consideration its load factor. Use utility methods from Guava
to get a properly-sized set.

Change-Id: I568be3040b924a8ed77aa0224ab45673104c4664
Signed-off-by: Robert Varga <>
17 months agoRemove sal-binding-broker dependency on javassist 95/82395/2
Robert Varga [Tue, 4 Jun 2019 12:05:27 +0000 (14:05 +0200)]
Remove sal-binding-broker dependency on javassist

We are not using Javassist anymore, remove the dependency on it.

Change-Id: I46e8f30e002d13d324779f461ac0365b67a6f70a
Signed-off-by: Robert Varga <>
17 months agoRemove use of thread-local output 90/82390/6
Robert Varga [Mon, 3 Jun 2019 19:12:44 +0000 (21:12 +0200)]
Remove use of thread-local output

All callers can be safely migrated to using a non-shared writer,
as they all are expected to be apex implementations.

Change-Id: I87cfa1d3ec415b79c40c29a91ccbd8da2869b27a
Signed-off-by: Robert Varga <>
17 months agoBump mdsal to 4.0.2 66/82266/6
Robert Varga [Tue, 28 May 2019 07:18:32 +0000 (09:18 +0200)]
Bump mdsal to 4.0.2

This picks up the latest fixes from upstream.

Change-Id: I34ef8d8f5d61c6ca9f795369ab99b06f4488d5f3
Signed-off-by: Robert Varga <>
17 months agoBump yangtools to 3.0.1 65/82265/5
Robert Varga [Tue, 28 May 2019 07:18:08 +0000 (09:18 +0200)]
Bump yangtools to 3.0.1

This bumps yangtools to 3.0.1, picking up latest fixes and

Change-Id: Id7ca8d3e23865070896b9da31e7d09a509b21578
Signed-off-by: Robert Varga <>
17 months agoSwitch CompositeModification to bypass thread-local streams 88/82388/5
Robert Varga [Mon, 3 Jun 2019 17:43:22 +0000 (19:43 +0200)]
Switch CompositeModification to bypass thread-local streams

Thread-local streams have little control over the version being
used, as the stream is allocated all over the place.

This model cannot function without explicit setup, which is used
by CompositeModification. This patch migrates CompositeModification
to use a new NormalizedNodeDataInput-based interface, so
thread-locals become obsolete.

Change-Id: I1111be5d6fc8c0129fb72232c3e713004068f152
Signed-off-by: Robert Varga <>
17 months agoMove byte-based serialization method 89/82389/2
Robert Varga [Mon, 3 Jun 2019 18:52:36 +0000 (20:52 +0200)]
Move byte-based serialization method

This is only used in testing, move the method out to testing class.

Change-Id: Ib18be55467561a2471a74285b062b40d4e9e08c8
Signed-off-by: Robert Varga <>
17 months agoMake Modification/Externalizable interface visible 87/82387/1
Robert Varga [Mon, 3 Jun 2019 16:56:44 +0000 (18:56 +0200)]
Make Modification/Externalizable interface visible

Finding direct callers of these methods is not easy, override
them to get exact call sites. This will help us with versioning
of the output stream.

Change-Id: Idc9bb036fac786b4480c098bd592e272d5a4c0ad
Signed-off-by: Robert Varga <>
17 months agoRemove ask-based close transaction chain persistence 37/79037/9
Robert Varga [Thu, 23 May 2019 16:57:00 +0000 (18:57 +0200)]
Remove ask-based close transaction chain persistence

ask-based protocol has tracking disabled when encountered, hence
it does not make sense to persist anything.

Change-Id: I1ec87a9905aa76f3e2a57ae2440c8c693ff60848
Signed-off-by: Robert Varga <>
17 months agoRemove sal-connector-api 79/82379/2
Robert Varga [Mon, 3 Jun 2019 09:18:49 +0000 (11:18 +0200)]
Remove sal-connector-api

This artifact is not used anywhere, remove it.

Change-Id: I43a9df5d77abf28d652ea9289025b49638dad702
Signed-off-by: Robert Varga <>
17 months agoBump cds-access-api ABIVersion 12/82312/3
Robert Varga [Wed, 29 May 2019 15:33:08 +0000 (17:33 +0200)]
Bump cds-access-api ABIVersion

In order to take advantage of improvements to NormalizedNode stream,
we need to declare a new ABI version, which will actually use
the new stream version.

Change-Id: I458a24586888e0b2fcc1c636e16d0b8e9e6e681f
Signed-off-by: Robert Varga <>
17 months agoOptimize NodeIdentifier reading 79/82279/13
Robert Varga [Wed, 29 May 2019 10:10:52 +0000 (12:10 +0200)]
Optimize NodeIdentifier reading

NodeIdentifiers are just an alias for QName, which is heavily
reused. This patch adds a secondary cache for interpreting QNames
as NodeIdentifiers, with instance reuse.

This allows us to reduce the number of NodeIdentifier instances,
while not relying on NodeIdentifier.create() for the common lookup

Change-Id: Ifa0c5d572f7d39da49707529c7ddd9c9f36f2dab
Signed-off-by: Robert Varga <>
17 months agoAdd support for coded QNames/AugmentationIdentifiers 04/82304/7
Robert Varga [Wed, 29 May 2019 09:27:21 +0000 (11:27 +0200)]
Add support for coded QNames/AugmentationIdentifiers

Profiling is showing we are spending about 46% in readQName(),
which further breaks down to 30% in readCodedString() and 61%
in QNameFactory.create().

Out of 1.7M invocations of readQName(), we end up loading only
508 QNames -- which is understandable, as the number of QNames is
bound by the SchemaContext contents. This also applies to

This patch defines a new streaming format version, which uses
the same linear coding approach we take for Strings, taking
hash-based lookups out of the hot path -- bringing reconstruction
time down from 36 to 19-24 seconds.

It has also the effect of reducing the snapshot size, in our testing
bringing a 388.8MiB snapshot down to 210.6MiB.

Change-Id: Ibbc96c286d52b3261b04dd7de02e13bc6b44e803
Signed-off-by: Robert Varga <>
17 months agoSeparate lazy-versioned NormalizedNodeDataInput 78/82278/6
Robert Varga [Tue, 28 May 2019 13:28:55 +0000 (15:28 +0200)]
Separate lazy-versioned NormalizedNodeDataInput

NormalizedNodeInputStreamReader is really only one implementation
of the streaming format, which happens to be bound to LITHIUM_VERSION.

This effectively means NormalizedNodeInputOutput can give out either
a version-bound reader or an unbound reader -- which we facilitate by
creating VersionedNormalizedNodeDataInput which is a simple forwarder
with lazily-initialized delegate.

Since this increases the number of (NormalizedNode)DataInput
implementations, we share as much code as possible through the use
of forwarding implementations.

Change-Id: I28713346730ed5bf7f912bf8afa5d8722b202035
Signed-off-by: Robert Varga <>
17 months agoDo not format QNames to string on input 76/82276/4
Robert Varga [Tue, 28 May 2019 11:14:34 +0000 (13:14 +0200)]
Do not format QNames to string on input

Profiling has revelead the creation of String for QNameFactory
lookup is taking around 19% of overall cost of

Rather than using concatenation, use a dedicated Key object, which
holds the separated-out localname/namespace/revision strings,
thus allocating a small object instead of a full-blown String.

This improves snapshot deserialization perfomance on a 350MiB
snapshot by about 16%, also resulting in lower memory pressure.

Change-Id: If3e9e4b332b0a5acbe33aa10fca189bbc7da8c81
Signed-off-by: Robert Varga <>
17 months agoUpdate QNameFactory caching 75/82275/3
Robert Varga [Tue, 28 May 2019 10:42:02 +0000 (12:42 +0200)]
Update QNameFactory caching

This eliminates the use of soft references in favor of weak ones,
so that we do not rely on GC heuristics for retention.

We also make the maximum cache size runtime-configurable via

Finally we use QName.intern() to defer to the global QName cache,
resulting in better QName instance sharing.

Change-Id: Ice3596d23b96c4ecfc5775cb21968233b042a984
Signed-off-by: Robert Varga <>
18 months agoFix write transaction shard stats 89/82189/3
Tomas Cere [Tue, 21 May 2019 11:42:47 +0000 (13:42 +0200)]
Fix write transaction shard stats

With the changes in ask protocol due to to tell based
integration, shards stats had incorrectly reported
write transaction counts. Since backend doesnt care
about the difference between write-only and read-write
transactions and all are read-write from backend point of view
we can safely remove writeOnly counter.

Change-Id: I18046ad3b229755bbd0daa788a3fe62f59e4140d
Signed-off-by: Tomas Cere <>
18 months agoUse Files.delete() in LocalSnapshotStore 13/82213/2
Tomas Cere [Thu, 23 May 2019 08:04:16 +0000 (10:04 +0200)]
Use Files.delete() in LocalSnapshotStore

If delete fails, it needs to be logged so change this up.

Change-Id: If4051cb3e5b83dee51f919fc1713cc17f032597e
Signed-off-by: Tomas Cere <>
18 months agoDo not enque purges for ask based protocol 37/82137/16
Tomas Cere [Thu, 16 May 2019 12:12:01 +0000 (14:12 +0200)]
Do not enque purges for ask based protocol

With metadata disabled for ask based protocol, there is no need
to purge transaction on the backend and pollute the journal.

Tell based protocol has purges handled by the fronted so it
should be safe for us to remove them on the backend for ask

Change-Id: Ia5c9470782bb4d59b78bbaa605ee5d36c7fb1644
Signed-off-by: Tomas Cere <>
18 months agoRemove PersistAbortTransactionPayload 27/82027/14
Tomas Cere [Tue, 14 May 2019 10:56:04 +0000 (12:56 +0200)]
Remove PersistAbortTransactionPayload

With the metadata tracking disabled for ask based protocol
there is no need to track aborts for read only transactions
on the backend.

Change-Id: I189ae3231bb2f3c0eaa0bbe21a14342446708c5f
Signed-off-by: Tomas Cere <>
18 months agoDo not leak DataTree from backend actor 82/82182/1
Robert Varga [Tue, 21 May 2019 08:47:34 +0000 (10:47 +0200)]
Do not leak DataTree from backend actor

YANG tools is now exposing the read-only part of DataTree as
ReadOnlyDataTree. Use this interface to restrict the set of
methods accessible to frontend.

Change-Id: I9426d4cee1e68e3cc21f7a39322a0df09739e770
Signed-off-by: Robert Varga <>
18 months agoDisable transaction tracking for ask-based protocol 40/82140/8
Robert Varga [Thu, 16 May 2019 13:54:29 +0000 (15:54 +0200)]
Disable transaction tracking for ask-based protocol

When we encounter an ask-based protocol message on a leader,
we persist a payload to make sure no state tracking occurs
for that client.

Change-Id: I3d12a06ce9e5b65ada5b59bde9f9b5486e5e4ef7
Signed-off-by: Robert Varga <>
Signed-off-by: Tomas Cere <>
18 months agoAllow transaction tracking to be disabled 83/81983/13
Robert Varga [Fri, 10 May 2019 10:25:31 +0000 (12:25 +0200)]
Allow transaction tracking to be disabled

Ask-based protocol does not need tracking of transactions and
histories, as it is not retransmitting requests. It also does
not inform backend about purely-local aborted transactions
(read-write and read-only), which leads to transaction tracking
rangesets having holes where those IDs are used.

This adds the prerequisite handling of disabling from the leader
without adding the actual mechanics.

Change-Id: I133e7688b492336937f394f0f6c3f080a05a820f
Signed-off-by: Robert Varga <>
Signed-off-by: Tomas Cere <>
18 months agoRemove QueuedNotificationManagerMXBean 71/82171/1
Robert Varga [Mon, 20 May 2019 10:43:49 +0000 (12:43 +0200)]
Remove QueuedNotificationManagerMXBean

This interface is available from yangtools directly, remove
it from controller.

Change-Id: Ic894c0e9fc5d114ebff07bb11f67f1ec67611031
Signed-off-by: Robert Varga <>
18 months agoDerive config-artifacts from odlparent-lite 71/82071/1
Robert Varga [Wed, 15 May 2019 07:58:52 +0000 (09:58 +0200)]
Derive config-artifacts from odlparent-lite

This is to fix failures seen on promoting to Central.

Change-Id: I6204d68c5144437e5af50bb5f2cb29431841d394
Signed-off-by: Robert Varga <>

©2013 OpenDaylight, A Linux Foundation Collaborative Project. All Rights Reserved.
OpenDaylight is a registered trademark of The OpenDaylight Project, Inc.
Linux Foundation and OpenDaylight are registered trademarks of the Linux Foundation.
Linux is a registered trademark of Linus Torvalds.