3 years agoRemove unneeded Java 9+ check 56/86456/1
Robert Varga [Tue, 17 Dec 2019 02:47:56 +0000 (03:47 +0100)]
Remove unneeded Java 9+ check

We are running at least on Java 11, hence this check is useless.

Change-Id: I802c80384e652b18ef9ec0586acbface73a2cde4
Signed-off-by: Robert Varga <>
3 years agoRemove ARIES-1923 workaround 40/86440/3
Robert Varga [Mon, 16 Dec 2019 21:07:05 +0000 (22:07 +0100)]
Remove ARIES-1923 workaround

We now have proper monkey-patch in place and can therefore remove
this workaround.

Change-Id: I2f4665131953964a1ed64848843e039ccc7015c3
Signed-off-by: Robert Varga <>
3 years agoFix mdsal-it-base karaf setup 55/86455/2
Robert Varga [Tue, 17 Dec 2019 02:43:17 +0000 (03:43 +0100)]
Fix mdsal-it-base karaf setup

Since we are setting up a karaf environment, we must make sure
we are not overriding opendaylight-karaf-empty, otherwise we are
not matching distribution (and not picking up its fixes,
to ARIES-1923 for example).

Change-Id: I9f723ee2edd13ee4848e302c572285517a1d552b
Signed-off-by: Robert Varga <>
3 years agoBump odlparent/yangtools/mdsal to 6.0.3/4.0.3/5.0.6 62/86362/6
Robert Varga [Wed, 11 Dec 2019 23:48:08 +0000 (00:48 +0100)]
Bump odlparent/yangtools/mdsal to 6.0.3/4.0.3/5.0.6

Pick up the latest upgrades and fixes.

Change-Id: I05d54f6ad7ec934f77f123efaffc9a8a090a6384
Signed-off-by: Robert Varga <>
3 years agoRemove unneeded javassist dependency 12/86412/1
Robert Varga [Thu, 12 Dec 2019 15:32:19 +0000 (16:32 +0100)]
Remove unneeded javassist dependency

The test is not using javassist, let's ditch the declaration.

Change-Id: I9226f2029cc4679f4fee8e126db8735e2e1ecd7d
Signed-off-by: Robert Varga <>
3 years agoBail faster on not found module 77/85977/4
Robert Varga [Tue, 26 Nov 2019 14:31:49 +0000 (15:31 +0100)]
Bail faster on not found module

If we fail to find config for a particular module, we can bail
directly without going through another empty check.

Change-Id: I47dc844d51ecbe1ca3f22fd355532539e07f798b
Signed-off-by: Robert Varga <>
3 years agoSwitch default stream output to Magnesium 95/84695/23
Robert Varga [Wed, 25 Sep 2019 06:16:56 +0000 (08:16 +0200)]
Switch default stream output to Magnesium

This flips the switch and uses Magnesium format for all unversioned
users. Also updates tests to reflect new expected stream sizes.

Change-Id: I0ed5abe495124b525b7a8c877a3cc7b09f3244ca
Signed-off-by: Robert Varga <>
3 years agoSwitch current {ABI,DataStore,Payload}Version to Magnesium 10/84710/7
Robert Varga [Wed, 25 Sep 2019 14:29:34 +0000 (16:29 +0200)]
Switch current {ABI,DataStore,Payload}Version to Magnesium

This mass-switches versioned resources to use Magnesium streaming
format. It also updates the hand-written test data to use Uint64
instead of BigInteger, as Magnesium does not support BigIntegers.

Change-Id: I4734fe3fe0064809fde899ea127e98380c8aacf4
Signed-off-by: Robert Varga <>
3 years agoAllow shard settle timeout to be tuned 41/85741/9
Tibor Král [Mon, 11 Nov 2019 17:23:31 +0000 (18:23 +0100)]
Allow shard settle timeout to be tuned

When we are starting the datastore, we wait up to a fixed number of
election timeout intervals (3) for shards to finish electing a leader.

This is not always enough, especially if recovery takes a long time,
hence introduce shardLeaderElectionTimeoutMultiplier to make this

Change-Id: Iba1d116d0248fc6046aeeae3ec30ecac50f373c9
Signed-off-by: Tibor Král <>
Signed-off-by: Robert Varga <>
3 years agoAdd javadoc links to yangtools-docs and mdsal-docs 78/85878/1
Robert Varga [Wed, 20 Nov 2019 08:14:27 +0000 (09:14 +0100)]
Add javadoc links to yangtools-docs and mdsal-docs

Cross reference controller-docs with upstreams.

Change-Id: I2143d6971b4bf4bceab331e350c7363755b03a4a
Signed-off-by: Robert Varga <>
3 years agoBump to odlparent-6.0.1/yangtools-4.0.2/mdsal-5.0.4 49/85549/1
Robert Varga [Mon, 4 Nov 2019 13:34:56 +0000 (14:34 +0100)]
Bump to odlparent-6.0.1/yangtools-4.0.2/mdsal-5.0.4

This picks up latest upgrades and fixes.

Change-Id: I8db358b0af81394a561071d98bffea168dda47cc
Signed-off-by: Robert Varga <>
3 years agoAllow AbstractClientActor generation to start from non-zero 97/85397/2
Robert Varga [Fri, 25 Oct 2019 13:50:59 +0000 (15:50 +0200)]
Allow AbstractClientActor generation to start from non-zero

This adds the ability to override the initial generation seeded
into persistence. The requested generation is controlled via
system properties on a per-client-type basis:

org.opendaylight.controller.cluster.access.client.initial.generation.datastore-operational = 500

This is useful during bringup of a zombie node, which lost all
of its persistence, for example during recovery from backup.

Change-Id: If32487fedcb5b6b39551a3359c069df940a6a725
Signed-off-by: Robert Varga <>
3 years agoAdd the ability to report known connected clients 73/85373/8
Robert Varga [Thu, 24 Oct 2019 11:54:38 +0000 (13:54 +0200)]
Add the ability to report known connected clients

In order to allow from-scratch recovery, we need the ability to
query all existing shards to provide the identifiers of clients
they know about. This patch adds the models to support such a query
as well as implementing it in cluster-admin.

Change-Id: Id69aeb9021c8111dad10930620c9eaacfccd0d94
Signed-off-by: Robert Varga <>
3 years agoRemove dependency section from mdsal parent 84/85384/1
Robert Varga [Fri, 25 Oct 2019 07:59:36 +0000 (09:59 +0200)]
Remove dependency section from mdsal parent

This pom acts as an extension to binding parent, it is not a good
idea to also add dependencies here -- each artifact should list
them as needed.

Change-Id: I1524a234043ce334ca1a82f4e72f4fec7acdacbf
Signed-off-by: Robert Varga <>
3 years agoImport scala.concurrent.Future 74/85374/2
Robert Varga [Thu, 24 Oct 2019 15:49:08 +0000 (17:49 +0200)]
Import scala.concurrent.Future

Since we are returning ListenableFuture from RPCs, we can freely
import Scala's Future and shorten references to it.

Change-Id: I77c911fa6d9eca5804547039715f3e57866bafcf
Signed-off-by: Robert Varga <>
3 years agoMigrate Props.create() call sites 71/85371/3
Robert Varga [Thu, 24 Oct 2019 10:58:27 +0000 (12:58 +0200)]
Migrate Props.create() call sites

Props.create() now takes a class identifying the class being
instantiated. Fix callsites to do that.

Change-Id: I4a0169663465b0c1ba62fc87ca13423cf5a4464f
Signed-off-by: Robert Varga <>
3 years agoUse singleton NodeIdentifierWithPredicates 66/85366/1
Robert Varga [Thu, 24 Oct 2019 09:38:23 +0000 (11:38 +0200)]
Use singleton NodeIdentifierWithPredicates

NodeIdentifierWithPredicates has a specialized single-entry class,
rendering SharedSingletonMap(Template) use unnecessary, which is
flag as a warning. Use the singleton constructor directly.

Change-Id: Ib16348192040d4e4f9d135e792880d8519cad4d6
Signed-off-by: Robert Varga <>
3 years agoRemove unused model imports 40/85340/1
Robert Varga [Wed, 23 Oct 2019 09:07:24 +0000 (11:07 +0200)]
Remove unused model imports

pyang is pointing out these imports as unused, fix them up.

Change-Id: Id3d59d35d764bc91e1d381de72caca2622ae4a6d
Signed-off-by: Robert Varga <>
3 years agoRemove unneeded checkstyle/spotbugs declaration 81/85281/2
Robert Varga [Mon, 21 Oct 2019 03:59:08 +0000 (05:59 +0200)]
Remove unneeded checkstyle/spotbugs declaration

odlparent is enabling these, remove superfluous configuration.

Change-Id: I80179e97046653b7e96ee79dbbd4e392ab47ae0f
Signed-off-by: Robert Varga <>
3 years agoAdd locate-shard RPC 38/85038/5
Robert Varga [Thu, 10 Oct 2019 13:36:42 +0000 (15:36 +0200)]
Add locate-shard RPC

This adds the ability to query what is the preferred node to talk
to a particular shard.

Change-Id: Idcdf9e60829d14c2f1a8cc4aceec0b3b989ef07d
Signed-off-by: Robert Varga <>
3 years agoCleanup cluster-admin.yang 33/85033/6
Robert Varga [Thu, 10 Oct 2019 11:08:00 +0000 (13:08 +0200)]
Cleanup cluster-admin.yang

We are redefining the same leaves over and over, make sure we
define them in a grouping for common definition. Also ensure
shard name requires at least one character -- just as we expect
it to happen in the implementation.

Change-Id: I917d8a7767467dceb4f3e076ffca4b5aa4b048a1
Signed-off-by: Robert Varga <>
3 years agoRegister action MXBean only during start 01/85201/1
Robert Varga [Fri, 18 Oct 2019 10:58:28 +0000 (12:58 +0200)]
Register action MXBean only during start

We should not be registering resources until we are started, this
fixes an exception seen in Genius UT.

Change-Id: I5a95d7e2ddbc71eca23c57843eea64f0c58dfc7c
Signed-off-by: Robert Varga <>
3 years agoRemove unneeded version declaration 47/85147/1
Robert Varga [Wed, 16 Oct 2019 12:01:45 +0000 (14:01 +0200)]
Remove unneeded version declaration

The model is properly exposed from artifacts, do not override the
managed version.

Change-Id: Iaf5fe83bc0e0f7c7fb5a20a0b23b848b572e73e6
Signed-off-by: Robert Varga <>
3 years agoUse ConcurrentHashMap.newKeySet() 46/85146/1
Robert Varga [Wed, 16 Oct 2019 11:51:16 +0000 (13:51 +0200)]
Use ConcurrentHashMap.newKeySet()

Instead of using Sets.newConcurrentHashSet(), use the utility
method provided by ConcurrentHashMap.

Change-Id: I796e8b6b260f865138a2716f1bb61c983972a4d4
Signed-off-by: Robert Varga <>
3 years agoSplit DistributedShardRegistration into its own file 44/85144/2
Robert Varga [Wed, 16 Oct 2019 10:32:42 +0000 (12:32 +0200)]
Split DistributedShardRegistration into its own file

This interface is holding us back at Java 10 due to ARIES-1923,
as blueprint fails to process DistributedShardFactory. As this
interface really is private, factor DistributedShardRegistration
into its own file and move to Java 11.

Change-Id: Idea1c31b7e82a9df5cef34da8e2bd95bb51d4f00
Signed-off-by: Robert Varga <>
3 years agoMigrate deprecated testing components 41/85141/2
Robert Varga [Wed, 16 Oct 2019 09:26:42 +0000 (11:26 +0200)]
Migrate deprecated testing components

org.scalatest.junit.JUnitRunner and org.mockito.Matchers have
replacements, migrate to using them.

Change-Id: Iff6fa0c937361bb68d4d001feee29af21a430836
Signed-off-by: Robert Varga <>
3 years agoUse careful byte-masking/shifting in Mg Input 99/85099/2
Robert Varga [Mon, 14 Oct 2019 08:16:02 +0000 (10:16 +0200)]
Use careful byte-masking/shifting in Mg Input

We really want to operate on byte and not int, the bytes are to
be treated as unsigned, which gets wrecked by sign extension.

Introduce an explicit mask(byte, byte) method and use it where
we are potentially sign-extending.

Change-Id: I0484f440c1589859fb268947f5a80a2b4969451e
Signed-off-by: Robert Varga <>
3 years agoDeprecate old MD-SAL APIs for removal 57/84857/10
Robert Varga [Tue, 1 Oct 2019 17:05:29 +0000 (19:05 +0200)]
Deprecate old MD-SAL APIs for removal

These APIs are slated for removal in Aluminium, make sure our
warning is more stern.

Change-Id: Ib4f7dcfd5183b87dd10b9ce2b773a59a548892c9
Signed-off-by: Robert Varga <>
3 years agoDo not use Binding DTO compat methods 12/84712/20
Robert Varga [Wed, 25 Sep 2019 14:47:24 +0000 (16:47 +0200)]
Do not use Binding DTO compat methods

This removes the use of compat methods, preferring explicit
instantiation of the correct type.

Change-Id: Ie3b739e1e31ba2762cdda92444f4734a8ac47ae8
Signed-off-by: Robert Varga <>
3 years agoBump mdsal to 5.0.3 61/85061/1
Robert Varga [Fri, 11 Oct 2019 14:09:15 +0000 (16:09 +0200)]
Bump mdsal to 5.0.3

This bumps mdsal to 5.0.3 to workaround issues with Blueprint

Change-Id: I848c9220ce06ec7b4af304ab64948bdc30257dbd
Signed-off-by: Robert Varga <>
3 years agoRegister MXBean only during start 57/85057/1
Robert Varga [Fri, 11 Oct 2019 12:32:14 +0000 (14:32 +0200)]
Register MXBean only during start

We should not be registering resources until we are started, this
fixes an exception seen in Genius UT.

Change-Id: I550b586f905ec95a06cc9aaec167eb584592ba46
Signed-off-by: Robert Varga <>
3 years agoDisable slf4j SSL link 42/85042/3
Robert Varga [Thu, 10 Oct 2019 18:59:08 +0000 (20:59 +0200)]
Disable slf4j SSL link

slf4j does not provide apidocs over TLS (WTF?!?!?!), link HTTP
site instead.

Change-Id: I2158c043a2d9e810a329587159fc6af9d57d8e56
Signed-off-by: Robert Varga <>
3 years agoRemove CheckedFuture from RemoteSchemaProvider 36/85036/5
Robert Varga [Thu, 10 Oct 2019 12:05:49 +0000 (14:05 +0200)]
Remove CheckedFuture from RemoteSchemaProvider

The API contract we are implementing is defined in terms of
ListenableFuture, hence we can drop the use of CheckedFuture.

Change-Id: I9d99b2e42c3995c4c11b44e595a81a93b2b31b7e
Signed-off-by: Robert Varga <>
3 years agoBump mdsal to 5.0.2 03/83803/53
Robert Varga [Tue, 20 Aug 2019 14:37:14 +0000 (16:37 +0200)]
Bump mdsal to 5.0.2

This bumps mdsal to 5.0.2 as part of Magnesium MRI.

Change-Id: I93e10e95b7a241d678829afb732e2adef82c1a24
Signed-off-by: Robert Varga <>
3 years agoBump yangtools to 4.0.1 04/83804/49
Robert Varga [Tue, 20 Aug 2019 14:37:51 +0000 (16:37 +0200)]
Bump yangtools to 4.0.1

This bumps yangtools to 4.0.1 as part of Magnesium MRI.

Change-Id: I1ac7d3f856af310f4e8015875223d30eac3a8276
Signed-off-by: Robert Varga <>
3 years agoBump odlparent to 6.0.0 02/83802/42
Robert Varga [Tue, 20 Aug 2019 14:36:13 +0000 (16:36 +0200)]
Bump odlparent to 6.0.0

This bumps odlparent to 6.0.0 as part of Magnesium MRI.

Change-Id: I084c6a32c4facafd85e3b87c6d8a9a29a33848a8
Signed-off-by: Robert Varga <>
3 years agoSplit up transaction chunks 53/84553/13
Robert Varga [Thu, 19 Sep 2019 23:50:56 +0000 (01:50 +0200)]
Split up transaction chunks

This adds the infrastructure to prevent allocation of large byte[]s,
as when those exceed 0.5-16MiB, under G1GC they end up in humongous
object region. Not only that, but after some cut-off point, the
copying of arrays starts to dominate performance.

What we do here is ensure we always receive up to a configurable
number of bytes, defaults to 256KiB, and keep those chunks in a list.
This way we may end up with larger overhead, but that really is
neglibeble -- even a 2GiB payload would end up using only about
8K arrays.

While the input/output streams are similar to, the design here
is geared towards having the intermediate representation available
as well as devolving to a single byte[] for memory efficiency

Change-Id: I2b79a633ebf4fdf8d68d2accc644326e30b41f22
Signed-off-by: Robert Varga <>
3 years agoUse ReusableNormalizedNodeReceiver 65/84765/3
Robert Varga [Fri, 27 Sep 2019 05:12:32 +0000 (07:12 +0200)]
Use ReusableNormalizedNodeReceiver

This simplifies some APIs by using ReusableStreamReceiver, as it
is an API-level construct rather than a particular implementation.

Change-Id: Id9d14d512ac7150a3e71c104577546e219cd86d0
Signed-off-by: Robert Varga <>
3 years agoFixup docs references 91/84791/1
Robert Varga [Fri, 27 Sep 2019 15:50:34 +0000 (17:50 +0200)]
Fixup docs references

This updates references very slightly, so that we do not have
outdated or unnecessary pointers.

Change-Id: I5c9429cf8ec1af244cbd4e73772a80213af91a85
Signed-off-by: Robert Varga <>
3 years agoMove DataTreeCandidate extraction to where it is applied 10/82410/15
Robert Varga [Wed, 5 Jun 2019 10:45:15 +0000 (12:45 +0200)]
Move DataTreeCandidate extraction to where it is applied

This patch just moves the code that is extracting the candidate
from payload into the same method that is going to apply it, giving
us an opportunity to merge the two interfaces.

Change-Id: I44a5e2f7181147dca9377d0a9516bbe6b3ba85e3
Signed-off-by: Robert Varga <>
3 years agoAdd uint adaptation to AbstractNormalizedNodePruner 16/84716/3
Robert Varga [Wed, 25 Sep 2019 15:34:53 +0000 (17:34 +0200)]
Add uint adaptation to AbstractNormalizedNodePruner

When migrating from old snapshots to magnesium format we have to
make sure we also legacy uint mapped values, so that after recovery
we end up with proper types being present.

Change-Id: I86ee49efee0d6c0519680197ea3e3985b1566e0c
Signed-off-by: Robert Varga <>
3 years agoRework AbstractNormalizedNodePruner 20/81320/41
Robert Varga [Tue, 14 May 2019 08:15:19 +0000 (10:15 +0200)]
Rework AbstractNormalizedNodePruner

This refactors NormalizedNodePruner to use
ReusableImmutableNormalizedNodeStreamWriter for creation of the
actual NormalizedNode tree.

It simplifies the implementation and allows the reuse of minor
optimizations like LeafInterner.

Change-Id: I93bd00bcff13245d808f7adf6122a6c20156e9fd
Signed-off-by: Robert Varga <>
3 years agoDo not update term from unreachable members 76/82876/10
Tomas Cere [Tue, 2 Jul 2019 07:58:13 +0000 (09:58 +0200)]
Do not update term from unreachable members

In case of a one way connection problem, a follower that cannot
receive messages but can send them, it will change into Candidate
and continuosly bump its term while it cannot receive any

Since the messages it sends can actually get through, this means
that the healthy part of the cluster will have its leader dropped
every 2 election timeouts.

Change this up so we only update term when receiving a RequestVote
from a reachable node.

Change-Id: Ia8a0b59cfb2d0cd68096172a1d2d47f68e7ed473
Signed-off-by: Tomas Cere <>
Signed-off-by: Robert Varga <>
4 years agoSwitch default stream output to Sodium SR1 39/84539/40
Robert Varga [Wed, 25 Sep 2019 06:01:47 +0000 (08:01 +0200)]
Switch default stream output to Sodium SR1

This flips the switch and uses Sodium SR1 format for all unversioned
users. Also updates tests to reflect new expected stream sizes.

Change-Id: I9973bb93f469c1b59533ce580c795cd30c9edda4
Signed-off-by: Robert Varga <>
4 years agoSwitch current {ABI,DataStore,Payload}Version to Sodium SR1 09/84709/4
Robert Varga [Wed, 25 Sep 2019 14:25:08 +0000 (16:25 +0200)]
Switch current {ABI,DataStore,Payload}Version to Sodium SR1

This mass-switches versioned resources to use Sodium SR1 streaming

Change-Id: Ic712fa04e16c68481eeecccc61d8e6a7dfabac52
Signed-off-by: Robert Varga <>
4 years agoDefine PayloadVersion.MAGNESIUM 05/84705/10
Robert Varga [Wed, 25 Sep 2019 10:45:18 +0000 (12:45 +0200)]
Define PayloadVersion.MAGNESIUM

This adds the PayloadVersion which uses Magnesium normalized node

Change-Id: Ie43e9932a19bd19c92f14890da13646bf56e2f6f
Signed-off-by: Robert Varga <>
4 years agoEliminate reference to YangInstanceIdentifier.EMPTY 11/84711/3
Robert Varga [Wed, 25 Sep 2019 14:45:57 +0000 (16:45 +0200)]
Eliminate reference to YangInstanceIdentifier.EMPTY

This constant has been removed, migrate to its replacement.

Change-Id: I0937b4602552ec8101132319889bc92e63de06cb
Signed-off-by: Robert Varga <>
4 years agoDefine DataStoreVersions.MAGNESIUM_VERSION 03/84703/10
Robert Varga [Wed, 25 Sep 2019 10:38:17 +0000 (12:38 +0200)]
Define DataStoreVersions.MAGNESIUM_VERSION

This adds Magnesium version, which is using Magnesium normalized
node stream.

Change-Id: Icc1314d61e37b8a4bf2df759860b4d0bc170c90c
Signed-off-by: Robert Varga <>
4 years agoAdd cds-access-api MAGNESIUM version 04/84704/9
Robert Varga [Wed, 25 Sep 2019 10:41:24 +0000 (12:41 +0200)]
Add cds-access-api MAGNESIUM version

This adds a new ABI version, which uses Magnesium normalized node

Change-Id: Ie27b6cbd090b6f155c9600055ac6ef12304b1a99
Signed-off-by: Robert Varga <>
4 years agoDefine PayloadVersion.SODIUM_SR1 02/84702/9
Robert Varga [Wed, 25 Sep 2019 10:33:15 +0000 (12:33 +0200)]
Define PayloadVersion.SODIUM_SR1

This adds the PayloadVersion which uses Sodium SR1 normalized node

Change-Id: I3dfa6ea053ea356bb68b458f3d6787904b2904c0
Signed-off-by: Robert Varga <>
4 years agoMark historic DataStoreVersions deprecated 00/84700/2
Robert Varga [Wed, 25 Sep 2019 10:26:43 +0000 (12:26 +0200)]
Mark historic DataStoreVersions deprecated

We do not support anything before Boron, hence mark these constants
as deprecated.

Change-Id: Ibda5ab8f5bbcb8136e01018d713685e1e3b6ee7c
Signed-off-by: Robert Varga <>
4 years agoSwitch ABIVersion/DataStoreVersions back to Neon SR2 08/84708/1
Robert Varga [Wed, 25 Sep 2019 14:16:22 +0000 (16:16 +0200)]
Switch ABIVersion/DataStoreVersions back to Neon SR2

This switches the current versions for use back to Neon SR2, reverting
the rollout.

Change-Id: I57175fb26a86a1de589602bcec3b4c022ddf8a72
Signed-off-by: Robert Varga <>
4 years agoUse explicit versioning in MetadataShardDataTreeSnapshot 07/84707/3
Robert Varga [Wed, 25 Sep 2019 13:39:28 +0000 (15:39 +0200)]
Use explicit versioning in MetadataShardDataTreeSnapshot

Rather than relying on the default version, make sure we use
whatever the version is defined for the message at hand.

Change-Id: Ic325ba93e5152570586f7a0baa8ce2abf8053c4e
Signed-off-by: Robert Varga <>
4 years agoDefine DataStoreVersions.SODIUM_SR1_VERSION 01/84701/3
Robert Varga [Wed, 25 Sep 2019 10:24:39 +0000 (12:24 +0200)]
Define DataStoreVersions.SODIUM_SR1_VERSION

This adds Sodium SR1 version, which is using SodiumSR1 normalized
node stream.

Change-Id: Ic79940cd80dfa1e2844079bc3003ab815c586662
Signed-off-by: Robert Varga <>
4 years agoAdd cds-access-api SODIUM_SR1 version 99/84699/4
Robert Varga [Wed, 25 Sep 2019 10:18:29 +0000 (12:18 +0200)]
Add cds-access-api SODIUM_SR1 version

This adds a new ABI version, which uses SodiumSR1 normalized node

Change-Id: I457c2fcc7e10f4ba2021ce6ddb4fbc2adbfdd596
Signed-off-by: Robert Varga <>
4 years agoAdd Magnesium stream version 94/84694/8
Robert Varga [Wed, 25 Sep 2019 06:07:50 +0000 (08:07 +0200)]
Add Magnesium stream version

This is a variant of Sodium SR1 streaming format, which does not
allow presence of BigInteger tokens, as those values are superseded
by Uint64.

Change-Id: I1dc5c7376bf1ecf105e965e42d74806ed99ab9b1
Signed-off-by: Robert Varga <>
4 years agoAdd LithiumSR1 input/output support 74/84274/53
Robert Varga [Mon, 9 Sep 2019 08:29:57 +0000 (10:29 +0200)]
Add LithiumSR1 input/output support

This is a code drop of both DataInput and DataOutput for Sodium SR1
streaming format. It uses its own dedicated tokes, along with quite
a few of stateful encoding rules.

Where Lithium did encoding did not perform any look-behind except
for LeafSetEntryNodes and only maintained simple coding tables,
this coder maintains simple 'parent' state tracking.

While the state tracking is not mandated on the encoder side, some
amount of tracking is required in the decoder side to provide full
coverage of coding constructs.

Two examples of this are:
- parent QName references, which are expanded to cover not only
  LeafSet/LeafSetEntry, but also UnkeyedList/UnkeyedListEntry
  and most importantly Map/MapEntry pairs
- Map/MapEntry tracking is also important for coding leaf nodes
  which represent keys -- in those cases emitting the leaf value
  is superfluous, as it can be looked up from MapEntry's predicates

Also this state tracking is maintained on-stack, thus making issues
related with it nigh impossible (unlike the Lithium LeafSetEntry

Change-Id: I5fcf85a76750301cd3f8bcbd505baa0f95397cb5
Signed-off-by: Robert Varga <>
4 years agoAdd Magnesium encoding tokens 57/84557/13
Robert Varga [Fri, 20 Sep 2019 08:07:10 +0000 (10:07 +0200)]
Add Magnesium encoding tokens

Changes to how uint8/16/32/64 types are mapped results in the need
to transfer the new value types. These represent 4 brand new tokens,
which need to be handled.

One option would be to fork NeonSR2 and modify it, somehow dealing
with value lookups (i.e. Uint8 is not valid in SR2 stream, but tokens
are shared across Lithium/NeonSR).

Another option is to create a streaming format from scratch, coming
up with a completely new family of tokens and encoding rules. This
is attractive, as both current formats have three deficiencies:

1) they encode MapEntry key values twice, once in the leaf itself
   and once in the entry's NodeIdentifierWithPredicates
2) they are '0-happy', i.e. they do not recognize that integer codes
   and sizes are typically much smaller than the 4-byte range and thus
   much of the stream is just small ints encoded as 4 bytes
3) while they are simple and straighforward, they also end up wasting
   bits -- the four token families (codes, value types, node types,
   path argument types) each have at most 15 distinct values, and
   hence the bytes used to transmit them have 4 empty bits (or more)

Hence we take the other option and design a streaming format with
brand new tokens, which are structured to carry as much more
information per byte.

This is done by combining 'type' information with forward coding
hints, for example:

- a Boolean leaf value would previously require 2 bytes:
    (byte)    ValueTypes.BOOL_TYPE
    (byte)    true/falseIn the new tokens
  whereas new tokens express this in one byte:
    (byte)    LeafValue.BOOLEAN_{FALSE,TRUE}
- a byte[5] value would require 10 bytes to encode:
    (byte)    ValueTypes.BINARY_TYPE
    (int)     length
    (5 bytes) bytes
  whereas new tokens express this in 6 bytes:
    (byte)    ValueTypes.BINARY_0 + 5 (works up to 127)
    (5 bytes) bytes

A similar approach is taken when encoding NormalizedNode types. Here
we recognize there are only 14 node types (currently) and hence we
can provide up to 4 additional bits for information about how the
node's identifier is encoded.

For PathArguments, we also use separate coding, where the format
is flexible based on the 4 types on identifiers there are, each
having slightly different format. Most notable here is the integration
of QName reference coding (for normal PathArguments) and integrated
set size coding (for AugmentationIdentifiers).

This this patch adds just the token definitions, along with basic
documentation and version declaration.

Change-Id: I9f6f58a7cb77d9d98c46e13b8dc6955c8e3c0737
Signed-off-by: Robert Varga <>
4 years agoAdd an explicit namespace sharing test 91/84691/2
Robert Varga [Tue, 24 Sep 2019 13:16:15 +0000 (15:16 +0200)]
Add an explicit namespace sharing test

This adds a test to target QName/QNameModule/String reference

Change-Id: Iad736a81f627da4a04753b380acb530ccc8ba219
Signed-off-by: Robert Varga <>
4 years agoSuppress modernization 33/83833/28
Robert Varga [Tue, 20 Aug 2019 19:29:51 +0000 (21:29 +0200)]
Suppress modernization

Modernize as much as possible, but as we have legacy APIs in play,
we have to add also some suppressions.

Change-Id: I844821f129ad0bb0e186b6a4848a6d2937513f7b
Signed-off-by: Robert Varga <>
4 years agoRework PathArgumentTypes lookup 90/84090/19
Robert Varga [Fri, 30 Aug 2019 09:08:37 +0000 (11:08 +0200)]
Rework PathArgumentTypes lookup

This lookup relying on final methods, which is not a correct
assumption, with NodeIdentifierWithPredicates having at least two
properly-hidden implementations.

Rework the lookup in terms of instanceof checks, which saves a table
lookup and unboxing operation.

Change-Id: Ia6b272715d19e432e7e0839b6294035e12b1d293
Signed-off-by: Robert Varga <>
4 years agoAdd more serialization assertions 22/84622/3
Robert Varga [Mon, 23 Sep 2019 06:23:29 +0000 (08:23 +0200)]
Add more serialization assertions

This adds a few more assertions to ensure our serialization format
does not move without us knowing.

Change-Id: Ieed326e9e57fb15ea46cd7a088d713222963e2e5
Signed-off-by: Robert Varga <>
4 years agoMove Neon SR2 tokens into its own class 63/84563/3
Robert Varga [Fri, 20 Sep 2019 17:42:15 +0000 (19:42 +0200)]
Move Neon SR2 tokens into its own class

Same as with base Lithium tokens, these can create confusion about
where they are used. Make sure we encapsulate them in their own
class, so no confusion ensues.

Change-Id: I197b9118a5d9a09562db0a899bb6b8dc0dfae28b
Signed-off-by: Robert Varga <>
4 years agoRename ValueTypes to LithiumValue 62/84562/3
Robert Varga [Fri, 20 Sep 2019 17:59:04 +0000 (19:59 +0200)]
Rename ValueTypes to LithiumValue

These constants are only used in Lithium-based streams, make sure
we reflect that in the name of the class, so they do not end up
being confused.

Change-Id: I2e880b825d82b8a9b76008248d7663eab4bc1d71
Signed-off-by: Robert Varga <>
4 years agoRename PathArgumentTypes to LithiumPathArgument 61/84561/3
Robert Varga [Fri, 20 Sep 2019 17:55:39 +0000 (19:55 +0200)]
Rename PathArgumentTypes to LithiumPathArgument

These tokens are only used in Lithium-based streams, but the global
name is slightly confusing. Rename to LithiumPathArgument, which
makes the context more obvious.

Change-Id: I2b92be4c469de8dffba6e071449b5ae947b9ace8
Signed-off-by: Robert Varga <>
4 years agoRename NodeTypes to LithiumNode 60/84560/3
Robert Varga [Fri, 20 Sep 2019 17:54:04 +0000 (19:54 +0200)]
Rename NodeTypes to LithiumNode

This makes it clear these constants are only used in Lithium

Change-Id: I1fb44f49d3409fdd007fe58dc6ce92b0eb6744b8
Signed-off-by: Robert Varga <>
4 years agoMove Lithium tokens to their own class 59/84559/3
Robert Varga [Fri, 20 Sep 2019 17:36:10 +0000 (19:36 +0200)]
Move Lithium tokens to their own class

TokenTypes assumes singularity, which can create confusion, as we
have multiple multiple distinct sets. This separates out
Lithium-specific tokens into their own class.

Change-Id: Id8c47be7c432b44980dbeac8ef41d430c64e5490
Signed-off-by: Robert Varga <>
4 years agoMigrate from YangInstanceIdentifier.EMPTY 91/84091/17
Robert Varga [Fri, 30 Aug 2019 09:10:53 +0000 (11:10 +0200)]
Migrate from YangInstanceIdentifier.EMPTY

This migrates to the replacement empty() method.

Change-Id: I32ca026a33835edb16f8e1e6142e9192c880076b
Signed-off-by: Robert Varga <>
4 years agoBump mdsal to 4.0.6 20/84420/7
Robert Varga [Fri, 13 Sep 2019 18:56:56 +0000 (20:56 +0200)]
Bump mdsal to 4.0.6

This picks up latest fixes.

Change-Id: I64e1f277bd00e132f40647eb0a2e7fc00eb56705
Signed-off-by: Robert Varga <>
4 years agoBump yangtools to 3.0.5 85/84385/8
Robert Varga [Fri, 13 Sep 2019 14:13:53 +0000 (16:13 +0200)]
Bump yangtools to 3.0.5

This picks up the latest fixes and improvements.

Change-Id: Icb9c44cd6f8206e78e7d50322dde3356405721c8
Signed-off-by: Robert Varga <>
4 years agoBump odlparent to 5.0.2 01/83801/24
Robert Varga [Tue, 20 Aug 2019 13:59:12 +0000 (15:59 +0200)]
Bump odlparent to 5.0.2

This bumps odlparent to pick up latest references.

Change-Id: I73af7ad69f686cb5bacc6ccfe0f57d226a43994b
Signed-off-by: Robert Varga <>
4 years agoAdd a 100K-entry test 52/84552/2
Robert Varga [Thu, 19 Sep 2019 21:59:24 +0000 (23:59 +0200)]
Add a 100K-entry test

This adds a test for serialization of a Map of 100K entries,
each of which has the key leaf and a simple constant non-key leaf.
Unlike other tests, this quantifies how well the encoding works
when faced with large maps.

Change-Id: I306e5175ba595a0967432393766f8317e7205492
Signed-off-by: Robert Varga <>
4 years agoAdd encoding size asserts 50/84550/1
Robert Varga [Thu, 19 Sep 2019 19:20:16 +0000 (21:20 +0200)]
Add encoding size asserts

There is a number of places where we check serialize/deserialize
operations through ByteArrayOutputStream. In order to check stability
(and evolution) of our serialization formats, it is useful to check
the sizes of the intermediate bytes.

Change-Id: I171894ba3791245a53fe39c2d3238d2341d8ffc4
Signed-off-by: Robert Varga <>
4 years agoSeparate out AbstractNormalizedNodeDataInput 14/84514/2
Robert Varga [Tue, 17 Sep 2019 21:35:14 +0000 (23:35 +0200)]
Separate out AbstractNormalizedNodeDataInput

As we are going to introduce a streaming format for Magnesium, which
uses very different token set, it is convenient to have a common
base class which does not have conotations about what the tokens are.

We provide a baseline counterpart to AbstractNormalizedNodeDataOutput
and base AbstractLithiumDataInput on it. The asymmetry of
writeYangInstanceIdentifier() is fixed by moving it to

Change-Id: I55d318349f2c89508f3834c8ca5b69e69b7171b1
Signed-off-by: Robert Varga <>
4 years agoAdd @SupressFBWarnings around Await.result() 31/84431/4
Robert Varga [Sat, 14 Sep 2019 06:27:19 +0000 (08:27 +0200)]
Add @SupressFBWarnings around Await.result()

Await.result() is throwing Exceptions we need to catch, raising
the ire of SpotBugs. Suppress that.

Change-Id: I1a880c86a8ab1759c5741dd27c8341d89452c7e8
Signed-off-by: Robert Varga <>
4 years agoMake sure we know the version we encountered 12/84512/1
Robert Varga [Wed, 18 Sep 2019 00:11:23 +0000 (02:11 +0200)]
Make sure we know the version we encountered

Hitting this exception is not providing sufficent detail as to
what the proposed version is. Add that information.

Change-Id: I44662e904774b636b351beb4c6b1f218e84f8ada
Signed-off-by: Robert Varga <>
4 years agoLost commit index when a snapshot is captured 21/83421/5
Tomas Cere [Tue, 6 Aug 2019 11:46:32 +0000 (13:46 +0200)]
Lost commit index when a snapshot is captured

When we have no journals and a snapshot is recovered and
immediately captured again, if there was no movement in the
datastore we loose the lastIndex and term from the previous snapshot.
We can safely reuse these from the previous snapshot when no
entry is present in the ReplicatedLog.

Change-Id: Iaeb71edc7ec865bec18dbcb436af76e592eea69d
Signed-off-by: Tomas Cere <>
Signed-off-by: Robert Varga <>
4 years agoOptimize anyxml output 44/84344/1
Robert Varga [Thu, 12 Sep 2019 15:33:59 +0000 (17:33 +0200)]
Optimize anyxml output

We do not need to instantiate a new factory every time we perform
encoding, keep the factory as a shared constant.

Change-Id: Ideff56076a1516784cd82f67438e5b740c4d3eeb
Signed-off-by: Robert Varga <>
4 years agoMove Lithium-specific logic from AbstractNormalizedNodeDataOutput 32/84332/5
Robert Varga [Wed, 11 Sep 2019 20:09:49 +0000 (22:09 +0200)]
Move Lithium-specific logic from AbstractNormalizedNodeDataOutput

The binding to TokenTypes and co. are really part of
AbstractLithiumDataOutput, as different serialization streams
versions can use completely different tokens and sizing.

Change-Id: I703c1da26ee2f99dee406e4de26db702beb05a06
Signed-off-by: Robert Varga <>
4 years agoRemove ensureHeaderWritten() from writeNode() 29/84329/2
Robert Varga [Wed, 11 Sep 2019 20:05:19 +0000 (22:05 +0200)]
Remove ensureHeaderWritten() from writeNode()

This updates documentation of AbstractNormalizedNodeDataOutput
to make it clear that the NormalizedNodeStreamWriter aspect of
it is an implementation detail and that the stream has been
header-initialized by the time any of those methods are called.

The clarification renders calling ensureHeaderWritten() from
writeNode() superfluous, as at that point it is guaranteed to
be a no-op.

Change-Id: I85a5c1304849fe5bc737fa51bd4b40ff2dd8c08c
Signed-off-by: Robert Varga <>
4 years agoReorganize AbstractNormalizedNodeDataOutput 27/84327/2
Robert Varga [Wed, 11 Sep 2019 15:44:47 +0000 (17:44 +0200)]
Reorganize AbstractNormalizedNodeDataOutput

This reorganizes AbstractNormalizedNodeDataOutput so that it
implements all NormalizedNodeDataOutput with final methods,
deferring implementation as need to abstract methods.

This way we have precisely one implementation of public methods,
making it easy to audit for correctness. Furthermore it eliminates
writeAugmentationIdentifier() override in NeonSR2 writer.

It also introduces writeQNameInternal(), which skips header checks,
as the call sites have already performed those checks.

Change-Id: Icb4a7e994c04e3d52bd561d99c7743dfdfd88738
Signed-off-by: Robert Varga <>
4 years agoSplit out AbstractLithiumDataInput 72/84272/5
Robert Varga [Sun, 8 Sep 2019 18:26:18 +0000 (20:26 +0200)]
Split out AbstractLithiumDataInput

This splits out AbstractLithiumDataInput and disconnects Lithium
and NeonSR2 reader implementations. This allows Neon SR2 to be loaded
without Lithium, thus reducing the number of actual possible
implementations at runtime.

Change-Id: I90863c342d7bbea4946fd27d25dfb822efb4ed6e
Signed-off-by: Robert Varga <>
4 years agoCleanup PathArgumentTypes 23/84323/2
Robert Varga [Wed, 11 Sep 2019 13:09:13 +0000 (15:09 +0200)]
Cleanup PathArgumentTypes

Do not use long qualifiers for individual types, preferring
importing them. Also drop public modifiers, as this class is
package-private. Finally import static checkArgument().

Change-Id: I267d0f5675911b2d9990e8f7c3fdd49265c10d8b
Signed-off-by: Robert Varga <>
4 years agoAbstractNormalizedNodeDataOutput fails to write out header 24/84324/1
Robert Varga [Wed, 11 Sep 2019 14:06:59 +0000 (16:06 +0200)]
AbstractNormalizedNodeDataOutput fails to write out header

In case AbstractNormalizedNodeDataOutput is used in a way, where
NormalizedNodeDataOutput.writePathArgument() is the first method
invoked, the stream header would not be output, leading to it
being unreadable.

Change-Id: I4ababb360129b2ef673d0546333599dd10fbdd9e
Signed-off-by: Robert Varga <>
4 years agoTighten AbstractLithiumDataOutput.writeString() 69/84269/2
Robert Varga [Sat, 7 Sep 2019 09:55:28 +0000 (11:55 +0200)]
Tighten AbstractLithiumDataOutput.writeString()

We can only ever pass nulls from Revision encoding, refactor to
make that clear (and enforced) in code. This more clearly defines
the context of TokenTypes.IS_NULL_VALUE and where it can be emitted.

Change-Id: I14045c9fc3eaf04b4bde858d322de45d3160de26
Signed-off-by: Robert Varga <>
4 years agoRemove NormalizedNodeOutputStreamWriter 68/84268/1
Robert Varga [Sat, 7 Sep 2019 09:22:39 +0000 (11:22 +0200)]
Remove NormalizedNodeOutputStreamWriter

This class is tied to NeonSR2 serialization format through subclassing,
where it really is a package-private detail. This leads to two concrete
classes being in existence for a particular format, which simply does
not make sense.

Eliminating this subclass allows us to make Neon SR2 final, thus arriving
and monomorphic or bimorphic invocation of all methods based on whether
we have seen Lithium (well, Oxygen) format at runtime.

Change-Id: I9e0463ec4879900e82cf757006dccfdbeb0d7297
Signed-off-by: Robert Varga <>
4 years agoDisconnect {Lithium,NeonSR2} implementations 67/84267/2
Robert Varga [Sat, 7 Sep 2019 07:54:32 +0000 (09:54 +0200)]
Disconnect {Lithium,NeonSR2} implementations

We will need to define two new serialization formats: one
sharing object value serialization (for NeonSR3) and one
disconnected from it (for Magnesium).

In order to maintain scalability with so many formats, we want
to avoid JIT considering implementations which were not referenced
at runtime. This should make CHA and method dispatch more

Since we are isolating things, we also pull ValueTypes lookups
and keep it in the same class for easier understanding.

Change-Id: I3db4e9633db4a75016e17ac31bb38452bcb9f42a
Signed-off-by: Robert Varga <>
4 years agoLower ValueTypes constant visibility 66/84266/1
Robert Varga [Sat, 7 Sep 2019 07:01:43 +0000 (09:01 +0200)]
Lower ValueTypes constant visibility

This is a package-private class, hence 'public' is superfluous.

Change-Id: I4dd03b3ff1ae0cc4cc11f823cbd9538e6932d6b3
Signed-off-by: Robert Varga <>
4 years agoFixup chunk offset movement on resend 17/83317/7
Tomas Cere [Tue, 30 Jul 2019 09:48:06 +0000 (11:48 +0200)]
Fixup chunk offset movement on resend

Since we are moving the offset backwards on failures, it doesnt
matter whether we have seen a failure or success when incrementing
offset. We always need to move it to indicate a chunk is in flight,
otherwise canSendNextChunk() will allow us to send the same chunk
again and again even when it is already inflight.

Change-Id: If5dfd1913bda4328c5dfab0869f2df697bbd3cfc
Signed-off-by: Tomas Cere <>
4 years agoClean up opendaylight-inventory model 10/84210/2
Robert Varga [Thu, 5 Sep 2019 06:52:10 +0000 (08:52 +0200)]
Clean up opendaylight-inventory model

This is just a formatting change, cleaning up descriptions and

Change-Id: I2757280371851e2586221b01de143e8b987ed1ad
Signed-off-by: Robert Varga <>
4 years agoRevert "Bug 3871: Deprecate opendaylight-inventory model." 09/84209/2
Robert Varga [Thu, 5 Sep 2019 06:34:00 +0000 (08:34 +0200)]
Revert "Bug 3871: Deprecate opendaylight-inventory model."

This reverts commit 973bde38d5f02b4e65f20363ef03d17587433a21,
as the migration is not coming along and deprecated constructs
are causing massive number of warnings, hiding more important

Change-Id: I1e3d27374ffba0e584f194d468cebcfa9cecfe81
Signed-off-by: Robert Varga <>
4 years agoFix checkstyle/spotbugs violations 64/84164/1
Robert Varga [Tue, 3 Sep 2019 07:34:01 +0000 (09:34 +0200)]
Fix checkstyle/spotbugs violations

Enabling checkstyle/spotbugs by default finds these, fix them up.

Change-Id: Ib407830f0a15cb4c04492f3c694fb463cdb30bad
Signed-off-by: Robert Varga <>
4 years agoCleanup ValueTypes lookup 36/84036/1
Robert Varga [Wed, 28 Aug 2019 12:56:04 +0000 (14:56 +0200)]
Cleanup ValueTypes lookup

Change type of TYPES to ImmutableMap, so we are binding to a concrete
class when looking up. Also clean up initialization to follow fluent
builder pattern.

Finally inline requireNonNull() call into the invocation, so that its
return value is not wasted.

Change-Id: I4487bd43232131ca548441f4c8901a8c5dc60efa
Signed-off-by: Robert Varga <>
4 years agoDeduplicate MapNode key leaf values 98/83998/9
Robert Varga [Mon, 26 Aug 2019 20:44:30 +0000 (22:44 +0200)]
Deduplicate MapNode key leaf values

When we are encountering a leaf node inside a MapEntryNode, it can
be a leaf corresponding to a key -- in which case its value is
already present in NodeIdentifierWithPredicates and we do want to
de-duplicate those objects.

Change-Id: I2ed65c311f9921aa77c9f23bd1f7681d7f11355a
Signed-off-by: Robert Varga <>
4 years agoMove common code to startNode() 08/84008/4
Robert Varga [Tue, 27 Aug 2019 09:34:34 +0000 (11:34 +0200)]
Move common code to startNode()

We are duplicating null checks in all entrypoints which lead to
startNode(). We can centralize these in startNode(), leading to
denser code and fixing Sonar warnings.

Change-Id: I3f2a0c53216179e9d25624a3e3139340eff551fa
Signed-off-by: Robert Varga <>
4 years agoRename SodiumNormalizedNode* classes 01/84001/2
Robert Varga [Mon, 26 Aug 2019 22:08:05 +0000 (00:08 +0200)]
Rename SodiumNormalizedNode* classes

These are implementing Neon SR2 serialization format, hence they
should be named as such.

Change-Id: Iaa509e3490b420b8005a2724105bf99b53d95cf2
Signed-off-by: Robert Varga <>
4 years agoDrop public modifier from NodeTypes 99/83999/2
Robert Varga [Mon, 26 Aug 2019 21:17:51 +0000 (23:17 +0200)]
Drop public modifier from NodeTypes

The class itself is not visible outside of the package, there is no
point to declare the constants as public.

Change-Id: I4136a71c538452aba33a5abb3337e0f8fc7e8da8
Signed-off-by: Robert Varga <>
4 years agoRestore javassist dependency for tests 08/83908/1
Stephen Kitt [Thu, 22 Aug 2019 14:02:54 +0000 (16:02 +0200)]
Restore javassist dependency for tests

It’s still used in BindingBrokerTestFactory.

Change-Id: Id162dcc1c45b776644888a7bd4d0c7f603f1216f
Signed-off-by: Stephen Kitt <>
4 years agoReduce reliance on Guava Fuction/Supplier 40/83840/4
Robert Varga [Wed, 21 Aug 2019 08:55:57 +0000 (10:55 +0200)]
Reduce reliance on Guava Fuction/Supplier

We have JDK8+ equivalents, use those instead.

Change-Id: I3796c993cc7c13d3a278e1ffdffce03801be21e7
Signed-off-by: Robert Varga <>
4 years agoFix modernization issues 32/83832/8
Robert Varga [Tue, 20 Aug 2019 19:06:59 +0000 (21:06 +0200)]
Fix modernization issues

This fixes issues pointed out by Modernizer, which mostly boil
down to using Objects.requireNonNull() and direct java.util

Change-Id: Id32530a6722cd101f96c23f6a745f91b2f09e2f9
Signed-off-by: Robert Varga <>