Robert Varga [Wed, 21 Aug 2019 08:55:57 +0000 (10:55 +0200)]
Reduce reliance on Guava Fuction/Supplier
We have JDK8+ equivalents, use those instead.
Change-Id: I3796c993cc7c13d3a278e1ffdffce03801be21e7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
45371f6048ab259f7ed536962bd081d1eb5ae2ef)
(cherry picked from commit
dc73c367b45e4e70c7b486dcf9e5ff17c3e02331)
Robert Varga [Tue, 20 Aug 2019 19:06:59 +0000 (21:06 +0200)]
Fix modernization issues
This fixes issues pointed out by Modernizer, which mostly boil
down to using Objects.requireNonNull() and direct java.util
collections.
Change-Id: Id32530a6722cd101f96c23f6a745f91b2f09e2f9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
3859df9beca8f13f1ff2b2744ed3470a1715bec3)
(cherry picked from commit
37cdbd3d1e388cae9aaae2558dd6f465c545937d)
Robert Varga [Wed, 18 Sep 2019 00:11:23 +0000 (02:11 +0200)]
Make sure we know the version we encountered
Hitting this exception is not providing sufficent detail as to
what the proposed version is. Add that information.
Change-Id: I44662e904774b636b351beb4c6b1f218e84f8ada
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
10006bbb875a6b90e14bda6b8c32d4d801dd228c)
(cherry picked from commit
5d9451d60b44d614c88222b32e057cea97a741ca)
Robert Varga [Wed, 11 Sep 2019 13:09:13 +0000 (15:09 +0200)]
Cleanup PathArgumentTypes
Do not use long qualifiers for individual types, preferring
importing them. Also drop public modifiers, as this class is
package-private. Finally import static checkArgument().
Change-Id: I267d0f5675911b2d9990e8f7c3fdd49265c10d8b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
183262480ae244c68ef056773259e9be751b31c1)
(cherry picked from commit
86aed4a9a300889bec436a6600e53a3a94854bd0)
Robert Varga [Sat, 7 Sep 2019 07:01:43 +0000 (09:01 +0200)]
Lower ValueTypes constant visibility
This is a package-private class, hence 'public' is superfluous.
Change-Id: I4dd03b3ff1ae0cc4cc11f823cbd9538e6932d6b3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
58e49354ba8af7a48a037519281fd96cc4165204)
(cherry picked from commit
44c2a672ae5753108cc9e68773121272275e7863)
Robert Varga [Mon, 26 Aug 2019 22:08:05 +0000 (00:08 +0200)]
Rename SodiumNormalizedNode* classes
These are implementing Neon SR2 serialization format, hence they
should be named as such.
Change-Id: Iaa509e3490b420b8005a2724105bf99b53d95cf2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
013e3c18d3c48fa1d6c62d410bc1a9d94d2fb560)
(cherry picked from commit
84c278d49ad65500adabd0ba5d35a73ceabf7748)
Robert Varga [Tue, 20 Aug 2019 22:31:55 +0000 (00:31 +0200)]
Migrate away from Throwables.propagate()
This method has been deprecated, move off of it.
Change-Id: Ida34fb165dad1c0cf1786c773325e077de53a4c2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
9c8276bd8fff1367dfb50400f71437602531c7a2)
(cherry picked from commit
d826e994fcccc85db9f1651a9b1b68c25d9a002c)
Robert Varga [Tue, 20 Aug 2019 20:19:57 +0000 (22:19 +0200)]
Refactor AbstractBrokerAwareActivator
This simplifies the implementation by encapsulating state.
Change-Id: I673e71f431026302974057af5933dba14e4c19c3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
1c992d01fb7be6dfb310ac82839a8f1876d890de)
(cherry picked from commit
c10add9e6704642b14c44da3c7d3ed1c7ec01073)
Robert Varga [Tue, 20 Aug 2019 20:56:26 +0000 (22:56 +0200)]
Move createProxy()
This is a simple internal method, inline it.
Change-Id: I5568f95f31c2487441fb19dbf3d33e02de5ada2a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
8a01ebe93fac21b1ae80dcfcc81c21543ec1a687)
(cherry picked from commit
04fe8d4a2937b74c4059467eb3f093349111897a)
Robert Varga [Tue, 20 Aug 2019 19:04:19 +0000 (21:04 +0200)]
Fixup checkstyle
Updated checkstyle does not like some our docs, fix them up.
Change-Id: I2d4f608fe9c44a56cdcac511cc1e2ae75c03b413
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
aa1c4a51361239aeaa064eb11eb935be194b6eeb)
(cherry picked from commit
240154300715823c76a1d964d63819a1294812f3)
Tomas Cere [Tue, 30 Jul 2019 09:48:06 +0000 (11:48 +0200)]
Fixup chunk offset movement on resend
Since we are moving the offset backwards on failures, it doesnt
matter whether we have seen a failure or success when incrementing
offset. We always need to move it to indicate a chunk is in flight,
otherwise canSendNextChunk() will allow us to send the same chunk
again and again even when it is already inflight.
Change-Id: If5dfd1913bda4328c5dfab0869f2df697bbd3cfc
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
e01deeaf7005438f702f9d8531ad8d04ce635db1)
Tomas Cere [Tue, 6 Aug 2019 11:46:32 +0000 (13:46 +0200)]
Lost commit index when a snapshot is captured
When we have no journals and a snapshot is recovered and
immediately captured again, if there was no movement in the
datastore we loose the lastIndex and term from the previous snapshot.
We can safely reuse these from the previous snapshot when no
entry is present in the ReplicatedLog.
Change-Id: Iaeb71edc7ec865bec18dbcb436af76e592eea69d
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
aad11d48fcace37e3365388f387e16fa67257a25)
Robert Varga [Wed, 21 Aug 2019 09:40:15 +0000 (11:40 +0200)]
Another round of checkstyle fixes
This fixes failures reported by upgraded checkstyle, as well as
some non-enforced violations.
Change-Id: I8b6a9667c6586bef528587d455f7a8597b4f483f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
3b9a2d4870110d14d7e2b6ea672023fd18328c8b)
Robert Varga [Wed, 28 Aug 2019 12:56:04 +0000 (14:56 +0200)]
Cleanup ValueTypes lookup
Change type of TYPES to ImmutableMap, so we are binding to a concrete
class when looking up. Also clean up initialization to follow fluent
builder pattern.
Finally inline requireNonNull() call into the invocation, so that its
return value is not wasted.
Change-Id: I4487bd43232131ca548441f4c8901a8c5dc60efa
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
b08cec09b7113965fbda5ef1757ca23d7588e5cc)
Robert Varga [Tue, 3 Sep 2019 07:34:01 +0000 (09:34 +0200)]
Fix checkstyle/spotbugs violations
Enabling checkstyle/spotbugs by default finds these, fix them up.
Change-Id: Ib407830f0a15cb4c04492f3c694fb463cdb30bad
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
842ae4586043f7a230ab701696e4e4f0b7ff49f4)
Robert Varga [Thu, 5 Sep 2019 06:52:10 +0000 (08:52 +0200)]
Clean up opendaylight-inventory model
This is just a formatting change, cleaning up descriptions and
comments.
Change-Id: I2757280371851e2586221b01de143e8b987ed1ad
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
f5f4cf3ffb6e5841a1398f365c4786d35659f5cf)
Robert Varga [Thu, 5 Sep 2019 06:34:00 +0000 (08:34 +0200)]
Revert "Bug 3871: Deprecate opendaylight-inventory model."
This reverts commit
973bde38d5f02b4e65f20363ef03d17587433a21,
as the migration is not coming along and deprecated constructs
are causing massive number of warnings, hiding more important
ones.
Change-Id: I1e3d27374ffba0e584f194d468cebcfa9cecfe81
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
0ff4645c53789c32f45e4587938ef2ff491d6e59)
Robert Varga [Mon, 26 Aug 2019 21:17:51 +0000 (23:17 +0200)]
Drop public modifier from NodeTypes
The class itself is not visible outside of the package, there is no
point to declare the constants as public.
Change-Id: I4136a71c538452aba33a5abb3337e0f8fc7e8da8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
372d56a7e856b994f07d6c68dbea15cefb5008df)
jenkins-releng [Sat, 31 Aug 2019 12:44:15 +0000 (12:44 +0000)]
Bump versions by x.y.(z+1)
Signed-off-by: jenkins-releng <jenkins-releng@opendaylight.org>
Change-Id: Ida80d53acca2d1532692bf231b68e05bed738dbe
Robert Varga [Tue, 20 Aug 2019 06:35:03 +0000 (08:35 +0200)]
Remove triemap-bom import
Triemap is provided by odlparent, remove the reference to it.
Change-Id: Ib37bc9957742a7e9f4e094248b6b8129a32ccd73
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Tomas Cere [Wed, 24 Jul 2019 10:40:29 +0000 (12:40 +0200)]
Do not log the whole DataTree at debug level
This needs to be at trace, as its polluting logs otherwise.
Change-Id: I6141a1793402e87ab49defe40007114b40d16335
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
cd648b7a1662577cd998670f5b39bc839e0f9070)
Robert Varga [Thu, 18 Jul 2019 13:34:16 +0000 (15:34 +0200)]
Bump mdsal to 3.0.10
This brings in latest fixes/improvements.
Change-Id: Ida49ef543f765fd8f828761d7804e5b51a93b1b3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 10 Jul 2019 09:54:38 +0000 (11:54 +0200)]
Raise EOS unsuccessful request reporting to error
When we fail to send a message to the backend we are entering an
inconsistent state, where our users think the candidate is registered,
but the backend knows nothing about it -- which warrants an ERROR
level output.
Also place a FIXME on operations which should be retried.
JIRA: CONTROLLER-1904
Change-Id: I0799afe2e0786468cdef5f80c1064cb08d3c1af0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
bdce894fa73714aa9f68eadad3524cfc94dc71d2)
Robert Varga [Wed, 10 Jul 2019 09:59:42 +0000 (11:59 +0200)]
Enforce RegisterListenerLocal arguments
The fields need to be initialized to non-null, make sure the object
does that (instead of its users).
Change-Id: I545421167bd33e06ea7221a0fbcc136083bfc80d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
bba56a2445d4ba38c7ecdee30a568bdc43d1d9de)
Tomas Cere [Thu, 11 Jul 2019 10:19:47 +0000 (12:19 +0200)]
Change remaining SnapshotSelectionCriteria to prioritize timestamp
These are also needed and were missed in the original run-through.
Change-Id: Icede0673c580f793ffe81ccecf728c15dc6e83ca
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
2f606b3ee1b8e47977320374e37f3d5c1d546f9b)
Robert Varga [Mon, 8 Jul 2019 13:38:03 +0000 (15:38 +0200)]
Do not depend on odl-mdsal-broker-local
odl-mdsal-trace depends on both odl-mdsal-broker (clustered) and
odl-mdsal-broker-local (non-clustered), which is not appropriate.
Remove the latter.
Change-Id: Ifdeb1c242004b6e365c133360c96af00c17c67c7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Tomas Cere [Mon, 3 Jun 2019 13:02:08 +0000 (15:02 +0200)]
Fix multiple snapshots present after journal removal
After journal removal the sequence number starts counting from 0
once again. Therefore we can have multiple snapshots present and
snapshots with higher sequence number always take priority for loading
even when a newer snapshot is present.
Change this up in 2 ways:
1. disregard sequence number while deleting snapshots(delete all
snapshots with an older timestamp)
2. while loading snapshots prioritize timestamp rather than seqNr.
Also simplify LocalSnapshotStore.compare().
Change-Id: I205ea0ddf48d73b0a09297a1ce4e9fd514531993
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
609076e66a0298d0d3c912ade66e813a464c4c8a)
Robert Varga [Mon, 1 Jul 2019 06:50:24 +0000 (08:50 +0200)]
Add transaction debugs
This adds a few debug messages allowing logs to be introspected
to see the timing of readying/applying transactions.
Change-Id: I1dde184023a034c6a0fa30e9a3898ab65db2decc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
c7fb53c205eddf3324a686b024e600480410fdb1)
Robert Varga [Tue, 4 Jun 2019 17:51:43 +0000 (19:51 +0200)]
Take advantage of AugmentationIdentifier.create()
When we are reading an augmentation identifier for the first
time we can optimize the number of identifiers by consulting
the global weak cache.
JIRA: CONTROLLER-1898
Change-Id: I5cd2d87e0a7c3f01357beb1b097d4a16a6eda3e0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
7c06dc0a88d544a0096ecf55e95680748f949b0f)
Robert Varga [Mon, 1 Jul 2019 06:52:27 +0000 (08:52 +0200)]
Bump mdsal to 3.0.9
This picks up latest fixes and improvements.
Change-Id: I7bd0b8375ab88eac9fcfee6eebed6d05dd60ff29
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Mon, 1 Jul 2019 06:51:57 +0000 (08:51 +0200)]
Bump yangtools to 2.1.11
This picks up latest fixes.
Change-Id: I8cfc4fab605d31a639eb5282e0a7d66124a6dcfe
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Mon, 1 Jul 2019 06:51:29 +0000 (08:51 +0200)]
Bump odlparent to 4.0.11
This adopts latest upgrades.
Change-Id: I233507906a8a4c4e4ec433839c7c5c4dbb0efe7e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Mon, 1 Jul 2019 12:09:55 +0000 (14:09 +0200)]
Migrate use of AbstractDOMStoreTransaction
This utility class has been migrated to mdsal, use the instance
from there.
Change-Id: I9aad875f631903d4ce9dfb2779dfce6f8883a50c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Fri, 28 Jun 2019 11:37:50 +0000 (13:37 +0200)]
Rename SODIUM versions to NEON_SR2
We are shipping these improvements in Neon SR2, which means we
should reserve this version not as Sodium.
Change-Id: If5caa0dd7510c6e1ffe3c483c74c456798ca6bb3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 29 May 2019 16:23:04 +0000 (18:23 +0200)]
Bump persisted PayloadVersion
Since the NormalizedNodeStream format has changed, shard persisted
state is affected.
This patch bumps PayloadVersion and applies that bump to snapshots
and CommitTransactionPayload.
On recovery, a snapshot's need to migrate is reflected in its state,
and it is examined just as MigratedSerializables are.
CommitTransactionPayload is not examined, as understanding its
stream version would require deserializing at least its header.
JIRA: CONTROLLER-1888
Change-Id: I678527be4487ee1729123ba8b9dcd2269e6cf262
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
2658d126f67512fcaac684d2ccb7197fe12f61f4)
Robert Varga [Wed, 29 May 2019 14:59:01 +0000 (16:59 +0200)]
Bump datastore version
This bumps the ask-based protocol datastore version, so that
SODIUM_VERSION is using Sodium NormalizedNode{Input,Output}.
JIRA: CONTROLLER-1888
Change-Id: Ib4ad1c27566179db958ad7776f81417ab99a78cf
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
9d6b18257717db4f6f722422ee3224154150cbf7)
Tomas Cere [Tue, 25 Jun 2019 12:16:50 +0000 (14:16 +0200)]
Reset replyReceivedForOffset correctly
replyReceivedForOffset needs to be set to the initial -1
otherwise we can have multiple sendSnapshotChunk()'s called
before receiving any replies. Which would fail due to the already
running stopwatch.
Change-Id: Ie708f0c2454a87f77cba97e4fed8a9ffc866b624
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
78282f605d76598e3b0e30cfc660a3a6c38ace48)
Tomas Cere [Tue, 25 Jun 2019 08:45:15 +0000 (10:45 +0200)]
Failed chunks should move offset backwards
If a chunk has failed it needs to move the offset backwards,
otherwise this would misalign the stream, or make it possible
to call sendNextChunk() right after resend which could be
seen as the chunkTimer attempted to be started while already running.
Change-Id: I6d44195ae9bcb23075ffea9c4f1d1ab1d0540c93
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
6d4d0642f80555c2b5c8d2ff87681b600a22b4a0)
Robert Varga [Tue, 18 Jun 2019 15:39:37 +0000 (17:39 +0200)]
Fix IMDS test synchronization
Before we close the DTCL we must make sure we flush all changes,
otherwise we could end up losing changes simply because we are no
longer interested in them.
JIRA: MDSAL-429
Change-Id: I6df384c375ea1692a10c9f460d3dceb626468061
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Tomas Cere [Tue, 4 Jun 2019 09:17:10 +0000 (11:17 +0200)]
Reset snapshot progress on IOExceptions
Also log the whole LeaderInstallSnapshotState of failures.
Change-Id: I33b1c38f2277f08acbe77bcb0c16d2dfe41d8851
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
a7da626a1de6eaa413cb33211d6aaccaaf1d57ab)
Robert Varga [Mon, 3 Jun 2019 19:37:43 +0000 (21:37 +0200)]
Remove use of thread-local input
All callers are only called from top-level constructs, thus
not having a relationship with the context in which they are
deserialized.
JIRA: CONTROLLER-1888
Change-Id: I856f9109480fb83475a20ce4430d02a1dd055522
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
728a8db811f7a257a0b69243cbf76bb5ccf69658)
Robert Varga [Tue, 11 Jun 2019 13:42:34 +0000 (15:42 +0200)]
Send commitIndex updates to followers as soon as possible
When the commitIndex gets updated when we reach consensus, we should
propagate this fact to the sender peer. Otherwise data tree change
listeners on peers may be unnecessarily delayed, as they end up
waiting for heartbeat interval (or non-batched entries) to be propagated
out.
The results in observably-better behaviour between leader and follower,
as the leader is sending out notifications of consensus, without
delayling for heartbeat -- which is reflected in the changes to
LeaderTest.
Furthermore, DelayedMessagesElectionScenarioTest is detecting the change,
as two members may achieve asynchronous consensus -- hence the leader
would be sending two messages (request to persist, commitIndex) to push
the consensus out. This exposes a race in the test, where we do not know
the order between TimeoutNow and AppendEntries. If AppendEntries is
processed by member 2 after TimeoutNow, leader will know about the new
term due to AppendEntriesReply we send out. The cluster will still
converge, but the assert for leader will be different if it manages to
process the message. Fix this by isolating member 2.
JIRA: CONTROLLER-1900
Change-Id: I695ef25c7a4cf8799c9c5e04c2c33fbf3e2f21df
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
c2b0e92d2c3ba05abf0fefb24413c68fd1c56bea)
Robert Varga [Fri, 7 Jun 2019 22:44:58 +0000 (00:44 +0200)]
Simplify recoveryTime computation
Stopwatch.stop() returns self, take advantage of that to shorted
the code to compute the formatting string. Also drop toString()
call, as it is implied by string concat.
Change-Id: I4fa645e42fc38b5f9f0775d986fd17aa693725b1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
49f3024ee0a621729e31279f8b2d46051fc003f1)
Robert Varga [Mon, 3 Jun 2019 19:12:44 +0000 (21:12 +0200)]
Remove use of thread-local output
All callers can be safely migrated to using a non-shared writer,
as they all are expected to be apex implementations.
JIRA: CONTROLLER-1888
Change-Id: I87cfa1d3ec415b79c40c29a91ccbd8da2869b27a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
6a32d897d2dcb4fa54977b3b2defe76dc0a5d5e2)
Robert Varga [Mon, 3 Jun 2019 17:43:22 +0000 (19:43 +0200)]
Switch CompositeModification to bypass thread-local streams
Thread-local streams have little control over the version being
used, as the stream is allocated all over the place.
This model cannot function without explicit setup, which is used
by CompositeModification. This patch migrates CompositeModification
to use a new NormalizedNodeDataInput-based interface, so
thread-locals become obsolete.
JIRA: CONTROLLER-1888
Change-Id: I1111be5d6fc8c0129fb72232c3e713004068f152
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
b12758e87e3da8b71b574614f8829e5fcbce11e8)
Robert Varga [Mon, 3 Jun 2019 16:56:44 +0000 (18:56 +0200)]
Make Modification/Externalizable interface visible
Finding direct callers of these methods is not easy, override
them to get exact call sites. This will help us with versioning
of the output stream.
JIRA: CONTROLLER-1888
Change-Id: Idc9bb036fac786b4480c098bd592e272d5a4c0ad
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
a5e45c08c64f450aef9f3bf2d1f98714d01db1ed)
Robert Varga [Wed, 29 May 2019 15:33:08 +0000 (17:33 +0200)]
Bump cds-access-api ABIVersion
In order to take advantage of improvements to NormalizedNode stream,
we need to declare a new ABI version, which will actually use
the new stream version.
JIRA: CONTROLLER-1888
Change-Id: I458a24586888e0b2fcc1c636e16d0b8e9e6e681f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
b998131396132a7630f271f9baea225b45fde789)
Robert Varga [Thu, 6 Jun 2019 17:36:06 +0000 (19:36 +0200)]
Add QNameModule coding
Coding on QNames is effective when there are a lot of same QNames,
but there are cases where QNames themeselves are not reused --
for example YangInstanceIdentifiers and small read subtrees.
These typically end up using different QNames from a small set
of modules, hence coding modules actually helps them quite a bit.
This patch adds a namespace/revision cache, so that encoding such
QNames is also efficient.
JIRA: CONTROLLER-1898
Change-Id: I6cd0885ef605e4a1bf7b768ffe215e9eb3e2fbc5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
f746b92d9aa9495a13ce92c01c5c1ae3228bb662)
Robert Varga [Wed, 29 May 2019 10:10:52 +0000 (12:10 +0200)]
Optimize NodeIdentifier reading
NodeIdentifiers are just an alias for QName, which is heavily
reused. This patch adds a secondary cache for interpreting QNames
as NodeIdentifiers, with instance reuse.
This allows us to reduce the number of NodeIdentifier instances,
while not relying on NodeIdentifier.create() for the common lookup
case.
JIRA: CONTROLLER-1898
Change-Id: Ifa0c5d572f7d39da49707529c7ddd9c9f36f2dab
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
5f540931c1f8972b59e51556b6cba14b2c8f3f40)
Tomas Cere [Tue, 4 Jun 2019 13:32:20 +0000 (15:32 +0200)]
Add buffering to LocalSnapshotStore
These improve snapshot save/load load times by around 15-20%
on my environment.
Change-Id: I67236f7e97f519993031d462c947588004857c33
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
Robert Varga [Tue, 4 Jun 2019 17:54:22 +0000 (19:54 +0200)]
Optimize readQNameSet()
We are providing a bad hint to HashSet, as we are not taking
into consideration its load factor. Use utility methods from Guava
to get a properly-sized set.
JIRA: CONTROLLER-1898
Change-Id: I568be3040b924a8ed77aa0224ab45673104c4664
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
11666b718f70c7846cd64ebf949a4e91d84e0e65)
Robert Varga [Thu, 23 May 2019 16:57:00 +0000 (18:57 +0200)]
Remove ask-based close transaction chain persistence
ask-based protocol has tracking disabled when encountered, hence
it does not make sense to persist anything.
JIRA: CONTROLLER-1628
Change-Id: I1ec87a9905aa76f3e2a57ae2440c8c693ff60848
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
8182b2647acc326e66da9f66fbe69e16435567d6)
Robert Varga [Wed, 29 May 2019 09:27:21 +0000 (11:27 +0200)]
Add support for coded QNames/AugmentationIdentifiers
Profiling is showing we are spending about 46% in readQName(),
which further breaks down to 30% in readCodedString() and 61%
in QNameFactory.create().
Out of 1.7M invocations of readQName(), we end up loading only
508 QNames -- which is understandable, as the number of QNames is
bound by the SchemaContext contents. This also applies to
AugmentationIdentifiers.
This patch defines a new streaming format version, which uses
the same linear coding approach we take for Strings, taking
hash-based lookups out of the hot path -- bringing reconstruction
time down from 36 to 19-24 seconds.
It has also the effect of reducing the snapshot size, in our testing
bringing a 388.8MiB snapshot down to 210.6MiB.
JIRA: CONTROLLER-1898
Change-Id: Ibbc96c286d52b3261b04dd7de02e13bc6b44e803
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
657b0b025a92f9d0ad6647d79950071031d7b0b4)
Robert Varga [Tue, 28 May 2019 13:28:55 +0000 (15:28 +0200)]
Separate lazy-versioned NormalizedNodeDataInput
NormalizedNodeInputStreamReader is really only one implementation
of the streaming format, which happens to be bound to LITHIUM_VERSION.
This effectively means NormalizedNodeInputOutput can give out either
a version-bound reader or an unbound reader -- which we facilitate by
creating VersionedNormalizedNodeDataInput which is a simple forwarder
with lazily-initialized delegate.
Since this increases the number of (NormalizedNode)DataInput
implementations, we share as much code as possible through the use
of forwarding implementations.
JIRA: CONTROLLER-1898
Change-Id: I28713346730ed5bf7f912bf8afa5d8722b202035
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
69e914b3889a82fa2f7daa8d55e1aa0de23fb454)
Robert Varga [Mon, 13 May 2019 05:53:27 +0000 (07:53 +0200)]
Centralize read{Augmentation,Node}Identifier() methods
We are using the same construct in multiple places, make sure we
centralize it and mark a FIXME for some efficiency increase.
Change-Id: Ie65089a1a7bb6a2752887498d354a61cb32f0e40
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
60fa71ddf45a8ffb37b69eb726305bdd0697185a)
Robert Varga [Tue, 28 May 2019 11:14:34 +0000 (13:14 +0200)]
Do not format QNames to string on input
Profiling has revelead the creation of String for QNameFactory
lookup is taking around 19% of overall cost of
NormalizedNodeInputStreamReader.readQName().
Rather than using concatenation, use a dedicated Key object, which
holds the separated-out localname/namespace/revision strings,
thus allocating a small object instead of a full-blown String.
This improves snapshot deserialization perfomance on a 350MiB
snapshot by about 16%, also resulting in lower memory pressure.
JIRA: CONTROLLER-1897
Change-Id: If3e9e4b332b0a5acbe33aa10fca189bbc7da8c81
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
a69604185f71923bdfc16d2d056490fff9a8d90e)
Robert Varga [Tue, 28 May 2019 10:42:02 +0000 (12:42 +0200)]
Update QNameFactory caching
This eliminates the use of soft references in favor of weak ones,
so that we do not rely on GC heuristics for retention.
We also make the maximum cache size runtime-configurable via
org.opendaylight.controller.cluster.datastore.node.utils.qname-cache.max-size
property.
Finally we use QName.intern() to defer to the global QName cache,
resulting in better QName instance sharing.
JIRA: CONTROLLER-1897
Change-Id: Ice3596d23b96c4ecfc5775cb21968233b042a984
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
dc54b87a6b2fb11d233b8294b684268f4d5161de)
Tomas Cere [Thu, 23 May 2019 08:04:16 +0000 (10:04 +0200)]
Use Files.delete() in LocalSnapshotStore
If delete fails, it needs to be logged so change this up.
Change-Id: If4051cb3e5b83dee51f919fc1713cc17f032597e
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
d8ef44c67accea4006489e4cbb6c8f0d4d428b6a)
Tomas Cere [Thu, 16 May 2019 12:12:01 +0000 (14:12 +0200)]
Do not enque purges for ask based protocol
With metadata disabled for ask based protocol, there is no need
to purge transaction on the backend and pollute the journal.
Tell based protocol has purges handled by the fronted so it
should be safe for us to remove them on the backend for ask
protocol.
JIRA: CONTROLLER-1879
Change-Id: Ia5c9470782bb4d59b78bbaa605ee5d36c7fb1644
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
4944f54d4e1fc24404d55e4ab74b6de212844dcd)
Tomas Cere [Tue, 14 May 2019 10:56:04 +0000 (12:56 +0200)]
Remove PersistAbortTransactionPayload
With the metadata tracking disabled for ask based protocol
there is no need to track aborts for read only transactions
on the backend.
JIRA: CONTROLLER-1879
Change-Id: I189ae3231bb2f3c0eaa0bbe21a14342446708c5f
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
7011afc1380ca25ad3de4cc8865f12fe3f5e09cf)
Robert Varga [Thu, 16 May 2019 13:54:29 +0000 (15:54 +0200)]
Disable transaction tracking for ask-based protocol
When we encounter an ask-based protocol message on a leader,
we persist a payload to make sure no state tracking occurs
for that client.
JIRA: CONTROLLER-1879
Change-Id: I3d12a06ce9e5b65ada5b59bde9f9b5486e5e4ef7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
c413251d63152cc70fce767de0a3776f16b9a27d)
Robert Varga [Fri, 10 May 2019 10:25:31 +0000 (12:25 +0200)]
Allow transaction tracking to be disabled
Ask-based protocol does not need tracking of transactions and
histories, as it is not retransmitting requests. It also does
not inform backend about purely-local aborted transactions
(read-write and read-only), which leads to transaction tracking
rangesets having holes where those IDs are used.
This adds the prerequisite handling of disabling from the leader
without adding the actual mechanics.
JIRA: CONTROLLER-1879
Change-Id: I133e7688b492336937f394f0f6c3f080a05a820f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
33ade248cf6070455349fe343c0d0fd48d274717)
Robert Varga [Tue, 7 May 2019 17:42:26 +0000 (19:42 +0200)]
Poison entries outside of main lock
Poisoning entries may involve reaction from their callbacks, which
can attempt to circle back through connections.
Make sure we poison them outside of lock context, so that any
callbacks end up seeing a poisoned connection, but without the lock
being held -- hence the locks can be acquired in-order.
JIRA: CONTROLLER-1893
Change-Id: I26551d052307812e76f3e45024a77dbb83312b17
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
e983d61d93fe2da50f9c4112fa28c7fe4ee5ffef)
Robert Varga [Wed, 15 May 2019 07:58:52 +0000 (09:58 +0200)]
Derive config-artifacts from odlparent-lite
This is to fix failures seen on promoting to Central.
Change-Id: I6204d68c5144437e5af50bb5f2cb29431841d394
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
e85651156b6687818741f4170efc80a55ddb3efb)
jenkins-releng [Wed, 15 May 2019 00:05:51 +0000 (00:05 +0000)]
Bump versions by x.y.(z+1)
Change-Id: Ib788879087c63dfe5c9e66707ec3e8ba302727b8
Signed-off-by: jenkins-releng <jenkins-releng@opendaylight.org>
Robert Varga [Wed, 8 May 2019 10:28:24 +0000 (12:28 +0200)]
Allow SnapshotBackedReadTransaction customization
In some specific cases we need to customize abort-like handling,
which is already implemented for write-like transactions, but is
not present for read-only transaction.
This patch adds the capability to attach a close() handler and
makes sure AbstractSnapshotBackedTransactionChain takes advantage
of it.
JIRA: CONTROLLER-1879
Change-Id: Ic7027956556b5dd25120ee81613a6151e5dbc501
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 7 May 2019 14:23:19 +0000 (16:23 +0200)]
Enqueue purge payload directly
When we are sending abort+purge, make sure we propagate it out
as soon as possible.
Change-Id: I2b63aa20867a8b99b779115f0bb448c0e8ac9a22
JIRA: CONTROLLER-1879
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
04547138454b730148250f38979436cb839b9892)
Robert Varga [Tue, 7 May 2019 11:22:10 +0000 (13:22 +0200)]
Make sure we purge transaction on abort
In case of read-only transactions it is not sufficient to just
persist the abort payload, we need to also purge the transaction.
Change-Id: I8761040d074743b5a1c5eebbecc6449c30abe47e
JIRA: CONTROLLER-1879
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
bc301a153a07be7c23327e85179974422624c80a)
Robert Varga [Fri, 3 May 2019 11:01:14 +0000 (13:01 +0200)]
Improve segmented journal JMX beans
This patch updates metric naming to properly tie them to individual
journals -- so that the statistics are actually useful.
JIRA: CONTROLLER-1884
Change-Id: I6f4f398894860566e234562d731462535b62801e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
bb366ba1be306e7678d81a116c1415876fa9fc75)
Robert Varga [Tue, 2 Apr 2019 21:40:20 +0000 (23:40 +0200)]
Integrate SegmentedJournalActor with metrics/JMX
This adds the wiring needed to expose segmented journal metrics
through JMX.
JIRA: CONTROLLER-1884
Change-Id: I42f0b29985410a34ec91639ce8be2903bbe7dd5b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
096e92fc0595f6f87db4b681b0fe9cdeeb365164)
Robert Varga [Fri, 3 May 2019 10:48:34 +0000 (12:48 +0200)]
Change segmented journal naming
Using Base64 encoding is not quite nice for debugging purposes,
use URLEncoder, so that we are consistent with snapshot store
JIRA: CONTROLLER-1884
Change-Id: I55f7bef79b7d5c14ab10180444770fedcc23eab9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
849e994c06fe5fd78bf72bddbf07e7d32576b84b)
Maros Marsalek [Fri, 3 May 2019 12:04:56 +0000 (14:04 +0200)]
Fix memory leak in BA mount service
The cache used in BA mount service uses weakKeys() so the keys (DOM mountpoint)
could be GCed when no ono, except cache, referenced them).
However the values in the cache (BA mountpoint) also kept a reference to
the key, so the key was never weakly reachable and thus never GCed
resulting in a memory leak.
Which is especially visible in case of frequent reconnects of netconf
mountpoints where after every reconnect a new DOM mountpoint is created,
requiring a new BA mountpoint creating new entry in the cache of BA mount service...
Added also a simple test verifying proper cache cleanup
Change-Id: I9c109dd6d499d6185842d6a1ad5a59d75565af5c
Signed-off-by: Maros Marsalek <mmarsalek@frinx.io>
(cherry picked from commit
225e9ce237996d321f420291ff6ba59c48a70d9f)
Tomas Cere [Tue, 30 Apr 2019 08:49:02 +0000 (10:49 +0200)]
Reset snapshot progress after timeout has been hit
Add a stopwatch to LeaderInstallSnapshotState that allows
us to keep track whether a follower took too much time to respond
to InstallSnapshot for whatever reason allowing us to retry
chunks that time out.
JIRA: CONTROLLER-1891
Change-Id: Id443a4cc7a069ad4d9982d537d9a8d82b845ac35
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
f33beecf2a10955a9219757529ba3017079816cc)
Robert Varga [Wed, 24 Apr 2019 12:43:18 +0000 (14:43 +0200)]
Bump mdsal to 3.0.8
This brings in latest fixes and enhancements.
Change-Id: I7dcb7e9fa0fc0df8cf0af30c50b6c3719b7e46e7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 24 Apr 2019 12:42:28 +0000 (14:42 +0200)]
Bump yangtools to 2.1.10
This brings in the latest fixes and improvements.
Change-Id: I5ad3f25fcbae64f264da5b0da05c7eee5451d611
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 24 Apr 2019 12:41:35 +0000 (14:41 +0200)]
Bump odlparent to 4.0.10
This brings in updates across the board.
Change-Id: I66adb970ae2c9117ad46ab288668f4e92cba109b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Tomas Cere [Fri, 26 Apr 2019 10:37:38 +0000 (12:37 +0200)]
Harden ShardCommitCoordinator to survive EOS write retries
EntityOwnershipShard can break the usual shard rule of only one 3PC
running at one time, causing the retrying write attempting to modify
an already sealed modification.
Change-Id: Iaa56b68a30f9191f7aa14428f39d4d7524aeb09d
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
bb04f62f467cc6e6f0922ba8957b09715707940d)
Tomas Cere [Tue, 9 Apr 2019 08:46:18 +0000 (10:46 +0200)]
Do not break actor containment
During async persistence the actor containment is broken which
results in applyState being called directly from the persistence actor.
This means that the ClientRequestTracker might be missing an entry
for the transaction and the transaction being applied as a foreign candidate
in ShardDataTree.
If this happens there will be a transaction stuck in COMMIT_PENDING state,
blocking any further progress in the shard until its restarted.
JIRA: CONTROLLER-1890
Change-Id: I944b233d13103df08b68baeaf3907c064d1d526e
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
7daddd72031b33ed686abe18a0813e41263aac8d)
Robert Varga [Thu, 28 Mar 2019 18:54:43 +0000 (19:54 +0100)]
Fix checkstyle in toaster-it
Simple whitespace cleanup to keep checkstyle happy.
Change-Id: Id51f7b87b632cce92c29cbae8794662fe6feb250
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
4360becd2b948480f8246f2e584f0c92a9bfba4e)
Robert Varga [Thu, 28 Mar 2019 16:55:17 +0000 (17:55 +0100)]
Fix transaction mocking
We should not confuse FluentFutures and CheckedFutures, as the latter
is deprecated and with Guava 27 they are not compatible anyway.
Change-Id: I7489b4470df6fa435aab190b6e38eaf154522fe7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
a85197fcfedf2894b98439ea0ad66b3f5a0eb00b)
Robert Varga [Thu, 28 Mar 2019 16:26:24 +0000 (17:26 +0100)]
Migrate to use openCursor()
createCursor() has been deprecated long time ago, use its replacement
instead.
Change-Id: Id8862470ca4ad8c9fdec5c05cbac044c6715e153
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
4db06286a59f00d8cda6c952d5704e89dac9268c)
Robert Varga [Thu, 28 Mar 2019 13:32:44 +0000 (14:32 +0100)]
Fix logging messages
Upgraded spotbugs is finding more violations, fix them up.
Change-Id: I8956c5b7a6eef38286f4c2b6fb07ca6a775e6965
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
90f03193700920e656596d2a2257802cb22a90c7)
Robert Varga [Thu, 28 Mar 2019 12:01:48 +0000 (13:01 +0100)]
Use ReusableNormalizedNodePruner in PruningDataTreeModification
Rather than instantiating a pruner for each invocation, keep an
instance around for reuse. In order to facilitate free-standing
modifications, we also keep a reference to the DataSchemaContextTree
bound our current SchemaContext.
JIRA: CONTROLLER-1887
Change-Id: I5d310b12feb1ff46dc2af0e67294a925d48a4957
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
8f30cbc0ba2676e883faadba10f757881983b2d0)
Robert Varga [Thu, 28 Mar 2019 11:46:34 +0000 (12:46 +0100)]
Add ReusableNormalizedNodePruner
This version of NormalizedNodePruner is reusable across invocations,
and improving CPU and memory efficiency in bulk operations due to
not needing to allocate stack nor lookup DataSchemaContextTree.
JIRA: CONTROLLER-1887
Change-Id: I1798f5f6255cd23702bf3792c1f4f5149f92d208
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
7bef3c9e3f411d3e4abdb5ff43a618537cc9c936)
Robert Varga [Thu, 28 Mar 2019 10:22:55 +0000 (11:22 +0100)]
Move checkNotSealed() to addBuilder()
addBuilder() is called after previous check for seal from all callers,
who do not have any side-effects aside allocating arguments to
addBuilder().
Move the call to checkNotSealed() to addBuilder(), making the code
more straightforward and possibly preventing bugs from creeping in.
JIRA: CONTROLLER-1887
Change-Id: I127385378957deb7f38f1203870a0e52cd630644
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
35fe9a2bb63d451d820707f5142349539334a7ad)
Robert Varga [Thu, 28 Mar 2019 10:16:27 +0000 (11:16 +0100)]
Remove NormalizedNodePruner.findSchemaNodeForNodePath()
DataSchemaContextTree.findChild() provides equivalent functionality,
so use that instead of brewing our own.
JIRA: CONTROLLER-1887
Change-Id: Ia55ac99ce8fa37daf06ce0e7e1615f1bd0a07123
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
b62499730040256c5be1539f409f4a98b557099e)
Robert Varga [Thu, 28 Mar 2019 09:58:49 +0000 (10:58 +0100)]
Do not use Optional in NormalizedNodeWrapper
Use nullable schema instead, as the caller is package-private and
ends up unpacking the return anyway. This simplifies
NormalizedNodePruner a bit.
JIRA: CONTROLLER-1887
Change-Id: I395dd8266bd7198f66890c4ae3633ecda9281726
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
c4a5948ad86ce0f982d3f00cb7d300c9244b123c)
Robert Varga [Thu, 28 Mar 2019 07:48:02 +0000 (08:48 +0100)]
Hide NormalizedNodeBuilderWrapper
This class is not instantiable outside of the package is it does
not leak from NormalizedNodePruner -- therefore it cannot be used
by anyone else. Make sure it's package-private and clean it up
a bit.
JIRA: CONTROLLER-1887
Change-Id: I17911508273d68c2795d38e2156c9407522f55fe
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
8fe9ec767ec28d41f55e9714ca69fe08e64f71f0)
Robert Varga [Wed, 27 Mar 2019 22:51:57 +0000 (23:51 +0100)]
Remove NormalizedNodePruner.SimpleStack
This custom implementation is highly inefficient, because:
1) peek/pop operations perform get(int) and remove(int) operations
on ArrayList, which are highly inefficient O(N)
2) it allocates an object for each entry
Replace this class with a stock Deque, which takes care of the first
problem, as push/pop/peek are amortized O(1).
For implementation use an ArrayDeque, which takes care of the second
problem, as stack entries are reused.
JIRA: CONTROLLER-1887
Change-Id: Ie59e2fb47f841e4e5fc08c3ab7e164ad0cb368ad
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
8e3fb7b97f23fe0aba388b295fbe45d67d21e91c)
Robert Varga [Wed, 27 Mar 2019 22:45:15 +0000 (23:45 +0100)]
Clean up NormalizedNodePruner
- shorten YangInstanceIdentifier.PathArgument references
- do not declare runtime exceptions as thrown
- use a space after comma
JIRA: CONTROLLER-1887
Change-Id: Id0f09d2cd72bc619331c2d16e8f184db784c9976
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
f7d34eae1aa1938b98a05e8cf4b107b9933a5be5)
Robert Varga [Fri, 22 Mar 2019 19:32:32 +0000 (20:32 +0100)]
Update javadoc links
Our javadoc links are pointing to deprecated Guava methods, update
them to point to their equivalents, as they are going away in our
next upgrade.
Change-Id: Ia82471897ba860f38be45219676b44f3d55d977c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
0f69f9e83ca8602d7ad7646ff4e146f802c534c1)
Robert Varga [Fri, 22 Mar 2019 19:40:03 +0000 (20:40 +0100)]
Fix test mocking
MD-SAL store APIs require FluentFuture, not CheckedFuture, fix that
up.
Change-Id: I2222c6a56f8b7a1bcc6569ba84e01ce3b4904a51
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
04725a05b665b936cd6483c58079deb87b61c465)
Stephen Kitt [Mon, 26 Feb 2018 12:02:44 +0000 (13:02 +0100)]
Add missing license headers
Change-Id: Ie8405e6f1de698c958d347cfb53380a111d6a9ef
Signed-off-by: Stephen Kitt <skitt@redhat.com>
(cherry picked from commit
7b8631eb9a762dafe40bd099ceab14b3f5a3397e)
Robert Varga [Fri, 15 Mar 2019 11:33:05 +0000 (12:33 +0100)]
Use a simple list for string code table
Since the codes are assigned in a linear fashion, we can use
a simple ArrayList to track assignes codes, preventing the need
for boxing integer during population and lookups. Lookups also
end up requiring an array offset lookup rather than needing
to go through multiple indirections.
While this can in theory lead to a large array being allocated,
the set of strings we keep in the table is limited by the
SchemaContext and thus will be typically capped ad couple of
thousand.
The cost of growing the lookup should not be significantly
higher, even if ArrayList grows slower:
- 10, 16, 25, 38, 58, 88
versus HashMap (with .75 load factor)
- 12, 24, 48, 96, 192, 384
each resize operation is significantly faster.
Change-Id: I00c7a2e0985dbf901b6857a16424bf157e5af39d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
02a9d36756493691fda2bdd36a1c243ebc5e7cca)
Robert Varga [Fri, 15 Mar 2019 11:20:24 +0000 (12:20 +0100)]
Improve NormalizedNodeInputStreamReader defensiveness
If face of corrupted streams, for example due to CONTROLLER-1752,
we can end up interpreting unknown bytes as null strings, potentially
corrupting data.
This patch uses explicit type dispatch and detects wrong string
value types, as well as rejecting unknown string value codes.
Change-Id: Ic2e7887bc1d32a1dc7a364d1bff195752edd762d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
bfed3c0a9e208b7fd45d6a853f9c63ef95fde00e)
Tomas Cere [Tue, 12 Mar 2019 09:54:47 +0000 (10:54 +0100)]
Fix segmented journal replay
lastDelete was getting set with the incorrect value from the delete journal.
The actual written value from the delete journal needs to be used
instead of the index.
dataJournal compact also needs to be compacted upto lastDelete + 1 since
deleteUpTo is inclusive and compact keeps this value around.
Change-Id: I4a678be67fd1ad09c57273ef0fc3b7398a7d714f
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit
a47589fdd131b1f2363b331a3dd9266a08c40a57)
Robert Varga [Fri, 1 Mar 2019 20:45:48 +0000 (21:45 +0100)]
Introduce candidate election timeout divisor
When a candidate is waiting for an election to occur, it is sometimes
preferable to shorten the timeout interval down, so that we do not
wait the full election timeout.
This patch introduces the knob which serves to divide the normal
timeout -- i.e. it can be halved.
Change-Id: Idbe81c39bf7aecf56b9f13a242bea89a1a4ac4f1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
f9bfd1462a1cb57a34a4e567c1931b629df901ea)
Robert Varga [Mon, 4 Mar 2019 12:02:36 +0000 (13:02 +0100)]
Prevent a CME during actor instantiation
This issue was missed in review: createHandler() should not be
touching handlers, as it is already called from computeIfAbsent()
and hence would result in a CME with JDK11:
https://bugs.openjdk.java.net/browse/JDK-
8206399
JIRA: CONTROLLER-1884
Change-Id: I8d4940b8bec2cf727d97d68796f1c9cdaaaf89d3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
04107fb5efd9bd90809cb31897ec031bcb484df8)
Robert Varga [Mon, 4 Feb 2019 09:09:57 +0000 (10:09 +0100)]
Add SegmentedFileJournal
This adds an alternative Akka persistence journal implementation
based on Atomix's SegmentedJournal.
SegmentedJournal is a linear append-only log of entries, each of
which has a incrementing 63bit index. This logical structure is
stored in rolling segments -- i.e. entries are appended to current
segment until it is full, at which point a new segment is allocated
and writeout continues there. Old segments can be freed as long as
they are not needed.
This layout makes it good match to how Akka's persistence works,
with the one exception that the SegmentedJournal cannot explicitly
delete entries from the head of the journal.
SegmentedFileJournal allocates one SegmentedJournal for each
persistenceId it encounters and uses a dedicated actor for it. This
provides a major simplification in the implementation as well as
allows for concurrent persistence of multiple PersistentActors.
JIRA: CONTROLLER-1884
Change-Id: I78140b154bab44a3e17d5ffb76b040c62add3204
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
52fb7d830e43a9849c728c7b0d36470afe686fb5)
Robert Varga [Wed, 20 Feb 2019 10:46:03 +0000 (11:46 +0100)]
Cleanup java.util.Optional references
As a byproduct of gradual migration, we have a few places where
we use FQCN to reference java.util.Optional eventhough it is among
imports. Fix that.
Change-Id: Ibede76ffc614c02dbc5e29e78e6c421abcd81ec9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
7ad6a27e9b98861d9286977320198375a44a38f1)