git.opendaylight Code Review - controller.git/log

Bump versions to 2.0.11-SNAPSHOT

This starts the next development iteration.

Change-Id: I59145f3559202d28f36c39266e8be456c2ea90fd
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Handle ModifyTransactionRequest in forward path

When we have remote -> local handoff, we will use a
ModifyTransactionRequest, which we need to apply before moving on.

JIRA: CONTROLLER-2022
Change-Id: I04bdfda162d474966b153223c7480fd5b3af8099
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 920660971adda2a58672596e8ce93ac3c42b40bb)

Fix error message concatenation

We are missing a space here, fix that up.

Change-Id: I2eba64ddcb7db14a56eb65a84d25851e4869ba5a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 94c5a97a4c7e1c2374b88327de205ad91088104e)
(cherry picked from commit 5f6a60798e1d31b1bbb2b99d20896e51d66d81a4)

Enable testTransactionWithIsolatedLeader() for tell-based protocol

Behaviour differs between ask-based (which fails) and tell-based (which
continues retrying). Modify the test suite to account for this
difference.

JIRA: CONTROLLER-2018
Change-Id: I208fd8aa8088558dbe2160168ee35182e02c97fa
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 7bc006db12e2d24756192309515f3d0bc65442f1)
(cherry picked from commit 7696e7ef37313f76652683b5223d1939a569899a)

Add update FIXME to enable test

Add an explicit reference to the issue tracking enablement of this
test.

JIRA: CONTROLLER-2018
Change-Id: I62fb6377e138cfb00deb0a3fd9093ec13f76dc46
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit f37b08069ae01215fb72e8d85db045c34c0ba6c2)
(cherry picked from commit 48626c41ed546bf19cbf8ae4137f6068dc24fffa)

Enable testLeadershipTransferOnShutdown()

We are looking at cohort caches here -- and those are not used by
ClientBackedDatastore.

Change-Id: I3abfa3299d8222df6edfa98cc4a3d108a05454e4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 09d76ff670a3d54dac595336b9fe3151b793d630)
(cherry picked from commit 74adcd481bbb3c3700015820dd89d2ad6aed9c94)

Track skipped transactions

We are allocating transaction identifiers which are shared across
shards. As a transaction can touch only some shards, non-participating
shards will never see the transaction ID, leading to holes in their
UnsignedLongSets -- and those holes gradually eat up more and more
memory.

Track when we have such a hole and lazily forward a new request, which
purges one or more such identifiers -- plugging the memory leak.

The crux of the machinery is exchanging state between ProxyHistory
and FrontendHistoryMetadataBuilder.

JIRA: CONTROLLER-1991
Change-Id: I3817fa2841e5f9c405bb20ff1a104537ad459ce3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit c7078128d6f35eebee2f98108ff929dcccfc322d)
(cherry picked from commit a017289aefb2f937158b204b2593f146a10cb0cc)

Minor sal-distributed-datastore cleanups

Corrent a @GuardedBy annotations, add @NonNull annotations, fix javadocs
and make a few methods final.

Change-Id: Icb93aae229fb0aece5ece223a759ce9f36ee0297
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 27e0cf91aaba3d2a9f9e3f33e31f0faece502cd4)
(cherry picked from commit 1f3195f930ef03c58b25c0c70f3a2ed4f574fed1)

Handle SnapshotComplete message

This is an internally-generated message, make sure we handle it in
support to indicate when we have finished completely propagating
state.

JIRA: CONTROLLER-2019
Change-Id: I4f5c5231070140d54531c2e5379dffcaf1c5c4ee
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit e131c3498d286ff14890120ff5e9020ba89f10f9)

Disable state tracking on ReadyLocalTransaction

We are failing to recognize ReadyLocalTransaction as an ask-based
message, leading to us populating state.

JIRA: CONTROLLER-2021
Change-Id: I48b3c57e11430a5ca15e0559f2a8e13627f4bab1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 35b5462f1ebde45b569664af28199acb437a28d9)
(cherry picked from commit cec65b8fafb50088bc3eb6db615ac908db5122b0)

Rename UnsignedLongSet.size()

Since we have 'Set' in our name, size() invokes Set.size(), which is
not we provide w.r.t. the number of distinct elements that are contained
in the set.

We are communicating how many ranges are stored, hence the method's name
is 'rangeSize()'.

Change-Id: Iaf9999db86fe03a8e753107638fc2e189b680677
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a2779b483b7f4696d6c0f4973e2e392e870e03ed)

Move assumption

Let's assume things only just before we are hitting and error and tag
the FIXME with an issue.

JIRA: CONTROLLER-2017
Change-Id: I114ff0f3ef2e72fdc1be5e8eb61adac8ed00ff42
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit e8c92cf0ebc968dc37dd549f3eefbd9b09567c4f)
(cherry picked from commit 2e90ee3dd17759c12b2ce8dfe3e9caf5b56bbd48)

Fixup assertions a bit

The tests are using different transaction mix, pass down explicit
expected transaction count.

Change-Id: I5007fe0ad90009fcafb727bac8d37bd07dfaae94
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 07efe2f14bac8054dddca07b8d9b75238a0e14cd)
(cherry picked from commit 03fb974d6b3f3b4660c32d1fca6959910e7beb82)

Improve tell-based protocol assumptions

Make failures a bit more visible by comparing human-readable things.

Change-Id: I682c020be348c61fcdf4c3f31949ba0f666af020
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 8e4ea1b1109f2c4bf167cd2cca2a7ffe2e1f7b73)
(cherry picked from commit 67f471c7360afd147401bbde8511de4f220df0ba)

Cleanup DistributedDataStoreRemotingIntegrationTest

We have a few @Ignored tests and assumptions scattered a bit, perform
a general cleanup on them.

JIRA: CONTROLLER-1991
Change-Id: I70054276b5ffa179337a302e5d2498e925b65ce8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 4e097d8d56a7e3814c63a69da183765fd4c78a56)
(cherry picked from commit d8491296cf10f73975152484428fd7823d168d8c)

Partially enable testSingleTransactionsWritesInQuickSuccession()

Split the asserts into two methods, asserting the two datastore
options. The ask-based path remains disabled because it currently
fails tests.

Also improve tell-based protocol assertions, which need to wait
for the purge process to settle.

Change-Id: I8b2f3d84b2c7cd01dec4f7994eda716d022a98aa
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 9503012683559849cd2f01b882b57fdcbcca59f0)
(cherry picked from commit 4d487764ab5a6402b15e275db9a458ece8c6a296)

Make UnsignedLongSet.Entry immutable

Having entries mutable is just a drag when transferring them to
mutable, as we need to perform deep copies. Let's turn them into
immutables and be done with it.

JIRA: CONTROLLER-2015
Change-Id: I3be807cbdf71a51e6b9506e0932857a230924986
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 245165804cb8c11340fa5b225e5cccbe32c01337)

Fix UnsignedLongSet entry lifecycle

When we are copying things around, we have to ensure we copy Entries,
because they cannot be shared.

JIRA: CONTROLLER-2015
Change-Id: I8a484a3bf52ffeaf4f4d0e770bc77dba31dbb221
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 1a13d6e1410df52e3b8fd9d4fe6b6e7b04490e4a)

Fix Eclipse compilation error

JDT has trouble with Scala classes here, make sure we specify
more general scala.collections.Set instead of its immutable counterpart.

Change-Id: I60b56e6b7ec5705409aee8d1e30a8ae9e5579bfb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 25e140045dd07b173278460f20e17bd2c704e806)

Expose ImmutableList from FrontendClientMetadata

ImmutableList is an API contract, exposed it to callers.

Change-Id: I7fb2a09c2b225ce48ece7400128c8215d70292fe
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit bf54edf4eb5fffb48e005d07a0a20c8971b77a99)

Cleanup cds-access-api dependencies

We are using Guava, declare that as a dependency.

Change-Id: Ibe905f0d2d87c5e7dfb4fad598acb27b4323c7b8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 66b01aae92d17af1e06bad0c0e380feb5c6cf2de)

Add MutableUnsignedLongSet.addAll()

Add the ability to merge two UnsignedLongSets. This is useful when
we are using ImmutableUnsignedLongSet as a data interchange format
to communicate changes to a MutableUnsignedLongSet.

JIRA: CONTROLLER-2015
Change-Id: Ia84474b685872586722914f20db8f96a3c172f97
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 824dce54df4b23120461e112574d2ff2effafcf6)

Add UnsignedLongBitmap

Rather than using a ImmutableMap<UnsignedLong, Boolean>, we can do
have a much denser representation with a specialized class. Here we
are introducing an explicit UnsignedLongBitmap, which stores an array
of longs and an array of booleans, ditching the intermediate objects.

Also clean up error reporting, throwing an IOException instead of a
VerifyException when things should go south unexpectedly.

JIRA: CONTROLLER-2013
Change-Id: Ie64da0af68ea2898dc77368afd4fce8abd2cccea
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 3402cfce32b05957219e54754dd7ca5b0a54cd0e)

Sprinkle @NonNull annotations

We have a few return methods which could use @NonNull annotations,
sprinkle them around.

Change-Id: I48c0f8879127673eb122321dd6c92d4572098f68
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 26b290518d362a85311c5b5591e3ef72e0772cf2)

Inline Entry.contains()

MutableUnsignedLongSet.add() has half of the check done in contains()
as an invariant. Inline the second part there, improving performance a
bit.

Thisleaves only UnsignedLongSet.contains() as the sole caller of
Entry.contains(). Inline it there, reducing the footprint a bit.

JIRA: CONTROLLER-2014
Change-Id: Ie38283a1fc46aff8fbc717a23aa9a068805636db
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a8ebf0b87a2fd613a5b08bd9e295006415f7de27)

Remove UnsignedLongSet.addImpl()

We now do not share utilities in UnsignedLongSet, hence relocate
implementation code to MutableUnsignedLongSet.add().

JIRA: CONTROLLER-2014
Change-Id: Id1578a8ea639ed512d55b2e6dca26de5abdf3ab0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit ec1d15d2b52cb8f378ff685c2c3400b71c5ec360)

Improve UnsignedLongSet efficiency

We can use NavigableMap's higher() and floor() methods to elide object
object allocations.

JIRA: CONTROLLER-2014
Change-Id: I75747c58027caee2678e32187dbd628d60ce6604
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit fb830fa236f32513f759893270552e320c122c5d)

Use ImmutableList.Builder in FrontendClientMetadata

We are performing a defensive copy in constructor, hence it is
better to allocate a builder, skipping an untrusted copy operation.

Change-Id: I95d7aa90f6da325aaae6a340caf49a6e17fe9f64
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 42769c3ac0d99f8fe4f845fed66408ff4dfb2388)
(cherry picked from commit 127deda9e888af9f7f33fcc039570b9ca7eda0fb)

Use ImmutableSortedSet for small ImmutableUnsignedLongSets

This adds a bit of indirection, but for a few entries this ends up
using arrays instead of an RB tree, offering better density.

JIRA: CONTROLLER-2012
Change-Id: I09714c6bf3272946f6cee4360f80a106453d0e61
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 318c0bbc4ecfdf532a4cc3e219508a7c5744d160)
(cherry picked from commit 48b7ce82dfd92584a0a1bfc33e72b5154a3bff79)

Use UnsignedLongSet instead of RangeSet in metadata

Split UnsignedLongSet into two implementations, mutable and immutable.
Use the mutable implementation in runtime tracking of identifiers and
use the immutable implementation for tracking identifiers in metadata.

The serialization format is kept compatible with RangeSets, although
that implies a minor penalty in serdes.

This switch ends up potentially using more objects for small sets, but
that is offset by not having Cut indirections and most notably being
resistent to allocation of huge arrays.

JIRA: CONTROLLER-2011
Change-Id: I0c84ffaaa4ce39299cef9006784b8aff78dd0f83
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit e9efe27538adb5ae575f77fda90f147d46341801)
(cherry picked from commit 4da04f2f6e68f8dc07860c9e440bef26d69f36a4)

Reimplement UnsignedLongRangeSet

We are tracking simple discrete values, for which a RangeSet is an
overkill due to it being generalized utility working with all possible
ranges.

Reimplement UnsignedLongRangeSet as UnsignedLongSet, tracking discrete
closed ranges of longs, which improves both memory footprint and speed
of contains()/add() operations.

This also spots a failure to disconnect purged histories during
FrontendHistoryMetadataBuilder.toLeaderState().

JIRA: CONTROLLER-1720
Change-Id: I24fae4174201fd133a282f27589d6c274c06c8dc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 19a6bcd20358c883478ee3b82e67cb147113f1c0)
(cherry picked from commit 9468993ee0c3941c19f838586773537968921d63)

Improve Frontend{Client,History}Metadata

We are using Guava's immutable implementations, make sure we annotate
fields with @NonNull and use concrete types.

Change-Id: I4fc2a0ff39b2b8166deca41425771ea9ed04e469
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a66474c41883733413a6851d49fb5ade892764b3)
(cherry picked from commit c3bbe471b3743b5145c67325839b0c2a36bc5275)

Sprinkle @NonNull on FrontendClientMetadataBuilder fields

We have two fields which are guaranteed to be @NonNull, mark them as
such, making it clear we comply to getIdentifier() contract.

Change-Id: I9159823fb6c8a4a3c064796cb6752f57d98846c2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 535d499672a88b772ca0b296fa40882f8fd0cf03)
(cherry picked from commit 68017211f4b33f17d02266c50c786856c33b959b)

Correct Frontend{Client,History}Metadata deserialization

We are turning Range.closedOpen() into Range.closed() during
deserialization, which the resulting ends up also covering the upper
bound, which it should not.

JIRA: CONTROLLER-1942
Change-Id: Ib8f9016e2eddcf014ff5e451ac82cd77b66d7019
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 546cd1fd100dbaa36908b22c2f422320dbd8c4b2)
(cherry picked from commit ceb67a6db5bef27cd1dd5374c1a75f1036c3c1d1)

Correct annotations

Correct @GuardedBy vs @Holding and add @NonNull annotations.

Change-Id: I2142b1c5910f2388e8f30a04f7ff8bc8237dde26
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d502ae8e1529b69af55a59c3e664a02457c05ec6)
(cherry picked from commit 8510e2ac3a9e182858de6f3de169429319048275)

Bump versions to 2.0.10-SNAPSHOT

This starts the next development iteration.

Change-Id: I6631ef805109cc48220bc1e7b6f63685d9c390b8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump mdsal to 6.0.11

Pick up a few fixes from upstream.

Change-Id: I874ac236df92359badf741042f2a9765c60b5378
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Ignore DataTreeChangeReply message

DataTreeChangeListenerActor is sending replies towards shard on each
processed datatree change. This is not handled by Shard, which ends up
flooding dead letters.

Make sure we ignore these messages, so that the logs are cleaner. Also
speed up the non-null check in DTCLActor.

JIRA: CONTROLLER-1986
Change-Id: I11b172d778c5b8cdf16aec58e77ec7faa9b6b146
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d6bc529eaf624afe74d9bf9ee9ab0cb926bba576)

Fix sal-common-util dependencies

We have a number of unused dependencies here, clean them up.

Change-Id: Ibc534ab9ba28a7e70270a042a1b8e55f8934bc28
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 7adae02ae943d3d537dafcda1f10877ab5040ea2)

Upgrade lz4-java to 1.8.0

https://github.com/lz4/lz4-java/releases/tag/1.8.0

Change-Id: Ica9bc96f8db76cb8dd8eda19264f72bfb1ea674c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 47de86692709658b4495d2566c91e7a61d62de91)

Correctly forward DOMDataTreeChangeListener.onInitialData()

When registering proxy listener, make sure we do not lose
onInitialData() events.

Change-Id: I2529d64d10f477ad52e90dec15d88c6228a47af0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d33f2a4667e423fcbff25454f28412a5b3d01a42)

Bump versions to 2.0.9-SNAPSHOT

This starts the next development iteration.

Change-Id: I7c11560ab1a604feb73b43b1d38b91e9c5c5413e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstream versions

Adopt:
- odlparent-7.0.10
- yangtools-5.0.10
- mdsal-6.0.10

Change-Id: I4d59befcaf1ba83daf6ca60d8ab2636f701f78d0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump to yangtools-5.0.9/mdsal-6.0.9

Pick up latest updates from upstream.

Change-Id: Ib2efc9777468e85e43c364115e4d53b53d4991fe
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix docs version parsing

We do not want to pick just any <version> tag that comes first,
as that well may be parent/version.

Change-Id: Ic51ca768f42de498dd86f470f839e64a063c1dd4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b49f5b6f1bc751c045ef16d8c916291c15872543)

Bump versions to 2.0.8-SNAPSHOT

This starts the next development iteration.

Change-Id: I34b05ca6ddd2975ae79cdf31751ccd2967f0aa12
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstream versions

Adopt:
- odlparent-7.0.8
- yangtools-5.0.8
- mdsal-6.0.8

Change-Id: Ic2e5c097607efd1874d843f7825de6f9d63cd92f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix initial datastore configuration

Shards need to be initialized with initial configuration upon creation,
otherwise they end up being reconfigured. Make sure callers are aware
of this caveat by requiring explicit properties on instantiation.

JIRA: CONTROLLER-1970
Change-Id: I3d6f96780d26435ac178ec9935f0f67befe3687a
Signed-off-by: tadei.bilan <tadei.bilan@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 282fd5f43263464180a2ecc806c0eb333f0f2c5c)

Enable modernizer in sal-distributed-datastore

We are down to a single offender, suppress it and enable moderinizer
enforcement.

Change-Id: I9daf9f54276c48aac09e999327add7ade92dfc95
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 1b8b80be1a8c7dbc3de11d8e4e283a755fa9a14f)

Use java.lang.ref.Cleaner for ClientBackedTransaction

Improve temporary file clean up by using a Cleaner to dispatch
cleaning tasks. Since it gives us a Cleanable, we can dispense with
a tracking map and removal from it -- Cleanable makes sure it is
called exactly once.

JIRA: CONTROLLER-1911
Change-Id: I02d2ee57a9fada2c54ba06008b6c28681a709748
Signed-off-by: tadei.bilan <tadei.bilan@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b0ea37067f7ac8d03cb785ce70616fa16a5ba5a4)

Speed up root overwrite check

We can use YangInstanceIdentifier.isEmpty() and extract the single
ModificationType we care about.

Change-Id: Ie29c32ea0b197547bce67a7b2ffdd530e79276da
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 5d761355a11fc761e8faccb36b05453013f1e4d1)

Use Objects.requireNonNullElse()

Modernizer is pointing this one out, drop the MoreObjects
dependency.

Change-Id: Ibfe3f79dae3b53bc30d130ae68bfab83214ee128
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 98137e91bb5896d111293f8109bed42b3454b943)

Use java.lang.ref.Cleaner in controller.cluster.io

Improve temporary file clean up by using a Cleaner to dispatch
cleaning tasks. Since it gives us a Cleanable, we can dispense with
a tracking map and removal from it -- Cleanable makes sure it is
called exactly once.

JIRA: CONTROLLER-1911
Change-Id: I5fb715102912359cf002129d25a7433199826982
Signed-off-by: tadei.bilan <tadei.bilan@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit e0e956ecf0a4e5085f33e67f4fb73141876f1668)

Bump versions to 2.0.6-SNAPSHOT

This starts the next development iteration.

Change-Id: Id8dbfcef3080fc10bcb627d2b7cead0e6371e8c5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Make sure to clear candidate from memory

The candidate needs to be cleared even when processing
replicated candidate(ie on a follower) as there is no need
to keep it deserialized in memory

JIRA: CONTROLLER-1928
Change-Id: I1403dbe27def5e0040a8d9f145a92ac06aae9d1e
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
(cherry picked from commit 5a824a836a00ff21a855d7339a2fa9b64678db52)

Expose AskTimeoutException counter from DatastoreInfoMXBean

This is a natural place to keep track of all exceptions which
happen in datastore context. Hook an onComplete() to each future
which goes by and update the counter.

JIRA: CONTROLLER-1963
Change-Id: Ib208b377e1ab533993597660228b5fc87919e75b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 9045f6fc943e9794055463442dcfa9bce64b68d3)

Add multi journal configuration for segmented journal

We dont need to have large segments for operational shards.
Add in multi journal configuration that gets used when shard has
persistence turned off.

JIRA: CONTROLLER-1938
Change-Id: I39349503079ef03177c8b9b52909078c5f35d6ba
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a3459dc797e4e0722b945b80bc8ef3123762fc8d)

Update DataJournal interface

The interface is not documented and we need to differentiate between
indices as viewed from segmented journal and the persistence journal
views.

JIRA: CONTROLLER-1954
Change-Id: Ibbc384b88f8d5567e9af18d83b9a84f7c9b9634c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 255e74efd633f2fbca7ce4f1372004d93cc81a10)

Migrate SnapshotManagerTest

Use MockitoJUnitRunner to initialize mocks, which shows we have
unneeded stubbings and that we are lacking two stubs.

Change-Id: I5a4ebde09a72bb4d2c2ba9affa524731413bd02c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a062d3218022ed84e91070bd8cf12bb6e76bd48a)

Migrate ReplicatedLogImplTest

Use MockitoJUnitRunner to initialize mocks, which shows we have
unneeded stubbings.

Change-Id: Ib907fa4b027d063d3a0c3d7b9c0e94115ab39495
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 860d2ae83c910ed7a232cee3750cce497d23d268)

Do not reset datasize after a fake snapshot

If we reset dataSize from SnapshotManager's fake snapshot, we would
not have correctly accounted for on-disk size.

Fix this by exposing a simple knob, which allows suppressing dataSize
update during snapshotCommit().

JIRA: CONTROLLER-1957
Change-Id: I553ddb523ac8504892cc5353bfe4b002c25436ce
Signed-off-by: tadei.bilan <tadei.bilan@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit ff29db5dc6012f77bbe53f57ddce929b0de093b3)

Log replicated log dataSize changes

This is a bit of critical state, make sure we recalculate it atomically
and log its changes.

JIRA: CONTROLLER-1957
Change-Id: I09532384ce5bbaf7218c0ffdb64799541037fafc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit e96021b2e492bbaf7efe0804628bfe055b9733f7)

Reformat AbstractReplicatedLogImpl

'final' on arguments, remove unnecessary type for empty list, add
a @NonNull return annotation.

Change-Id: I697c6fdd33e373478a645f3bbef987a054d8ed70
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 12cca9dd5a7008d707222c68b053421594ae82b4)

Clean up AbstractReplicatedLogImplTest

Use static imports for Assert methods, add final qualifiers.

Change-Id: I35f589b14e1188fa1fb94524065b562023687f27
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 41d3174038060e2c8dfd49dcb4f85e74dff50cd8)

Bump mdsal to 6.0.7

Pick up latest updates from upstream.

Change-Id: Icb194c5a20bffa43a3b9d64e1102095c243f9db8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Correct javadoc reference

We should not be referencing UntypedActor, but rather AbstractActor.

Change-Id: I6f17d818e2c03c9c4a9e0df26317212f28425734
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 3bdf1493c0ce4218a1a3e26cdf7c5c4af1d2aeeb)

Odl-jolokia package superfluous artifacts

From /odl-jolokia/target/feature/feature.xml:
wrap:mvn: jar content is already in present in jolokia-osgi

JIRA: CONTROLLER-1961
Change-Id: I3d689cdedb002f9d97937671ebfe38127e1095e0
Signed-off-by: Vladyslav Marchenko <vladyslav.marchenko@pantheon.tech>
(cherry picked from commit f3c17d0d7c3e356f60f47ef460258d0373512acf)

Bump verions to 2.0.6-SNAPSHOT

This starts the next development iteration.

Change-Id: I2c753eededdfd635799531df911dec6a0b3a578f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstream versions

Pick up odlparent-7.0.7, yangtools-5.0.7, mdsal-6.0.6 for latest
fixes.

Change-Id: I9a81b8eee714241b30bac8bd772db6f98697e958
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add a segmented DataJournal abstraction

We are going to change the way we organize journal entries within
a segmented file. Add DataJournal abstraction which exposes
a unified interface towards the SegmentedJournalActor.

Current implementation is split out from SegmentedJournalActor
as DataJournalV0.

JIRA: CONTROLLER-1954
Change-Id: I0d1de42b22e75610d0434548483091e3359123e4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Correct mdsal-it-base dependencies

This is a weird dance of dependencies, for which we need to match
the content of pax-exam-features.

Change-Id: Ia011022174f8d633f3ec3ac17cd2f8d7d33a7ee0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump versions to 2.0.5-SNAPSHOT

This starts the next development iteration.

Change-Id: I41c77600cfde36c0e74087472272a4890dd13122
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump mdsal to 6.0.5

Pick up the released version.

Change-Id: I785e24cdccb12a2f3da3ac03abe008a7c521681b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert messagebus-impl to OSGi DS

messagebus-impl is an extremely simple component, convert it to
Declarative Services.

JIRA: CONTROLLER-1882
Change-Id: I5131530f3ba907525a7a29775136cd5123f56f18
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert cluster-admin to OSGi DS

This is a very simple component, convert it to declarative services.

JIRA: CONTROLLER-1882
Change-Id: I664db0ef3cbf6b5cf900d969fa56bc04440c2604
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump odlparent/yangtools/mdsal to 7.0.6/5.0.6/6.0.5-SNAPSHOT

Since CONTROLLER-1882 needs a few changes from upstream, bump to
released odlparent/yangtools and snapshot mdsal.

Change-Id: Ic2a74eb7d989e8da859fec471ba7c442374bef70
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove java.desktop dependency

Remove java.beans dependency to be able to run without java.desktop module

JIRA: CONTROLLER-1910
Change-Id: Ib7cfba1e0ec4694e0f63586837486a00e9863ea3
Signed-off-by: tadei.bilan <tadei.bilan@pantheon.tech>

Add direct in-memory journal threshold

Some deployments benefit from placing an absolute numeric limit
on the retained memory. Introduce a new tunable, which overrides
the usual percentange limit.

JIRA: CONTROLLER-1956
Change-Id: I688e226b173386765bea74931b6aaf617bda30a8
Signed-off-by: tadei.bilan <tadei.bilan@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Doc update for "Extend Websocket streams for data-less notifications" enhancement.

Refer: https://git.opendaylight.org/gerrit/c/netconf/+/89928

JIRA: NETCONF-689
Signed-off-by: Nikhil Soni <nsoni@luminanetworks.com>
Change-Id: I5e36f30a61fb6313c85f34fbe9721c5a5fbc317d

Instantiate distributed datastore asynchronously

With OSGi DS we can have very explicit control over component
activations. Use component factory pattern, which provides async
publish of DOMStore into service registry. This allows us to
make startup asynchronous).

JIRA: CONTROLLER-1882
Change-Id: Id09f956ec9088388ec5cbd3162d128310baddf51
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor DataStore readiness tracking

Using a CountDownLatch is not composable, which leads to current
layout. Switch to using a SettableFuture, which can be accessed
via AbstractDataStore.initialSettleFuture().

This allows us to externalize the settle policy, letting callers
decide what to actually do.

JIRA: CONTROLLER-1882
Change-Id: Iaf9a359cfc2507ae35688fca3673c13713c2b427
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert netty-timer-config to OSGi DS

This is an extremely simple forwarder, convert it to OSGi DS.

JIRA: CONTROLLER-1882
Change-Id: I8fbb71fba704e273b8e1c8fa79b4e4dd90aeff31
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert netty-event-executor-config to OSGi DS

This is an extremely simple forwarder, convert it to OSGi DS.

JIRA: CONTROLLER-1882
Change-Id: Iff74bad52a78adcb6398b09300f6bfa7457dea7d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert sal-distributed-eos to OSGi DS

This is a simple component, there is just no need to use blueprint.

JIRA: CONTROLLER-1882
Change-Id: If64faff3cab0e66303b5e2d5db4af7f9ea2a2b50
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert sal-distributed-datastore to OSGi DS

This is a follow-up patch, converting more of blueprint wiring to
OSGi Declarative Services.

JIRA: CONTROLLER-1882
Change-Id: Ibb0a9f57c470e5572b64e85fdf87a093594d0367
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert FileModuleShardConfigProvider to OSGi DS

This is a simple injected holder, convert it to a full-blown
component.

JIRA: CONTROLLER-1882
Change-Id: I6676055801c178c4ffb4137ec6721128014dedde
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert DatastoreSnapshotRestore to OSGi DS

This is a simple holder used extract restoration code, split it out
of manual blueprint.

JIRA: CONTROLLER-1882
Change-Id: Ic7f97fa1a2b0302c7f1ccc14a9a79fe7fce57412
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Migrate DatastoreContextIntrospectorFactory to OSGi DS

We are slowly converting hand-written blueprint config to OSGi DS,
so that we can end up ditching blueprint completely. This is the
next small step towards that goal.

Change-Id: I398d71344bccfef752128c561d6d2ff851edc5c2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump versions to 2.0.4-SNAPSHOT

This starts the next development iteration.

Change-Id: Ie6bdb44e3c37ad598206520556a2380708d44430
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add optional lz4 compression for snapshots

Added ability to use lz4 compression both for snapshots sent to
followers and snapshots in storage.

JIRA: CONTROLLER-1936
Change-Id: I073120efddde869b10999450057b91e75f0ffe07
Signed-off-by: tadei.bilan <tadei.bilan@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add optional timeout parameter for backup rpc

Once the snapshot size grows large the backup rpc can take more
time than the default timeout value causing it to fail.
Add the option to override the timeout in the backup rpc.

Change-Id: I878066668f45abcfe758a7b90d34576bff1b7db0
Signed-off-by: Tomas Cere <tomas.cere@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump odlparent/yangtools/mdsal to 7.0.5/5.0.5/6.0.4

Pick up latest upstream fixes.

Change-Id: Ia8b756565d22c5b6f39c77b1cc9b8ce3dc0b2aa0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not fail on warnings for docs-linkcheck

This is a workaround due to sphinx-tabs being incompatible with the
Sphinx linkchecker which throws a warning. We mainly use linkchecker
to catch broken links so we don't really need to fail on warnings as
the main "docs" builder will still fail for warning level issues so
this is a reasonable workaround.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: If3e7cea79528ed31fadcea7c7a4c4e13315d1fcd

Speed up DatastoreContextIntrospector a bit

We can rely on static QNames and pre-computed YangInstanceIdentifier
to reduce the amount of transcoding we do.

Change-Id: Ibae5a6938703e12dc7584de002fb7cd0538d8289
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump odlparent/yangtools/mdsal to 7.0.4/5.0.4/6.0.3

Pick up latest upstream fixes.

Change-Id: Ic661675cfc75311d2dace18997f4626033a2cdd6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix controller-docs

We are referencing a few provided annotations, make sure docs
are seeing them as well.

Change-Id: Iffd0d7e9441441f44cb0c2126812d872e707755e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert sal-remoterpc-provided to OSGi DS

Declarative Services offer better integration with OSGi lifecycle,
use them instead of blueprint.

Change-Id: I49b41fa0fba4714f54e097820b85aefe7da67364
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Factor out ActorSystemProvider from blueprint

ActorSystemProvider is quite simple, make it completely independent
of blueprint.

Change-Id: I45233cad19ec8580d45d387efe627ab0935ce2eb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Turn FileAkkaConfigurationReader into a component

This is a clearly standalone implementation of a common service,
turn it into a properly-injected component.

Change-Id: I521c3433093ab3aa911aa870c10b3d4e2bff8418
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>