Isolate Pekko Snapshotter methods in RaftActor

RaftActor is an abstract class, which leaks Snapshotter interface to
users. Override methods to mark them as deprecated and provide internal
methods to route to Pekko.

JIRA: CONTROLLER-2134
Change-Id: Iad9240562c44dceec9028b79f0b1c886bdca8f83
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Simplify appendReceived() signature

We only need the appended entry (for now), perhaps at some point not
even that. Simplify the metehod signature and add some documentation
about what is going on.

JIRA: CONTROLLER-2137
Change-Id: Ida272793dee645ffba571185b1ccb127a06d380d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Update applyRecoverySnapshot()

We should be receiving an plain SnapshotState here, so that we are
disconnected from serialization details.

JIRA: CONTROLLER-2134
Change-Id: I70028cae9b133143e05753327cf5817fd628061a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Split ReplicatedLog.appendAndPersist()

We have two distinct callers, making the resulting methods simpler and
making the on-follower case a tad more useful.

JIRA: CONTROLLER-2137
Change-Id: Idc1dd3aaa397209db10258d60471789e2bd04ba4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Update RaftActorRecoveryCohort

The cohort should not a full Payload: it only needs to see
StateCommands -- which is the point behind a FIXME in
RaftActorRecoverySupport.

Adjust name to applyRecoveredCommand() and propagate the type
information outwards.

JIRA: CONTROLLER-2137
Change-Id: I07e1b2dcaa349a2d60d6654dc83656f013aad45e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add ReplicatedLog.lookupStoredMeta()

AbstractLeader does not need the full ReplicatedLogEntry, it just needs
its metadata with a durability flag.

Eliminate a caller for ReplicateLog.get() with a dedicated method
needing only EntryMeta.

Since we are in the area, turn a TODO in consensus reaching into a FIXME
we a better analysis.

JIRA: CONTROLLER-2137
Change-Id: Ifa593ddfefcf3015502e8b42e30c37afb85768de
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove FollowerLogInformation.incrMatchIndex()

This method is completely unused, remove it to simplify reasoning about
its value.

Change-Id: Ibf43a0d6ac47676b20a4fd39c54b1a9bcf2a60e1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Document ISE exception

We are about to expand the exceptions being thrown here, document
current contract first.

Change-Id: If70bb8e494de3b1b93049d7594cf9549e4e587ed
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Report IOException from EntryWriter.flush()

A flush operation may fail, let's make sure users are aware of that.

JIRA: CONTROLLER-2137
Change-Id: Ia01bcbd79e47897e94a1fa640d9af66ade590bf8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove flushOnCommit

We are not using the flush-on-commit capability. Since it introduces a
potential error path, let's drop it.

JIRA: CONTROLLER-2137
Change-Id: Icc81c41e82c04fd9e518731a90a3d17333f6661f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Pass last segment to SegmentedByteBufWriter

lastSegment() is a potentially-failing operation, so let's skip it by
passing the segment, as we have just acquired it.

JIRA: CONTROLLER-2137
Change-Id: I1ea35cdaf27ef1a7a208ad19bacdb684f0c69bcb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Propagate SegmentedByteBufJournal.close()

SegmentedByteBufWriter holds a reference to the segment at which it
positioned, retaining the backing FileWriter.

Make sure clear that reference when SegmentedByteBufJournal is closed,
allowing the backing buffer to be released.

JIRA: CONTROLLER-2139
Change-Id: I935fff007c30c598ea47af1fe20cb69e32f9eeb8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Release FileWriter for inactive JournalSegments

In the DiskAccess case, the FileWriter holds on to its backing buffer,
which means we need to release() it when we switch to being inactive.

JIRA: CONTROLLER-2139
Change-Id: I02a40b32eee243f4ab31a213a669794459bf6399
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Inline checkState() calls

We do not really need these, inline them to explicit throws.

Change-Id: I9a3a5e6881b5b9ad0b58b08030f3fac329809d46
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve SegmentedByteBufJournal.loadSegments()

Combine initialization so that we do not re-index existing entries.
Also sprinkle a few @NonNull annotations.

JIRA: CONTROLLER-2137
Change-Id: I1ec44a3b645e5bf64cefdc3e6cbb918d21c51ad3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not mask IOException JournalSegment constructor

Make sure callers are aware of an error being possible here, leading
to improved error handlign in SegmentedByteBufJournal.

JIRA: CONTROLLER-2137
Change-Id: I03c6080d9628082a039e6885b8dbcebf5a206345
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Report IOException from SegmentedByteBufJournal.size()

Do not use UncheckedIOException to mask errors here.

JIRA: CONTROLLER-2137
Change-Id: I460de64f4a93f12777fcda96aa235f062db0cf38
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Report IOException from segment operations

We are reporting an UncheckedIOException on IO failures, push this
masking one level up, improving SegmentedJournalActor error handling.

JIRA: CONTROLLER-2137
Change-Id: I135ecdbe44ed9daf684a635e87d6e7f65b7b6779
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Close deleteJournal reader

We have a dangling reader here, make sure to close it when we are done.

Change-Id: I7b669bca3cd3cfaafb7811c6b83c941bdd04d897
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove StorageException

Use UncheckedIOException, as all remaining users are just masking an
existing IOException.

JIRA: CONTROLLER-2137
Change-Id: I4886ee6e13a07759da0b4e4df698ab5ebabeb9bd
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Factor out StorageExhaustedException

Use an IOException subclass to report storage exhausted.

JIRA: CONTROLLER-2137
Change-Id: I3a3b886a0d1359c5130fdfa4731948aa86f51dc4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Report IOException from EntryWriter.append()

Eliminate a silent exception in favor of being explicit.

JIRA: CONTROLLER-2137
Change-Id: I8640de39fc18c4755919788a945d64a50abc0531
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Simplify SharedFileBackedOutputStream callback

We only need a Runnable, not two separate objects.

Change-Id: I8d6c59ef1114ffc2795222349bfcd5618ff262a5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fixup applyIdentifierFor()

Simplify the declaration, so that SpotBugs does not get confused.

Change-Id: Ib79d9ea9fd4cb90d7b31a3743fecdee3e91f0b3e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor base.messages.ApplyState

This is a pure DTO, reduce its proliferation, opting for methods with
explicit arguments instead.

JIRA: CONTROLLER-2137
Change-Id: I17f87dd6600767996e957ca6111e7691f652951b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor getApplyStateFor()

Reduce ApplyState proliferation, as we only need to pick up the
Identifier.

JIRA: CONTROLLER-2137
Change-Id: I64c3c84c76dbbb77d51a11f2efc1e3b766d728c8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor MockPayload

Split it out into its own file, with MockCommand as the new name.

Change-Id: Ibee1897a4385a7da756b81c0efd180a2726a2b8b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Pull down Shard.persistPayload()

The decision whether or not to go through persistence/replication for a
particular submitCommand() is internal to RaftActor.

Pull down the logic from Shard down to RaftActor, allowing us to hide
RaftActor.hasFollowers().

JIRA: CONTROLLER-2137
Change-Id: Ic8fe37e19a172e561c41c370b7a7d4a17d267ea3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix raft-journal module info

ByteBuf implements ReferenceCounted, but is not a JPMS module, so we
need to mirror this dependency ourselves.

Change-Id: Ieedf559a5de734f249f208d330dcf39df118b29a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Drop ActorRef from ApplyState

ApplyState is carrying an ActorRef, which goes unused: the core driver
is Identifier here.

Drop ActorRef from ApplyState/Replicate and rename methods to clarify
lifecycle:
- persistData() becomes submitCommand()
- applyState() becomes applyCommand()

JIRA: CONTROLLER-2137
Change-Id: Ie6a23daadf17fcf5af2a947a8ab09aa199bdd8ab
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove a superfluous constructor

JournalSegmentWriter is only ever instantiated from Inactive state,
but let's lower the tangle index by eliminating the Inactive-taking
constructor.

Change-Id: I5205167f9c21d8d639babfd338abdc5bbad9108a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rename CompressionSupport to CompressionType

This is a simple name adjustment.

JIRA: CONTROLLER-2134
Change-Id: I0918e6d3791c55190b65e28bfd8dc85f03793da9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move InputOutputStreamFactory

This class is only used by LocalSnapshotStore, move it to the same
package.

Change-Id: I3caceae1b54672c0a09655c6d1d7b86c8c5b7151
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Switch to StateSnapshot.Reader to InputStream

This isolates the guesswork around InstallSnapshot content, making it
something internal to sal-akka-raft.

JIRA: CONTROLLER-2134
Change-Id: I48b8b2089f59d4ba21601f43187ad4159bec4ee1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove ReplicatedLogEntry.getData()

Replace all users with a call to command().

JIRA: CONTROLLER-2137
Change-Id: Ie21c2cd0de59058ce3bb7f3c42ae133dc9fbac2d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add EntryStore and LogEntry

Split out the methods supporting entries and define a baseline for how a
LogEntry looks like.

Naming is adjusted to match the RAFT paper, i.e. a LogEntry carries a
'command', not 'data'. This paves the way for a potential EntrySource
interface.

This clarifies RaftActorSnapshotCohort's relationship with
StateSnapshot, introducing a StateSnapshot.Support to hold the
indirection to reader/writer.

JIRA: CONTROLLER-2137
Change-Id: I1513ab5ba8533bc5816294b15a4c5fcc19b974a2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Expand PekkoRaftStorage

Teach RaftStorage about its root directory and consult it in
PekkoRaftStorage.

JIRA: CONTROLLER-2134
Change-Id: I8f6265d697f1f1a237109b0c875050824827995c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not pass ActorContext to ShardSnapshotCohort

We are no longer creating an actor, hence we do not need ActorContext.

JIRA: CONTROLLER-2134
Change-Id: Ic46b3a6abc367eeb7cd314161c15a0a32d6bf19e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix a raw type

Guarantee a ShardSnapshotCohort return, fixing a warning.

JIRA: CONTROLLER-2134
Change-Id: I4606cac4487e34a89f3cdbeb667f4094bb704e4c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Split out SnapshotStore methods

Move more methods from DataPersistenceProvider to SnapshotStore, in
order to isolate the scope of changes.

JIRA: CONTROLLER-2134
Change-Id: I39aff5675b0515ee5f35e15ff130ed883f81027d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Clean up streamToInstall()

Rather than having a clunky BiConsumer, use a sealed class hierarchy to
communicate success/error.

Change-Id: I0f8305f109d95c8ed9efb21d495a34889613d99c
JIRA: CONTROLLER-2134
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor FileBackedOutputStream

The lifecycle here is a bit hairy. Improve it by keeping track of what
state we are in.

While we are in the area, split out TransientFile(StreamSource), which
is backed by a temporary file.

Major improvement here is TransientFile, which acts as the GC anchor to
a file -- eliminating the need to hold on to FileBackedOutputStream.

JIRA: CONTROLLER-2134
Change-Id: I3fbdcca3e927551572a38dd4d0268b0d56b68aa9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove superfluous openBufferedStream()

StreamSource is already delegating to openStream(), hence there is no
need to do the same in other implementations.

JIRA: CONTROLLER-2134
Change-Id: I8b733e342f7bbb5fac8de755c6ad973509eb27db
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix RaftStorage actor confinement violation

We are invoking a callback from a background thread. This should be
routed via executeInSelf() to ensure we execute in proper context.

JIRA: CONTROLLER-2134
Change-Id: I92a40ebd7ed170ae7a77d7923eedefe285978692
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Ditch use of ByteSource

Rename InputStreamProvider to StreamSource with two specializations, based
on whether or not we know the size.

Allow conversion from StreamSource to SizedStreamSource, for the cost of
consuming the input stream once.

Also disconnect SnapshotSource from StreamSource: the former indicates
the format of the underlying InputStreams, while StreamSource provides
the streams themselves.

JIRA: CONTROLLER-2134
Change-Id: I214b990c9b9d573661de824cbb5d371bcd66ccb7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add cluster.raft.spi.EntryData et al.

We need to reign in ReplicatedLogEntry serialization, so that we can
operate without Serializable being in the picture.

Baseline semantincs of a ReplicateLogEntry is that it represents a step
in RAFT maintentenance, affecting either the contents of StateSnapshot
or a RAFT server state transition.

Introduce cluster.raft.spi.EntryData as the unifying concept, with
RaftDelta representing RAFT server transitions and StateDelta
representing user state transitions.

This provides a natural place for Reader/Writer interfaces, which allow
us to co-locate serialization support within the delta contract.

JIRA: CONTROLLER-2137
Change-Id: Icbf3b32a9ca68c6b60709ea79b509b3078793ea9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Convert sal-akka-raft into a JPMS module

We need to use sealed classes across packages, for which we need
sal-akka-raft to be a full module.

This is a bit painful, as we need to:
- move test utils into o.o.c.cluster.raft
- open that package to Pekko
- add a ton of package-info.java files to cover all the exports
- retain the explicit DynamicImport-Package=* declaration

JIRA: CONTROLLER-2137
Change-Id: Ia013cac658c054815ba3d06764f13d61b2b953d7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Split out cluster.raft.spi.StateSnapshot

This extracts the trait of being a snapshot from being Serializable --
which is replaced via explicit StateSnapshot.{Reader,Writer} interfaces.

The newly-enabled SnapshotFile.readSnapshot() is a testament to our
ability to read/write state snapshots without involving Java
Serialization. Unfortunately only testing Payloads are taking advantage
of this right now, as ShardSnapshotCohort still requires
Object{Input,Output}Stream.

JIRA: CONTROLLER-2134
Change-Id: Ia3cfd570f6176d629749f779bb86caf01d656929
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add cluster.raft.spi.SnapshotFileFormat

This is a first cut at having a sensible file format for storing
snapshots. Unlike Pekko-based persistence, which stored the complete
contents of a Snapshot, we just pick store data that cannot be inferred
from the way things are laid out.

The following o.o.raft.spi constructs are introduced:
- InstallableSnapshot, a handle on byte state that can be tranferred via
  InstallSnapshot RPCs
- SizedStreamSource, the moral equivalent of Guava's ByteSource
- FileStreamSource, a SizedStreamSource served from a section of a file

JIRA: CONTROLLER-2134
Change-Id: I9743ec8b701badd12857e483acefb95709d9b68a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix CompressionSupport.NONE

We should be handing out a PlainSnapshotSource, not a LZ4-compressed
one.

JIRA: CONTROLLER-2134
Change-Id: I41fc06d7a28a5d35e084645612c0deb6f3145c13
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rename SnapshotFileFormat to CompressionSupport

The idea of having a single compressed stream will not work quite well.
Take a first step by separating what we have into CompressionSupport.

JIRA: CONTROLLER-2134
Change-Id: I2140b1fa84ebdd9f473b07f14bb673049614a941
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Propagate stateDir to LocalAccess

We will need the top-level directory for persistent RaftStorage outside
of Pekko persistence. Make sure we make it available from LocalAccess.

JIRA: CONTROLLER-2134
Change-Id: I7c8130cc89a024f8d276a988787d32e9b0c93b53
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move base.messages.CaptureSnapshot

This class is no longer being sent to actor, but rather is a plain
holder used mostly by SnapshotManager.

Move the class into SnapshotManager, hiding its constructor and dropping
the ControlMessage part.

Change-Id: I8bd223bf4b6c16f2a9afc5c52bdf30c096ecbeb8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Clean up Leader

Use local variables to squash nullness warnigns and make
LeadershipTransferContext properly constant.

Change-Id: I50e48092d4f3d36b7ce8fee6df79cfa6c67317e2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve sal-akka-raft assertions

Use assertInstanceOf()/assertSame() instead of referencing the
behavior's role.

Change-Id: I58d65fea5676b95f76ca00b40d5a9e44a33d8dfe
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rewire SnapshotManager.captureToInstall()

Do not may a roundtrip through actor messages, but rather use the
facilities provided RaftStorage to perform off-loaded serialization.

This eliminates the offload actor from sal-distributed-datastore, as
well as the need CaptureSnapshotReply, as we now message completion via
executeInSelf() and callbacks.

JIRA: CONTROLLER-2134
Change-Id: I0518e7eb80559382ede5b4b4e7db41c66fd6a564
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add DataPersistenceProvider.streamToInstall()

This is the first step in having asynchronous access to a snapshot
bytestream: RaftStorage now exposes streamToInstall() method, which
writes out a snapshot in the background and invokes a callback once
that is completed.

JIRA: CONTROLLER-2134
Change-Id: Ic650dbf4b31c62135dbc7b28fa9682f1ef5bf824
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix RaftStorage startup

PersistenceControl fails to start enabledStorage, leading to all sorts
of mayhem. Fix that up.

JIRA: CONTROLLER-2134
Change-Id: I54023b01f83c7657846253da8e1f505fc52584f6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Silence PropertiesTermInfoStore

Log NoSuchFileException at trace(), reducing clutter in tests.

JIRA: CONTROLLER-2133
Change-Id: I40c1855c957ad82e84f3a02fcedb1167845dc28d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Allow TestDataProvider's execution to be adjusted

We have a use case where we would like to delay execution of
DataProvider callback. Ditch the shared instance and allow the executor
to be set.

JIRA: CONTROLLER-2134
Change-Id: I0e99bb51eb549f6db3e27f5fdc600f6c3c97f097
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Centralize MockRaftActorSnapshotCohort methods

We have 4 implementations doing the same thing. Use default methods to
reduce duplication.

JIRA: CONTROLLER-2134
Change-Id: I3969590e94b2d138ac665221d7d31b0f0ac3a064
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use ByteStateSnapshotCohort in SnapshotManagerTest

SnapshotManagerTest is dealing with ByteState mostly and
ByteStateSnapshotCohort can widely implement the deserializeSnapshot()
method.

Use ByteStateSnapshotCohort there, which will make further changes
easier.

JIRA: CONTROLLER-2134
Change-Id: I672f934ddaf31c8a5e2cfd5d17664fc8985c4858
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Propagate start/stop to RaftStorage

RaftStorage needs to be an active component to deal with async
persistence, synchronizing persistent content, etc. This patch adds to
lifecycle hooks in RaftActor to control lifecycle.

JIRA: CONTROLLER-2134
Change-Id: I4e225d68628c696b4bcc80b9fbb04fd7d157ee2f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add FileBackedOutputStream.Configuration

Encapsulate the two options and propagate them to RaftStorage for later
use.

JIRA: CONTROLLER-2134
Change-Id: I341b8ef3891f355c59167ba9235a960b7672bca6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use Path instead of String for temp directory

Let's be type-safe, so that things do not get mixed up.

JIRA: CONTROLLER-2134
Change-Id: I8f5400213fc3cf424d61d83379d90fabb192d58e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Clean up sal-clustering-commons pom.xml

sal-clustering-commons on longer needs checker-qual, remove that
dependency. We can also enforce modernizer issues.

Change-Id: I587ae6df660f122c63c983d663cce0b6c1b3bab9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Introduce raft.spi.ByteArray

This patch replaces uses of ByteSource with InputStreamProvider, of
which ByteArray is a convenient implementation of. This eliminates the
need to use yangtools.concepts.Either, as we are expressing the two
options via class hierarchy.

JIRA: CONTROLLER-2134
Change-Id: I249f572fe8be0e64de796fdf7d48cbf2ba9b95c3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rehost controller.cluster.io

FileBackedOutputStream is used to receive AppendEntries, which
eventually land in the RAFT journal. No ODL downstream is using this
facility, so it is fair to say that this is part of RAFT SPI.

This allows us to change cds-access-api's dependency from
sal-clustering-commons to raft-api. This is quite natural for the
datastore: at some point we want to expose a CommitInfo which contains
an EntryInfo reference to when the transaction was committed.

JIRA: CONTROLLER-2134
Change-Id: I7458e17055affd56b736f85476b004cb29a2b1d2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor SnapshotSource

Reduce the number of classes by introducing InputStreamProvider.

JIRA: CONTROLLER-2134
Change-Id: I261c714da7871aa8ed9e66524ee6783faaadec1f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Propagate SnapshotFileFormat to RaftStore

Each RaftStore needs to have a preferred file format. Hook it to
use-lz4-compression, hardcoding to 256KiB block size, just as we do when
we transfer to followers.

JIRA: CONTROLLER-1423
Change-Id: I7a59f386abc250fe7f813175650ad9374f4711f4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove InputOutputStreamFactory.lz4(String)

We do not have to use a String, just use the corresponding constant
directly. There is only one caller who can take ownership of the
corresponding code block.

Change-Id: Ie8bc8162f1cb47744e013fed1fee03f0af10fc03
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add RaftStorage.start()/stop()

Add internal thread pool and two methods to control it.

JIRA: CONTROLLER-2134
Change-Id: I87060447b86d7358f2f3cc5f9598168bd963058c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve ShardManagerInfo

Define TargetBehavior to offload type mapping to JMX.

Change-Id: If7d39639ae90aaca18807dfb116c795d5b047d18
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Document default backup-datastore timeout

We have an implementation-specific default of 60 seconds, let's make
sure it is captured in the YANG model.

Change-Id: I66020e33c73c770cbed8bf9a6a0c592bd272c5b2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move sal-akka-raft

We have a raft/ top-level directory, move sal-akka-raft there. Also
switch it from using mdsal-parent to using bundle-parent.

Change-Id: Idb597dafa423723443a5e6e344e9f2b06c8b1410
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rename findLatestSnapshot()

We really should be addint the interface to DataPersistenceProvider and
a 'tryLatestSnapshot()' is a better name.

JIRA: CONTROLLER-2134
Change-Id: I65af11051ff4cc473053186a7226ff6d052b98ac
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add SnapshotFileFormat

This is a useful utility, which we will use to build our
LocalRaftStorage.

Change-Id: I32027581f8eb55da435310aa35e0bbac7ab4436e
JIRA: CONTROLLER-2134
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add DataPersistenceProvider guidance

We will need to evolve this contract a bit, add more specific guidance.

JIRA: CONTROLLER-2134
Change-Id: Id1fcfb50fe96e7adfa8df089ac8dc9622240b770
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move TermInfo to o.o.raft.api

This is a natural raft-api thing. Move it there and cover it with tests.

Change-Id: If4dc97b144bedbbad81c21ba6fe795b24078e9eb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Promote (Immutable)RaftEntryMeta

These two constructs are raft.api material. Promote these two as:
- raft.api.EntryMeta as a replacement for RaftEntryMeta
- raft.api.EntryInfo as a replacement for ImmutableRaftEntryMeta

Change-Id: I93908e29f11ffad3342da7dc0f4a24678cecef35
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Split out raft-spi

We have a nice set of classes that could be more widely used. Let's
split them out. This has the neat benefit of making lz4-java an
implementation detail hidden from the outside world.

We also introduce odl-lz4 to package lz4-java independently of
everything else.

Change-Id: I3434bac53dba6935c022e9ab79fc40eead01e40b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Split out raft-api

We now have RaftRole to start a low-level RAFT API artifact. Introduce
raft-api, which holds RaftRole and its corresponding ServerRole. Use it
to improve type safety of ShardStatsMXBean.

Change-Id: I99eda413a9cac4c7fbae036512432a97630ef28c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Alpha-sort sal-akka-raft dependencies

Previous reformat has missed these, fix it up.

Change-Id: I54961bf1e42415974235d8e50fd539dea633433d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add SnapshotSource

We will need to deal with snapshots being available for reading in
multiple formats. This patch add the concept of a SnapshotSource, with
two formats: plain and LZ4.

This ends up being a framework, but that is completely fine, as it is
transparent and provides a fixed amount of functionality.

We plug it into RaftStorage, as that is where it is going to be needed.
The two RaftStorage implementations run a no-op, with TODOs for later
implementation.

JIRA: CONTROLLER-2134
Change-Id: I99a8a967e6d08da681fa45082927b3039a523a49
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Enforce sal-akka-raft dependencies

We have squeeky-clean dependencies, make sure it stays that way.

Change-Id: Icfedbdd3c037b35edbef87c70e4466b3da09d681
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove broken Export-Package declaration

This is a day-zero bug: 'Export-Package' is mis-spelled as
'Export-package' and this is ineffective.

Remove the declaration and provide documentation for the remaining
DynamicImport-Package.

Change-Id: I807768a908747bc13bb51cea7fd7c4cb50f3fe6b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove slf4j-simple dependency

This dependency is always added by our parent pom, hence there is no
point in repeating it here. Also reformat pom.xml to follow style we use
in most places.

Change-Id: I13d071fd2799bfb2256e6c8831ddbe2fb4e08648
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Require memberId() for RaftStorage

We need a consistent way of logging, make sure RaftStorage has it.

JIRA: CONTROLLER-2134
Change-Id: I67ca04c5c7e45358276775285ea05fd58fb7ce3e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Factor out RaftStorage

We are in a place where we can start pulling the persistence apart. This
takes the first step by introducing RaftStorage and dropping a number of
FIXME for future evolution.

JIRA: CONTROLLER-2134
Change-Id: I658ce51cff971e39dc53b8f25e42123ad5d78b3e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Reintroduce ForwardingDataPersistenceProvider

This is a neat utility which allows us to sit in front of persistence,
forwarding to it -- without exposing our real implementations.

While we are here, also improve testApplyStateRace by scheduling
callback invocation after the delegate is done with it.

JIRA: CONTROLLER-2134
Change-Id: I3fb4a423a1302a568bc5eecb77f26d2b5d444a4f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Clean up OnDemandRaftState

Use RaftRole and fix the builder pattern to be properly immutable.

Change-Id: I4630a5d920d545ca615e72dd7a0a8a017e8b517a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Promote RaftState

Let's start a new package, opendaylight.raft.api, which holds a single
enumeration for now -- RaftRole. The change in name frees up 'RaftState'
for use by something that actually has state.

This also shows us that we have a bug in Example actor -- using an
illegal cast in cast of IsolatedLeader.

JIRA: CONTROLLER-2134
Change-Id: I50b25b4d13cf426285face63ed8be2a8356ab83d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move cluster.notifications

Let's rehost these into sal-akka-raft for now, as that is where they are
used from. This allows us to tie 'role' with 'RaftState', leading to
improved type safety.

JIRA: CONTROLLER-2134
Change-Id: Icb4968774d89517486c50bff95393e8c68b8265b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Modernize RoleChanged/LeaderStateChanged

These are pure DTOs, use a record for that. While we're here, also make
sure to require memberId()/newRole(). The situation is a bit
complicated, because we use subclassing to carry more data.

JIRA: CONTROLLER-2134
Change-Id: I37a486dd75c1a313161908cfb287e61e18a5a401
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove RaftActor.currentTerm()

This method is not used anywhere, remove it.

JIRA: CONTROLLER-2134
Change-Id: I1ce49c91655086470f8c083629051bf5c8dbef40
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Introduce resetReplicatedLog()

A ton of our tests rely on replacing ReplicatedLog. This is a
huge-no-no, which is unfortunately exposed via RaftActorContext.

This patch splits of resetReplicatedLog(), which does the same thing,
except it has a different name and is amenable to being implemented
as a set of operations rather than a wholesale replacement.

JIRA: CONTROLLER-2137
Change-Id: I4774564b092fadbbd4f0f4a57f3d13e1234ec376
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove RaftActorContext.setCommitIndex()

Mass-migrate tests and eliminate the legacy method.

JIRA: CONTROLLER-2137
Change-Id: I6035aa1edf546d8d5ae519e5db3edd83ca632624
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove RaftActorContext.setLastApplied()

Mass-migrate tests and eliminate the legacy method.

JIRA: CONTROLLER-2137
Change-Id: I194b70975fe1cca97f3a81163410247d7a76e3c5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove RaftActorContext compatibility getters

Mass-migrate tests to use ReplicatedLog for commitIndex/lastApplied.

JIRA: CONTROLLER-2137
Change-Id: I9f30b3afb0b7df05ef6238944f79fcba050e1c3d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not reset ReplicatedLog

Replacing ReplicatedLog is a rather bad thing, as we need to be mindful
when we can and cannot cache it.

This patch takes the first step towards making it an invariant:
introduce a resetToSnapshot() method which resets the state and take
advantage of that.

We also deprecate setReplicatedLog() for removal.

JIRA: CONTROLLER-2137
Change-Id: Ifc9ce0b0910214d5d0f314f26623fc4583fd1849
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>