Use Empty for ON_TASK_COMPLETION_THREAD_TL

The object carried has no semantic meaning, but it has to be non-null.

Use Empty.value() instead of Boolean.TRUE for that function, which has
better semantic mapping.

Change-Id: Ice6a2e56676af73d12f22696995a4dc1f69f7b46
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Change ConcurrentDOMDataBroker.toString()

Use class identity without any properties for now.

Change-Id: I3694ba6139ebd7bbb466a6d4da56798b82877b18
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve DelegatingRunnable.toString()

Rather than pointing to an identity, which does not help with
identifying the runnable which is being offloaded, use ToStringHelper to
which exposes the delegate's toString() as a property.

Change-Id: I0752724fed249422e9d2ad66045101921cc7c2eb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use DefaultBindingDOMCodecFactory in test

Do not reference BindingCodecContext directly, but rather go through the
official DI endpoint.

Change-Id: I02fe67066a98b8f2c3ba58c4e28ce387e1cb982a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

eos-dom-akka should not use BindingCodecContext

Use DefaultBindingDOMCodecFactory as the intermediate to instantiating
BindingDOMCodecServices, which is what we should be coding against --
and it is not something that can be overridden.

We are also not using BINDING_RUNTIME_GENERATOR, so we remove it,
simplifying dependencies.

Change-Id: If69c45bc9f617c13644e11aad10cd326ec2c8ae9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump versions to 10.0.2-SNAPSHOT

This starts the next development iteration.

Change-Id: I859279e348aee10aaa131718591e81e544548f45
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Release controller

Bump upstreams

Adopt:
- yangtools-14.0.2
- mdsal-14.0.1

Change-Id: I44f99607d5aad87b2c9f91dd0d31f3e18fa90050
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump versions to 10.0.1-SNAPSHOT

This starts the next development iteration.

Change-Id: Ibd715336a00ef7e8bff7930dbb1c4d5a9a3313ec
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Release controller

Remove unused dataCenterControls

Remove unused reference DataCenterControl from OSGiClusterAdmin
constructor/OSGI activator.

Change-Id: I74677c8af750f898b0fc15ad8ec726493111eac1
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>

Pick up mdsal-14.0.0

Use release version of mdsal-14.

Change-Id: I4f22829645fd7046b5bc37f9186f5362badec433
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstreams

Adopt:
- odlparent-14.0.2
- yangtools-14.0.1

Change-Id: I389c0c9bc679202226122dd52c9574b43127c059
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Adopt released yangtools

Adopt:
- odlparent-14.0.1
- yangtools-14.0.0

Also adjusts to slight movements in mdsal, restoring the ability to
build.

Change-Id: I7c5dfa41448c63e009c2bb5aedecbc8c34640bd0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstreams

Adopt:
- odlparent-14.0.0
- yangtools-14.0.0-SNAPSHOT
- mdsal-14.0.0-SNAPSHOT

This patch uses yangtools/mdsal SNAPSHOT versions to do initial
interation.

Change-Id: I56bca3b5c105fbf77c90e03965a96b2d13fffdac
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove use of impl.schema.ImmutableNodes

Migrate the handler to eliminate deprecation warning.

Change-Id: I35f5d81b42ffdb7cdca476885158ef8f88ebc230
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not use EventListener/ListenerRegistration

This is a useless marker, drop it and use ObjectRegistration instead of
ListenerRegistration.

Change-Id: Iec4b9591e324e549ec5600c2c9d0075b8d2836fb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Separate out RaftEntryMeta

RaftEntryMeta captures entry index and term, without the data portion of
a RAFT journal entry.

While we are modifying things, also make sure to minimize data hunting
by remembering things, like the replicated log.

JIRA: CONTROLLER-2121
Change-Id: I9a107077fb965ef8921f62a58cce4aec69e85185
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump versions to 10.0.0-SNAPSHOT

This starts the next major development iteration.

Change-Id: Ie4a93f0070b979cadd2a9cf76d1671d7b982dc08
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Modernize AbstractLeader

Use List.of() and access first element only once.

Change-Id: I718f09a90e7c8c56362e0503c927ed5fdb842e34
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstreams

Adopt:
- odlparent-13.1.3
- yangtools-13.0.6
- mdsal-13.0.4

Change-Id: I73c9abdbf1db0e71b3d5ee48cf470cfe272edfb7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump versions 9.0.4-SNAPSHOT

This starts the next development iteration.

Change-Id: I9f4f637c14bf0da54ff2512095195d98b4d17703
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Release controller

Separate out raft-journal

Use a dedicated artifact to hold the baseline interfaces, so that we can
evolve it later without the Kryo baggage.

JIRA: CONTROLLER-2115
Change-Id: I7d6b0f8625078c816d9f99f8748e0b1bc3fcd092
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rename ByteBufJournal

Split out the base interfaces into controller.raft.journal, allowing us
to split out the implementation as well.

JIRA: CONTROLLER-2115
Change-Id: I429872d92da6b4044b393e2a0d5e05f5c9823ee4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Separate out {From,To}ByteBufMapper

We have overlapping method definitions on the read side. Let's split up
ByteBufMapper into two interfaces so they can be reusable.

JIRA: CONTROLLER-2115
Change-Id: I9c897a83e15818a9ca5269cf2b22d725d5d706b9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move {Journal,ByteBuf}Reader.firstIndex()

Similar to lastIndex(), firstIndex() is a journal-level thing. Move it
out.

JIRA: CONTROLLER-2115
Change-Id: I7257194d65dbd47960bf6be4af12ea56056d3c9f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Split up RecoveringClientActorBehavior.onReceiveRecover()

We will be expanding logic here, split the methods up so it is clearer
what is going on.

JIRA: CONTROLLER-2116
Change-Id: I6dbfb6fb6106c2e342e448b714b708c1e4e65d04
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move InitialClientActorContext allocation

The context is inherently tied to RecoveringClientActorBehaviour,
allocate it there.

Change-Id: I37a0c9d8ee2a187256b200035b320bd2f6b86c0b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Free disk buffers

We are allocating the buffer via a ByteBufAllocator, which may lead to
leaks is we do not release() the buffer.

JIRA: CONTROLLER-2115
Change-Id: I8a1aff9cd7ef1a3cd5937a5ac481a3c188a6c13f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Make Netty-3 dependency optional

Akka is using an long-obsolete version of Netty in for classic remoting.
While this is not a real problem by default, as the user needs to
configure it for the classes to be loaded, it is tripping up static
analysis tools.

This patch makes the org.jboss.netty.* import optional and removes our
packaging of Netty-3. This means that Akka Classic Remoting cannot be
configured unless the user also packages Netty-3 themselves.

JIRA: CONTROLLER-2119
Change-Id: I0a1bf611f140f2d5b6b83700893d0d69a3ba6115
Signed-off-by: Peter Suna <peter.suna@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump Scala to 2.13.14

https://github.com/scala/scala/releases/tag/v2.13.14

Change-Id: I4a0c0fafa504d3a164f2e08a4df92f1f538d7e91
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add MappedByteBuf

Add MappedByteBuf and switch File{Access,Reader,Writer} to use ByteBufs
as their lingua franca. This reduces friction between the internal APIs
and the user-facing ones.

JIRA: CONTROLLER-2115
Change-Id: Id05744378e883cbfbf386407945a64e34282e213
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Enable checkstyle

Fix the few remaining warnings and enable checkstyle.

Change-Id: I6eaeaefe921633d76dfa6283d0685a3c53320b95
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Separate out FileAccess

Rather than keeping mapping in MappedFileWriter, use MappedFileAccess
to keep its lifecycle.

The second part is keeping the information about last position, which
we encapsulate in JournalSegment.State and its two specializations.

Also move ByteBuffer allocator to DiskFileAccess, as it really is
specific to StorageLevel.DISK.

JIRA: CONTROLLER-2099
Change-Id: I7fba8da21a021a477c965d96045c1b0d4bf8cc29
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Clean up Segmented(ByteBuf)Journal

Remove SegmentedJournal.Builder(), as it really has no point. Also
expose the compact() method and restore maxEntriesPerSegment.

JIRA: CONTROLLER-2115
Change-Id: I7213cdca037d45ad9f15f2e577bd8cb1e6b75156
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve ByteBufMapper.objectToBytes() contract

We currently treat each IOException as if it were caused by an overflow,
but other causes may exist.

Differentiate between EOFException and plain old IOException, the former
indicating an overflow.

For JournalSerdes/KryoException examine the message as there is no other
indicator.

JIRA: CONTROLLER-2115
Change-Id: I58f0de1fdb5652f0887502f610cce42395d14cae
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move entry serialization back to ByteBufWriter

I8f6bac3192a0f38b627150be4c8ea128f1e233e5 moved serialization to heap,
causing unnecessary copies, while nominally simplifying the interface.

This patch undoes that move, restoring the logic, except working on top
of a ByteBuf. This requires a bit more logic to deal with the fact we
are no longer writing to the diskEntry nor are we flipping it.

JIRA: CONTROLLER-2115
Change-Id: I1d18f99cfdb5b7e6c6548a5833c824af9f31c166
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Merge SegmentedByteBufWriter.checkedTruncate()

This method has a single caller, just merge them together.

JIRA: CONTROLLER-2100
Change-Id: Ibf9424e71c9c1dd9e862b00a75d4ed76cd29cc0d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Reformat ByteArrayOutput/BufferAwareByteArrayOutputStream

These are simple classes, just reformat them to keep checkstyle happy.

Change-Id: I7e0331a3bd8b4373921bafb29da6362e02e57f19
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix checkstyle in Kryo{IO,Input,Output}Pool

Reformat and make minor adjustments to keep everyone happy.

Change-Id: I32371650660080adfac970d9530abb04609de7f6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Suppress warnings in Kryo505ByteBufferInput

This is legacy third-party code. We will not be fixing its huge amount
of issutes.

Change-Id: I6599301b4124e96fac910a98ac634c94f5c5aa11
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Modernize tests and fixup checkstyle

Reformat for checkstyle compliance and modernize with JUnit5, etc.

Change-Id: I6fab572f429578e6be924f6d8bcffcc6d083d333
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use eliminate magic JournalSegmentWriter constructor

Rather than performing internal initialization, move index entries to
acquire current position. This way the writer is always properly
initialized.

JIRA: CONTROLLER-2100
Change-Id: I4c6459d31438379d58dc1c8e769dca368565eaf5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not copy bytes when deserializing

ByteBufUtil.getBytes() performs a deep copy of the contents of the
buffer. This is completely superfluous, as we can use the underlying
nioBuffer() directly.

JIRA: CONTROLLER-2115
Change-Id: I0a9f0ec7b22682cd41063db93921c159ba66b082
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve TestEntry

Implement hashCode()/equals() for comparison clarity.

Change-Id: I90fd7113dfa3f900d61a99fac6989e6df9014664
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Utilize segment index to recover writer state after truncate

index.truncate() returns nearest indexed entry position which
can be used to minimize number of loops required to recover
writer state - current position and last written entry.

JIRA: CONTROLLER-2100
Change-Id: I6875c1697a2ac5f13b82b256850f293a7658f220
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>

Remove JournalWriter.truncate()

The truncate(N) operation is completely equivalent to reset(N + 1).
Remove this duplicity, preventing potential confusion.

JIRA: CONTROLLER-2100
Change-Id: If62e55edfd610c87174a6b83a15c0a1a0ab7836c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Clean up SegmentedByteBufWriter.reset(long)

The logic in writer reset needs to be a tad updated:

- it makes absolutely no sense to seek past last written index, unless
  we want to magically sync across multiply-open journals. We do not
  support that
- we need to realize when we are already at the correct index

JIRA: CONTROLLER-2100
Change-Id: I20b2a084c782069b893059f8d5bf535ecbbf33e3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Move JournalWriter.getLastIndex()

Last written index is a property of a particular Journal, not of a
writer -- and now that we maintain this in the index, we can make
shortcuts.

This also removes a source of confusion, as we have two methods taking
a 'long index' and performing some writer adjustments:
- reset(long) is equivalent of setNextIndex()
- truncate(long) is equivalent of setLastIndex()

Change-Id: I1bc4b5d1b3052c2b35808b8ec4ea2d88dcfca593
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Maintain last known position in JournalIndex

A JournalIndex is bound to see all entries that get written into a file,
hence it can easily maintain that Position. This simplifies
JournalSegmentWriter state maintenance, as we can consult the index for
JournalSegmentWriter.get{Last,Next}Index().

Also expand test suite to make explicit assertions on returns.

JIRA: CONTROLLER-2100
Change-Id: I57731f53cacfdfd548ab59c06706d41a013e7908
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor SegmentedJournalWriter.reset()

We have two methods reset() and truncate(), both of which take an index,
without a real documented distinction.

Document reset() as taking the index to read/write next and instantiate
a guard against attempts to use reset(0) -- as 0 is not a valid next
index.

JIRA: CONTROLLER-2100
Change-Id: I8a6b366fdb0827ab3cd5a494e7e9f5a741983264
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix segment population

There has been a mixup in previous patch, which ends up touching the
segments from loadSegments(). Fix that up.

JIRA: CONTROLLER-2115
Change-Id: I7932e691c9751a79ab2b407f9de4049721680c6d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not call nextSegment() from {first,last}Segment()

We always have at last one segment and nextSegment() is the slow path
here (requiring synchronization). Just assume {first,last}Entry()
returns non-null.

This assumption is violated while we are removing a segment, as we could
be removing the last segment. In that case we need to re-create it.

Also improve method names and documentation a bit.

JIRA: CONTROLLER-2115
Change-Id: I74bb1578e73828666ee795522a68f14ad112ec75
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use getCompactableIndex() to unmask firstIndex

We have duplicated code which can easily use the result of
getCompactableIndex() and work on top of that.

JIRA: CONTROLLER-2100
Change-Id: Ib580853424d445d82c448a86d82a706f4bba50d2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor Journal interface

There are two basic issues:
- isOpen() is completely unused
- close() is inherited from Closeable, not AutoCloseable

Address these and move implementation of openReader(long), so it
is canonical.

JIRA: CONTROLLER-2100
Change-Id: I1468ed5a3e9ee1abefe35a4bfaf653696763907f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Separate byte-level atomic-storage access

Byte level functionality was moved into *ByteJournal*
artifacts and now can be accessed independently.
SegmentedJournal is now acts as a type serialization
layer on top of ByteJournal.

// FIXME: refactor SegmentedJournal.Builder (in a subsequent patch?)

JIRA: CONTROLLER-2115
Change-Id: I2e4941bda3af76f0cd59e8c545131af85c668010
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Deprecate ByteArraySerdes

This is another serdes, it should be deprecated.

Change-Id: I6eab2a136cb1bc3f1f4629561752d8abcb26db56
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Deprecate more of serdes

The JournalSerdes is deprecated for removal, as are all the classes
directly related to it. This patch applies the second part of that
sentence.

Change-Id: I0ddb7293dfef539526ecb46598db7e194bdaf8f1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add JournalSegmentFile.map()

We already provide a hub for I/O operations. Mapping a file is one such
operation.

JIRA: CONTROLLER-2099
Change-Id: I4fba610eac00739691849454bb99f99796c24e9c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Centralize IO buffer allocation

JournalSegmentFile.maxSize() goes into the algorithm to choose I/O size.
Let's move the method there.

JIRA: CONTROLLER-2115
Change-Id: I9f0810cb9769217965375ee70c7e3e231adbf1d9
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Add SparseJournalIndex.toString()

Improve developer experience by noting positions.

JIRA: CONTROLLER-2115
Change-Id: I1c8d9618ceec6775b22b3bd129ecbc9be8a412d1
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Centralize CRC32 computation

We have essentially-duplicate codepaths between
JournalSegment{Reader,Writer}. Move computation to SegmentEntry.

JIRA: CONTROLLER-2115
Change-Id: I776b693b6e88d84ddb99c274371ac694aa536d1d
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not expose descriptor from JournalSegmentFile

JournalSegmentFile should expose interesting mapping -- the descriptor
is really an internal thing.

JIRA: CONTROLLER-2099
Change-Id: I2276cee3c4cb15d961aaa1b7f72d618f10dd744e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Retain RandomAccessFile in JournalSegmentFile

Having a RandomAccessFile is nice, as it internally holds a FileChannel.
This makes our files stateful -- which is okay, as we still manage their
lifecycle via JournalSegment.

JIRA: CONTROLLER-2099
Change-Id: Id8305c74dbd881eaf52d84191c11bb4ea2bc164b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Centralize JournalSegmentFile creation

There are only two callers of the constructor: we are either creating
a new file (in which case we initialize the descriptor) or we are
opening a file (in which case we read the descriptor).

JIRA: CONTROLLER-2099
Change-Id: Id3943a637067d3b7f2ba7384e417e935df65276a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve JournalSegment lifecycle

Disconnect JournalSegment from AutoCloseable, making its close() method
its own thing.

This makes it clear there are really two lifecycle entry points:
- close()
- delete()
Callers of delete() always invoke close(), so let's make sure delete()
does this internally.

JIRA: CONTROLLER-2099
Change-Id: I64b3c12f7a3301e513a36b8a156d8495a405313f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Expand JournalSegmentFile semantics

JournalSegmentFile also includes a descriptor, making it more useful
than just a File/Path holder.

JIRA: CONTROLLER-2099
Change-Id: Ia808512b7c3012bdb8e749429ac4b785643fa6c7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor JournalSegmentDescriptor

Converted JournalSegmentDescriptor to record, unused fields
removed. Also eliminated extra read of descriptor on initial
segment load.

JIRA: CONTROLLER-2107
Change-Id: I5ef45cd2bebfafe0a9dcc9838e056d9ed0bf7958
Signed-off-by: Oleksandr Zharov <oleksandr.zharov@pantheon.tech>
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Split out JournalSegmentDescriptor.readFrom()

We have a user which reads a one-time representation from a file. Split
out that use case into a dedicated factory method.

JIRA: CONTROLLER-2107
Change-Id: I11c335ce03aa59c7796d9c2c8e78801c5b2754c0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Reformat JournalSegmentFile

It is hard to navigate this file, which seems to be a rather pointless
class. Reformat it before we decide what to do with it next.

Change-Id: I33688e934e9860ae3945cb459e4026d6233bd0af
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstreams

Adopt:
- odlparent-13.1.2
- yangtools-13.0.5
- mdsal-13.0.3

Also adjusts for the removal of DOMActionResult.

Change-Id: I6a57d7b612203bfea2291432cec4946f11abd078
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Inline isServerConfigurationPayload()

We can use an instanceof pattern to improve safety. Inline the check
into its two callers.

Change-Id: Ia3758a754713818587bf3c7cd215764d05c2028b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not access payload data when not needed

We are just putting out a debug, make sure to make this operation
conditional.

Change-Id: I79c39aab6bb6c94efa03e5a878ceabdb552d93cd
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Improve segmented journal actor metrics

Update write time marked on actual flush not on flush request.

JIRA: CONTROLLER-2108
Change-Id: I92a66ae775cbae6aeea69bddf654df741f473dbd
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove JournalWriter.getLastEntry()

We really do not want to retain a written entry -- and there is only a
single real user. Remove this method and update its sole caller.

JIRA: CONTROLLER-2115
Change-Id: Ibea3d820fa7d1daa023be1cdcc1a513720744b6e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>

Remove bierman02 terminology from dev guide

Updated dev guide terminology after bierman02 removal.

JIRA: CONTROLLER-2088
Change-Id: I6e77b54c4c2fe1629b4a1014944c26110735a075
Signed-off-by: Oleksandr Zharov <oleksandr.zharov@pantheon.tech>

Use Netty to clean mapped buffers

Netty offers PlatformDependent.freeDirectBuffer(), use that to clean our
buffers.

Change-Id: Id737c9e951bc1c77376133d7053287e2840bafc6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Refactor JournalReader.tryNext()

We are usually translating an entry to an internal representation,
encode that in the API signature.

Since we have an API method to carry Indexed.* fields, we reduce our
reliance on this DTO. This leads to more efficient advance during reset,
as we do not need to instantiate Indexed instance.

JIRA: CONTROLLER-2115
Change-Id: I1d399e14997fb93b100397afab785778e47f985b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove JournalReader.getCurrentEntry()

JournalReader.currentEntry is just a single-entry cache which is not
used anywhere but tests. Remove the public method and private field.

While we are at it, make SegmentedJournalReader.tryNext() state
index management more obvious.

JIRA: CONTROLLER-2115
Change-Id: If92293e8393e71a4c1f402a0370973872a9593ac
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Cap index to sequenceNr before tryNext()

Rather than relying on Indexed.index(), which requires us to attempt to
read the next entry, use a simple getNextIndex() check.

JIRA: CONTROLLER-2115
Change-Id: Id1d7638dda0d909b8383af8f1e201fb9e7f192e0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Factor out FileWriter interface

We have two specializations of JournalSegmentWriter just to provide
specialized write access. Factor out the require interface into
FileWriter and specialize that instead of JournalSegmentWriter.

JIRA: CONTROLLER-2100
Change-Id: Ie6a2497f49d7e1e00f32836cccbf0680b17b15e3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>

Segmented journal benchmark

Performance test for sal-akka-segmented-journal moved
and refactored to standalone benchmark module, making
self-contained executable jar; 'benchmark' profile
executes benchmark via maven build using current
configuration (taken from sal-clustering-config)

JIRA: CONTROLLER-2114
Change-Id: I108f7aa99b089bdc1ae12b99fcc6ff4bf53bbd38
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>

Fixup checkstyle

Updated checkstyle is picking up a few new warnings. Fix them up.

Change-Id: Ib5fa92b84c7e4570098a1bd1163f0947a86ed7a9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

atomic-storage: remove type dependency at segment level I/O

Due to object de- and serialization was served by segment
I/O artifacts it required these artifacts holding serializer
instance and being dependent on its type (generics).

Moving serialization to upper level (journal) allows segment
reader/writer operate on byte level only with no extra data
conversions. Serializer interface simplified for further
isolation and removal of Kryo serializer.

JIRA: CONTROLLER-2115
Change-Id: I8f6bac3192a0f38b627150be4c8ea128f1e233e5
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix followerDistributedDataStore tear down

Fix closing of followerDistributedDataStore in
DistributedDataStoreRemotingIntegrationTest#tearDown.

Change-Id: Iba35cae665f29a9da0430baff1f792191a4d1287
Signed-off-by: lubos-cicut <lubos.cicut@pantheon.tech>

Bump versions to 9.0.3-SNAPSHOT

This starts the next development iteration.

Change-Id: I4263aa380442f375497222c0c2050e2a585c6adf
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Release controller

Defer up to maxEntrySize flushes by default

Batching flushes has performance benefits in terms of throughput.
Keeping the flush size at maxEntrySize makes the flush times more
consistent, as the journal's syncs will be amortized to fit writing
maxEntrySize'd entries.

JIRA: CONTROLLER-2108
Change-Id: Ie385738f65d9503fdeeed6a9e0b5ced37fde7fd3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not use BufferOverflowException for EOF signalling

A neat benefit of well-structured code is that it makes it clear what
the contracts are.

In this patch we eliminate a source of BufferOverflowExceptions and opt
to report the fact next segment is needed via a @Nullable return.

JIRA: CONTROLLER-2100
Change-Id: I1b9535053561709dc2a4b60446cd7cabd19da659
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Centralize JournalSegmentWriter.append()

We have almost-identical implementations of append(), differing only
slightly in how the do buffer management.

Expose a ByteBuffer-based API to writing to segment file, in terms of
abstract methods and pull the implementation down into the superclass.

JIRA: CONTROLLER-2100
Change-Id: I8d5629019d1c5d2978b0861e30e80346fbfc3c31
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rename JournalSegment.index()

'index' is an overloaded term. firstIndex() is much more descriptive
than 'index()', which is easily confused with public API argument names.

JIRA: CONTROLLER-2100
Change-Id: I7848a6bb0e1e7b4a1c7f63c8e6db5b02bd0dab76
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Unify JournalSegmentWriter.truncate()

Implementations of truncate are practically identical, merge them into a
common method. Also eliminate JournalSegmentWriter.firstIndex, as we can
get it from the segment at any time.

JIRA: CONTROLLER-2100
Change-Id: I019978461b67693f109cd6e90c8fba2ed8b18a3f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Unify JournalSegmentWriter.reset(long)

We have two distinct implementations here. The version for
StorageLevel.DISK is generic enough to work for StorageLevel.MAPPED, and
will guarantee consistency with reader implementation (which is already
shared across StorageLevels).

JIRA: CONTROLLER-2100
Change-Id: I3e9a5c8a72be766431e5cb527cae1f4809964dd7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rework MappedJournalSegmentWriter position tracking

Use an internal field to track the position. Also align entry reading
logic by slicing the appropriate buffer.

JIRA: CONTROLLER-2100
Change-Id: Iaa36fb32d9ffb7b215bef7829f8d3e7e5728e4b9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix SegmentEntry license

All this code is Apache-2 licensed, fix the header.

JIRA: CONTROLLER-2109
Change-Id: If0604f566c43a844da4522c2274a3b1a9f308a03
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Fix reader/writer buffer sharing

DiskFileReaders should share the buffer with with their writers.

JIRA: CONTROLLER-2109
Change-Id: I79203b7111827cd1b9885956fc6ca98bf621a12a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Shorten currentPosition

Our current file position is limited to 2^31-1, hence we can safely use
an int instead of a long. This is what StorageLevel.MAPPED needs anyway.

JIRA: CONTROLLER-2100
Change-Id: Idddea88cdc031f515675a39afb64115ded39fe0b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Allow segmented journal to flush periodically

Flushes to disk end up dominating our use of disk resources, as we issue
a flush after each write. This is not entirely efficient, as we may have
multiple outstanding writes in the actor queue -- and we ignore the
batching opportunity.

This patch makes it possible to configure an upper bound of the number
of outstanding bytes written which can remainin unflushed.

We flush whenever we reach this watermark or when we flush all messages
that have been submitted at the time the flush batch has been started.

JIRA: CONTROLLER-2108
Change-Id: I6f18de7871c89b5feffecc71580e1f440024f2a3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Rename patched ByteBufferInput

Hide the patched version and renake it to Kryo505ByteBufferInput.

JIRA: CONTROLLER-2109
Change-Id: Ie2b50dab13f1ae93e343f18acf4a05a890616576
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>