Unify Disk segment reading

DiskJournalSegmentWriter contains code duplicating DiskFileReader. Let's
remediate the situation by having an internal DiskFileReader, through
which we access the file in a read-only manner.

JIRA: CONTROLLER-2109
Change-Id: I0a5ffd1aebbea71513f6cd70e2138562755316fa
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 130476cd353b68f05e5dc26f64eade38fc1c68e1)

Add almost-verbatim kryo.io.ByteBufferInput from kryo-4.0.3

We are about to address a deeply-seated problem. Import souces almost
verbatim. Changes:
- inline MAX_SAFE_ARRAY_SIZE
- different package
- CR/LF changed
- javadoc corrected

JIRA: CONTROLLER-2109
Change-Id: I56cbaa7e541d9e78025095cb86effae8f97883c6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 4d1e330341bcd0788aca06505426e61ba25e6e1e)

Lower reader buffering

We are choosing buffer for two entries. This is a legacy choice, when we
could not compact our buffer and is actively hostile to large
maxEntrySize.

At the end of the day we really need space for one full entry. Since we
are being smart re. buffer management, we do not need the doubled buffer
size -- so allocate just what we need.

One exemption is when the buffer would be less than 8KiB -- in that case
just do not bother and cache up to 8KiB as needed.

JIRA: CONTROLLER-2109
Change-Id: Ic03df7e271e59588608f07c097b3831db83d7bd7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 5d2fc1dda3869fd05bbc231f1de1791bba65c7c0)

Factor out FileReader interface

As it turns out, JournalSegmentReader's specializations are providing a
very simple API they provide via abstract methods.

Extract this API into FileReader, allowing JournalSegmentReader to
become a final class, reducing mental requirements to understand what is
going on.

JIRA: CONTROLLER-2109
Change-Id: I4199f10a9483b18ac381f16e2b0d818f2d74ac16
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 323f56b4c4da7a19313dff1e35a60cf5e18f5942)

Remove JournalReader.getCurrentIndex()

This method is used only in tests and has the weird 'or 0' return.
Refactor test asserts and remove this method.

Change-Id: If397f2398e75d08de867cf4c674a1dd14e8add30
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 54cfea6258d62114fea95acdc20514dbad079c2a)

Refactor JournalSegmentReader

We have almost-duplicated code in {Disk,Mapped}JournalSegmentReader,
which looks very similar to what DiskJournalSegmentWriter revolves
around.

JournalSegmentReader revolves around tracking current position and
interpreting bytes read. Its specializations provide the means for
access.

JIRA: CONTROLLER-2109
Change-Id: I80b78cde5baf6b222d888e635dd5c854331f261a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 7f3eb922fce09a366143387ee71bf6f10b698473)

Do not use ByteBufferInput

ByteBufferInput ends up calling ByteBuffer.put() in readAscii(). Work
this around by integrating through InputStream. Not exactly nice, but
the best we can do right now.

JIRA: CONTROLLER-2109
Change-Id: I182c3cefb97ada0cd9fdc2b7440256f38341df5a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 9d980ed146bc4a7b5eede803b9d2450afb3c421f)

Move entry tracking to SegmentedJournalReader

We have duplicated state tracking just because JournalSegmentReader used
to provide the same API as SegmentedJournalReader.

Rehost tracking the next index and current entry into
SegmentedJournalReader, making it more obvious as to what is going on.

This allows Indexed to enforce the entry being non-null, as we do not
have to use a fake entry anymore.

JIRA: CONTROLLER-2109
Change-Id: I9ebd50fde94d695bb62315bba1b6adb1235b37f1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 88a1e0bc7269500b63f519e20145ae6f751485f1)

Simplify JournalSegmentReader entry access

The only user is SegmentedJournalReader, which is using these methods to
implement its tryNext() operation.

Mirror this with JournalSegmentReader.tryNext(), which simplifies some
of the logic.

JIRA: CONTROLLER-2109
Change-Id: Id104743b16199b4734b322d1397b85280f03346b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 926877f74ff564663d1e892cbad4c13a25e7e373)

JournalIndex.truncate() should return last entry

The only time we are calling truncate(), we are following up with a seek
to the appropriate entry.

Return the last indexed position from truncate(), so we can do better
than a linear search.

JIRA: CONTROLLER-2109
Change-Id: I701b0eefbf1f4d83f74ba95b93c41407289d95cb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2a6738b69c763377d1f727da9876ffdd9058262f)

Move ENTRY_HEADER_BYTES

We have SegmentEntry.HEADER_BYTES instead of ENTRY_HEADER_BYTES.

JIRA: CONTROLLER-2109
Change-Id: I4bc66155dccd3e177a61bad28be50e0bc62827ab
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 479ecb0109aaa1c8682dd083600b777cbd68dc07)

Introduce SegmentEntry

This is a helper DTO to communicate CRC32 and a buffer slice during
DiskJournalSegmentWriter's indexing.

This allows us improve the reset(long) loop to a manageable size, which
is mostly self-explanatory.

JIRA: CONTROLLER-2109
Change-Id: Ife1bddaf7684a304a0e7cdfd24ca9e774ac5143d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit c5e8613d497b97e32484b96f4bf1761cddc0d441)

Improve disk entry access

Eliminate superfluous compactions while a complete entry is available.
This is done by carefully looking at the buffer and figuring out how
many bytes we actually need progress.

JIRA: CONTROLLER-2109
Change-Id: If091b5c6f74da6a989629dae49137ed492725e2d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit ffa5f509ead53e7867ceeae9594e6b9906cf5da6)

SegmentJournal.log is static

This is derived from getClass() on a final class -- hence we can
amortize the logger into a static field, from whence it will be
constant-propagated.

Change-Id: Idcc6cf2d86c5ae687694fa5c16089687ac4a949a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b73237d7cfc9067dbeaed9dc7c602ee78884d002)

Bump versions to 8.0.6-SNAPSHOT

This starts the next development iteration.

Change-Id: Idf294275ab997cd6bd98b5196cb693267ec4b29c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Release controller

Bump upstreams

Adopt:
- odlparent-13.0.11
- yangtools-11.0.6
- mdsal-12.0.5

Change-Id: I169174144cff41d84774ea4ef6d8573198a3e3b8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Change FileChannel prefix to Disk

We use FileChannelJournalSegment{Reader,Writer} for StorageLevel.DISK,
and Mapped* for StorageLevel.MAPPED.

Fix this by a quick rename to DiskJournaSegment{Reader,Writer},
improving internal consistency and shortening class names.

Change-Id: Ia6d7311ed93527ff77cde3f7b57bb34689f962cd
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 1ecb4cd57462ff762de03ac89311cd4887ded651)

Add PANTHEON.tech copyright headers

These files have been significantly altered and hence deserve a
contributor copyright with a fresh year.

JIRA: CONTROLLER-2105
Change-Id: Ieceb00a54d204db5ba8bb92971a719f2b0815c59
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 3924193f0176e3f114a2dc82e45a49a684fe2375)

Normalized copyright header

Use a closed range and our standard 'and others' moniker.

JIRA: CONTROLLER-2105
Change-Id: I868b02415666da378f395ff711d5092a334d0196
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 046b4f0ce6d29eb651c12e85bd9907732b5fff06)

Futher improvements to ByteBuffer invocations

MappedJournalSegmentWriter should take advantage of fluent APIs of
ByteBuffer.

JIRA: CONTROLLER-2097
Change-Id: If12fe247bd08db450e00ea9878ecfafb7f75596b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2512ece32bd3452bf873176168711d1d270b2966)

Centralize entry header size

We jave 'Integer.BYTES + Integer.BYTES' strewn all across the code,
centralize this constant into a single place.

JIRA: CONTROLLER-2098
Change-Id: If15f944db20b01767ca387f652ddf11fed64e2fa
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 129be44b99a5e436beb30008552729b7f9583514)

Make JournalSegmentWriter.getNextIndex() final

We have duplicated code between the two implementations. Centralize it
to the common superclass.

JIRA: CONTROLLER-2098
Change-Id: I84b1a1d716c2ccbaaf2f75e3728bcbad0aa05ec6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b73ab4c9bbb2e80fc191d35c410a89a7774938dd)

Make JournalSegmentWriter.getLastIndex() final

We have duplicated code between the two implementations. Centralize it
to the common superclass.

JIRA: CONTROLLER-2098
Change-Id: I805e882566d8f891c23a1f25143d453c94284cd1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit f9f7739e90523361e38406d7df15cd99337219c5)

Improve ByteBuffer invocations

ByteBuffer has a fluent API, returning the same buffer from methods
which do not return something else.

Take advantage of this a chain invocations, resulting in denser byte
code.

JIRA: CONTROLLER-2097
Change-Id: Ic744e5e4f4d8c3fec5026f9d7edc58b7a5387c0e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d65837cffb7128b7a790eca9c73d5d8fabc3da57)

Improve FileChannelJournalSegmentWriter's buffer management

Minor refactor of logic in FileChannelJournalSegmentWriter class.
Removed unnecessary buffer clearing after each entry. New logic
remove already proccesed entry from the buffer and if needed reads
new data from FileChannel to it's capacity.

JIRA: CONTROLLER-2097
Change-Id: I571d531437b068cd468bdc92b3ab6a4223450a45
Signed-off-by: Oleksandr Zharov <oleksandr.zharov@pantheon.tech>
(cherry picked from commit bb9ef9945ec33f0025a4a426f44c416fb1e0531f)

Use correct JournalSegmentWriter for initial indexing

JournalSegment should look at StorageLevel and pick the appropriate
writer instead of always using FileChannelJournalSegment.

JIRA: CONTROLLER-2099
Change-Id: I060618a2808a837214b5ae61f4860f329c1799f0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 14a4cbf04ff2df99027bcaa6167e39394c4091ad)

JournalReader is not an Iterator

Iterator dictating a two method API, which is not quite appropriate, as
the writer could be manipulated between hasNext() and next() methods --
and next() has to perform validation again.

Introduce JournalReader.tryNext(), which returns a @Nullable Indexed --
either the next entry (as next()), or null (indicating !hasNext() case).

JIRA: CONTROLLER-2106
Change-Id: Ie0338a9869ece8e6381ae719a29e97fd569b442f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit cfe137aaad17ccc691f25ce517bf83a765216b63)

Catch JournalReader.next() violations in Mode.COMMITS

JournalReader.next() is specified as an Iterator and should be throwing
NSEs. This is not the case when we ask for only commits, as the
corresponding check is done only in hasNext() and thus we would give out
uncommitted entries.

Fix this by adding an override to check for this condition.

JIRA: CONTROLLER-2106
Change-Id: I1f391cc19b9674646786c93656b83519e0f1f47b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 86ff19a91c2d4aa71f88135898ebc75fd3caee85)

Use try-with-resources in testCompactAndRecover

Eclipse is flagging a warning here, make sure we silence it.

Change-Id: I7bfa9d490e8b9b71bf04511bc42d6d101345b179
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 10622ac62c756d5cacef0f32cb567a128712bbf1)

Cache reader's nextIndex during reset()

JournalSegmentReader.getNextIndex() involves some computation, so do not
call it twice when deciding how to reset SegmentedJournalReader.

JIRA: CONTROLLER-2106
Change-Id: I7527d8f0c47d35b7a59c30558715204a85b24eae
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 4aa75b385d864cff4409d47264453324b4150fa7)

Remove JournalSegmentReader.getCurrentIndex()

This method is only ever called by SegmentedJournalReader, inline it
there. This removes one source of the magic 0 index.

JIRA: CONTROLLER-2106
Change-Id: I0fcb16e806d027d22f8e6370788aab21884c9e71
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b428d29dff287a47dfacc5d02143ac62324169c5)

Modernize Indexed

Indexed.getType() is not used anywhere, hence remove it. While we are
here, also turn the class into a record, as it is a pure DTO.

Also add a FIXME to require the actual entry to be non-null.

JIRA: CONTROLLER-2101
Change-Id: I8bd42a815096a3d0993eb41c59cf39655a7ed0fd
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit dd05f95d0fe58515adddc0468a6e73d558a70f20)

Optimize SegmentedJournalReader.getCurrentEntry()

This method has very simple logic, the reasoning for which is left
unexplained.

Add a comment about what is going on and use assign-and-check construct
to trim a bit of byte code.

JIRA: CONTROLLER-2106
Change-Id: I5d4e5551ae089489636da38600109f7821fa306b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a73511c1db6a099397418d2708e073546e8e0a1b)

Optimize SegmentedJournalReader.hasNext()

hasNext() is an extremely hot method and its behaviour depends on
JournalReader.Mode, which is invariant for a particular instance.

Expose the invariant to the JVM by splitting the class into the base,
which services Mode.ALL and a subclass, which performs the commit index
check.

Also move segment acquisition and reader positioning to
SegmentedJournal, as otherwise we would be calling hasNext() from
constructor, which is getting flagged by SpotBugs.

JIRA: CONTROLLER-2106
Change-Id: I29fca69a211fc8f21619a54fe59a8ff4012b01d0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 87cf9d356ccaf6153ddbc0a763155c6787dc2932)

Fix JournalSegment{Reader,Writer} license

atomix-storage is licensed under APL, not EPL. Fix that.

Change-Id: I2a88fb978d0e1e9e22a5aecffcc4a7ca7fc2ae96
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 4daf276cc8c278a68f95828355ef22483b16d18d)

Refactor JournalSegmentReader.readNext()

The two implementations share the callout to getNextIndex(), pull it
down to the base class to better define the API we are after.

JIRA: CONTROLLER-2099
Change-Id: Id82a039dbb02156bef43ff9f9398d0e1883cc38f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 1880e76a77bfea77088107f241fc859248d3df1c)

Remove StorageLevel.MEMORY

This enum is defunct and results in DISK being used. Remove it.

JIRA: CONTROLLER-2101
Change-Id: Ibc6f6916f20e5b0fc0f3c8fe6651e65f24074d53
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b22f6f02b36c7de497848802e41387fe24e11b16)

Remove JournalWriter.append(Indexed)

This method is only used in a single test and can be realized via the
usual append(E) method. Remove it.

JIRA: CONTROLLER-2101
Change-Id: Ie1ec3507c18d9970be4968c9239b08ad8323f3a9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 42aa6677b87768cfc3a0f63539c5a6e6a2f509cb)

Clean up JournalSegment

We have plenty of unused and needlessly-public methods. Clean that up.

Change-Id: Ie93dd9eff7f847be23bf4d7fea9acefcac0beb60
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit fde1015a6e3a1c6c8aeecc81efe26fcdd92f81d5)

Hide SegmentedJournal{Reader,Writer}

These classes are define by Journal{Reader,Writer} interfaces, hide them
as they are pure implementation details.

JIRA: CONTROLLER-2098
Change-Id: I1149b970001cb64ffe503f128cce3864e396280a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 66dc95ca1b4cd88e8718086c6c902c9ad201286b)

Disconnect JournalSegmentWriter from JournalWriter

JournalSegmentWriter is really the internal API, so it should not have a
public API straightjacket. Disconnect the implementation, making method
properly package-private.

This also makes it very clear that JournalWriter is not something users
should close() -- because doing so would wreck internal machinery rather
thoroughly, rendering the segment writer inoperable -- and that should
only be possible via JournalSegment's lifecycle.

JIRA: CONTROLLER-2098
Change-Id: Ia18b0dc640cba11689bb36cf338c8cef3ae4e6f8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 45661c71a7e3d762ceb7ac12ced3c383ba65ac76)

Remove MappableJournalSegmentWriter

MappableJournalSegmentWriter is now just mediating implementation access
-- which is always is now guarded by a reference.

This means we can ditch this indirection and give out
SegmentedJournalWriter directly.

JIRA: CONTROLLER-2098
Change-Id: If15bdb10f9a317960aaf41aa80c57e929923ed21
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit cca2548d39eb1c398d98aeb0280c2e47b78852ce)

Rework JournalSegment writer access

The only user of JournalSegment.writer() is SegmentedJournalWriter,
which is using acquire/release to guard access to the writer.

Rework these methods so that acquire/release are internal and exposed
functionality is acquireWriter() and releaseWriter().

JIRA: CONTROLLER-2098
Change-Id: I345cec145fd82ee59c525d183584d69a9e599bf9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit ec6312137be61efb00cc03ef745bf6e0af31a0f8)

Disconnect JournalSegmentReader from JournalReader

JournalSegmentReader is the only used by SegmentedJournalReader, which
is the actual implementation of JournalReader.

There is really no point in JournalSegmentReader to conform to the
public API, as not all elements are needed.

This disconnects JournalSegmentReader, eliminating getFirstIndex(), as
that is simply not called -- and was not conforming to the API anyway.

JIRA: CONTROLLER-2098
Change-Id: Icd802ee07207d4f6787e3bdb71e36e553f2010c3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit f00cb3b9fb1237ee90b2c8bc07821213b4345fb0)

Eliminate MappableJournalSegmentReader

MappableJournalSegmentReader is an indirection used by
JournalSegmentReader to get a stable reader and the backend is
manipulated via map()/unmap() operations.

This is sort of a band-aid to lifecycle, as JournalSegment.close()
eagerly closes the writer, which in turn invalidates buffer and all that
jazz.

This dance is quite unnecessary, as JournalSegmentReader always acquires
the Segment, hence as long as it has a reader, the segment is guaranteed
to be mapped -- except for the case of close().

This patch reworks the logic in couple of ways:
- references are now guarding the writer itself, so as long as there are
  some, the writer is not closed.
- createReader() now increments the reference count, which means we can
  directly allocate the appopriate JournalSegmentReader implementation
- JournalSegmentReader.close() now routes to
  JournalSegment.closeReader(), which in turn releases the reference

JIRA: CONTROLLER-2098
Change-Id: I29b926dec5dc1a77e7e5e51ee3e2deb4f0ca6e33
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2e8a82c1b46bedf40e5e3d51f03efae75d72ca48)

Hide JournalSegment

JournalSegment is an implementation detail, which we do not want to leak
to the outside world. Hide it an make it final.

JIRA: CONTROLLER-2098
Change-Id: Ife71005f481c7ffb3c2eb6b89d3c5afa5e7f1609
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 08ea7b7900a387c0afb07664147d3c652dc1047e)

Clean up SegmentedJournalWriter

We have duplicated code in append() methods and their recovery is not
obvious. Move common code into a private method and reduce the size of
catch blocks.

JIRA: CONTROLLER-2098
Change-Id: I58e6ff19cf2eaa2e97c56499d70171cb93da6174
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit ac277cc850d78c906250263ca9ae196672fa438b)

Clean up SegmentedJournalReader

We have bit of duplicated code dealing with moving to next segment.
Concentrate it to a single method.

This shows the code actually has a bug, where it fails to close the
reader -- hence potentially leaking.

JIRA: CONTROLLER-2098
Change-Id: Ib58116d21cedde58c8f84d5fffd245e8e8408935
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit c23fa167527d507fe778e3ac838000edd9a81af3)

Do not store JournalSegment in JournalSegmentWriter

We are only accessing index() and maxSegmentSize. We already copy
index() to firstIndex, so let's do the same maxSegmentSize and use
firstIndex instead of accessing segment.index().

JIRA: CONTROLLER-2098
Change-Id: Ia8a24c7f73187a5803d279309775f361a39c91b0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 8442a2fb1750d67970672c484fa77f413692ff8c)

Inline SegmentedJournalReader.initialize()

This method is only invoked from the constructor, inline it there to
similify lifecycle.

JIRA: CONTROLLER-2098
Change-Id: I2c2790f785e13dd411a23eff299221b129a99630
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit fd16240a8173bf036906b3b815309382db1540b0)

Remove MappableJournalSegmentWriter.firstIndex()

This method is not used anywhere, remove it. This makes
MappableJournalSegmentWriter a pure delegator with map/unmap changing
the delegate object.

JIRA: CONTROLLER-2098
Change-Id: Ifcbc22edb0046ae890837415e9751a4a8274d4d3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2f27f3cc329eec9c6fdae8e2382fd25f115845d8)

Share FileChannel across all JournalSegmentReaders

Now that neither the writer not reader manipulates FileChannel state, we
can share a single channel. This has the nice effect of lifting its
lifecycle management completely to JournalSegment.

JIRA: CONTROLLER-2096
Change-Id: Ib7653d1494fdb53d1a1c73ad20ab103b29fedaa0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a9e7d69658a641eb49176078c64881f14add00a6)

Expand JournalSegmentReader

The two subclasses contain a lot of duplicated code. This patch
centralizes most code to reside in JournalSegmentReader.

JIRA: CONTROLLER-2098
Change-Id: I09a968ff8a283fc86f8591ab3d302cc2af0eb86a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 541edfced96299522aabbfd5c4b6123653efe038)

Introduce JournalSegmentReader

The two implementations for accessing the file have a common API and
potentially share a lot of code. Introduce JournalSegmentReader to act
as a common superclass.

JIRA: CONTROLLER-2098
Change-Id: Ie8b524812f6193206e1df100ac55ed4f11ad9f5c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 27d35a2e5f5db0ba2d637a86cbe888c3b874982a)

Perform proper handoff between JournalSegmentWriters

Switching between JournalSegmentWriter implementations is a matter of
just handing off state, which is limited to the last entry and position
in the file.

This patch introduces alternative constructors, which allow this state
to be moved to the new implementation without needing to re-read the
file.

JIRA: CONTROLLER-2043
Change-Id: I796cb4da4e09dfe6332ca82a7dfdb76740e30164
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit f9698667f48360df0d039e95db709fb140cf9e24)

Move JournalSegmentWriter switchover logic

This patch moves the logic to JournalSegmentWriter implementations,
which allows us to communicate internal state without the need to
re-establish it or leak it to the outside world.

JIRA: CONTROLLER-2043
Change-Id: Ifcb937fdaeaecd46b53a962c541ebfff689ecd40
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit d9569bc53baaf2e830144e0200f7b6baba15cbe0)

Introduce JournalSegmentWriter

MappableJournalSegmentWriter needs to flip between the two
implementations with seamless handoff.

Introduce JournalSegmentWriter as common superclass, exposing
currently-used interface.

JIRA: CONTROLLER-2043
JIRA: CONTROLLER-2098
Change-Id: I11f01ea027a757a32e25bf434d29abeca7d40f6e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b1773fc588ee5ea36f3bf5378ff54aa3dbffb64e)

Track channel position in explicit field

Rather than manipulating the channel, use an explicit field to keep
track of our current position.

JIRA: CONTROLLER-2095
Change-Id: I50adbef833251035f7c54ec624f9a03d59b5c7c9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 07f772e01ad41bfa6aacf037c2e19b2be87f3bc6)

Simplify FileChannelJournalSegmentWriter position tracking

This patch makes the obvious switch in state tracking: memory buffer
does not hold anything of value w.r.t. overall state, so there is no
point in marking/resetting and maintaining memory.position() beyond
what we need for reading from it.

Local 'position' acts now as the authoritative source of where we want
to go and we propagate it to channel position when we validate an entry.

JIRA: CONTROLLER-2095
Change-Id: Iab150a3c7348c714170223b397167cd4bb87f087
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 7dd119a473054c089c12e66780b136f11eb4f5d2)

Improve entry crc32 computation

We have a codepath difference here, where we use either a ByteBuffer or
a raw array to compute CRC32.

MappedJournalSegmentWriter does not make it immediately clear we use
this buffer twice -- once for CRC32 and once for deserialization.

Move acquisition of slice just after we have read the expected CRC32, so
it is clear it is something we would be doing even if there were no
checksum involved.

Mirror the same in FileChannelJournalSegmentWriter, as this will allow
us to further consolidate the code and stop mucking with memory.limit(),
which is causing us to invalidate our previously-set mark.

JIRA: CONTROLLER-2095
Change-Id: I355bd97cd8acb4f5d9d91310de97ecb2cbd70282
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b8dfbae75d8e13b78d3a7d5db48de2f5e262cd87)

Invert checksum check

We have a hardly-visible else branch which terminates our loop. Increase
its visibility by making it a dedicated if.

JIRA: CONTROLLER-2095
Change-Id: Ia0f6901793b1847a13afc5a3da3fdd5382180606
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b004038a282786fc29b4ba7fc9f8874debe21afd)

Do not use java.util.zip.Checksum

We are using this interface only for local variable type declaration,
use CRC32 directly, just as MappedJournalSegmentWriter does.

JIRA: CONTROLLER-2095
Change-Id: I054c639ae49b4fccef863a363c6e8665ea1322f6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit c6e60291d8e698572867a51dd0fbbf7f55b533b9)

Reduce position changes during read

Rather than using a read() which updates the position, requiring us to
set the position again, set it only once and use read with offset --
eliminating one syscall.

JIRA: CONTROLLER-2095
Change-Id: I478ba1cbb1f5737a751b43cb6ed0d018e49e6485
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit fe1081dadbe4aa27600ef4454bbad5357f50e331)

Clean up initial FileChannel writer reset's read

The code here bears marks of 'copy, paste and cudgel to compliance'
programming and is strikingly similar to what MappedJournalSegmentWriter
does -- except it needs to deal with reading as well.

This patch takes the first step towards sanity by eliminating a useless
if() and uses FileChannel.read(ByteBuffer, long) to reduce position
adjustments.

JIRA: CONTROLLER-2095
Change-Id: I528cb14608402a406e20d168a7e8520c9eb66edf
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 62351d41be04a723dce6c1268475f27a0ca0dfe6)

Optimize MappedJournalSegmentWriter.truncate()

Reduce code duplication by separating current position manipulation and
entry zero-out.

Also use a single putLong() instead of two putInt()s, eliminating one
range check.

Overall this makes the two implementations much more similar, allowing
sharing code at some later point in the future.

JIRA: CONTROLLER-2095
Change-Id: I9791b5e27a779210b5500a7923ccada7156f26e5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 5b9d16d22b8645011f8a79167da4d31a46195327)

Optimize FileChannelJournalSegmentWriter.truncate()

FileChannelJournalSegmentWriter.truncate() is inefficient in that it
zeroes-out its memory buffer and then writes it completely out.

This patch aligns it with what MappedJournalSegmentWriter does, which is
just zeroing out the entry header -- reducing both IO and CPU overheads.

JIRA: CONTROLLER-2095
Change-Id: I487157e78524cc9b00c9efff2064dbe892638ae4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 4a74d808e33fef2e002af1acc996b7a2ba228757)

Improve FileChannelJournalSegmentWriter.truncate()

Reduce the dance around channel position by explicitly passing it down,
thus reducing the number of syscalls we incur.

Also inline FileChannelJournalSegmentWriter.zero() and add a FIXME to
improve our zero-out strategy.

JIRA: CONTROLLER-2095
Change-Id: I885947fef51132a294c900396d902cf5481e2bd8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b2106c7c4fbc91f6cdfd8bf4371b723db6fdae2f)

Remove unused {FileChannel,Mapped}JournalSegmentWriter methods

We have a few methods which are not used anywhere, remove them to
simplify things.

JIRA: CONTROLLER-2095
Change-Id: I508d92cc2c761a6432e24e70db400636764753c8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit c15f343c9fdf01aaff0170c58398d0b8a7259822)

Optimize FileChannelJournalSegmentReader channel use

We use FileChannel to maintain our position for a particular reader,
which is a naive implementation.

Since FileChannel's position is tied to the underlying file descriptor,
each such manipulation involves locking and system calls.

Switch to maintaining the position ourselves and using
FileChannel.read(buffer, position), improving performance.

JIRA: CONTROLLER-2094
Change-Id: I7f462f3fd8aa9bc47c16201dd32faeb2a7c18a32
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2d9ad8ba93602b8b90b29a923a6d961b9cc42011)

Add RTD configuration

RTD requires configuration, add it.

Change-Id: I620a33a647f667c7b534f6b1ac19c50fe7dc6866
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit afdb785648021cd3143399d8551737880d2885ec)

Bump Scala to 2.13.13

https://github.com/scala/scala/releases/tag/v2.13.13

Change-Id: I365019424a8969ee87837c81c4022d9853e67539
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit dea84739bb45d034ac827d304bc525aed773c82d)

Enable requiremens in sal-distributed-datastore

The upstream issue has been fixed a few moons back, remove the
workaround we carry.

Change-Id: I5ed8030f98638f6202d4e9d67c309c704401840b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b794a34925a0e0a26496cdd5f7f777607ee8bd58)

Seal TransmitQueue

There should ever only be two implementations, seal the base class.

Change-Id: I2a511e49a5b29c89f5c3849972f298d98137462b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 2af83947ffd063f10811bb4de0d22a3c19378cac)

Migrate ActorBehaviorTest to JUnit5

Eliminate the need for mockito-subclass by migrating to JUnit5 and
correcting mocking configuration.

Change-Id: I3d3a772abe71e0d0576474aa9f2b22cef7fbea0a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a6874e13b031a6ef918affcc0942a0a78f09590b)

Deprecate EmptyExternalizable for removal

This class is not used anywhere, deprecate it for removal.

Change-Id: I3303a14570bfadaa524841e7622f20dd955f67e3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit b3616bb71b8dfb8aab44fcf25f0b3026487e04e0)

Deprecate SuccessReply for removal

This class is not used anywhere, deprecate it for removal.

Change-Id: I26b51aeda81ed9811171b53e9f797baa1c2bf994
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 7ffa5d19fcfb401db31dd5be595ff66492c0e106)

Do not use static import for Patterns.ask()

Improve call site visibility by explicitly referencing this method as
Patterns.ask().

Change-Id: I94f21ba9a5e2d0a2e3de89f6826896ea98a24641
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 9d48ffddd3e11efc5806efcc5895d5b5e997f5b2)

Use instanceof patterns in ShardManager

Reduce the amount of casts we perform by using instanceof patterns.

Change-Id: I918a15cf52f67ee072a1e7e88d5624bf161cdb3e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 96075120775de844c6863cd9cc16522cb87acddc)

Turn ShardContainer into a record

We have Java 17 now, use a record instead of a final class.

Change-Id: I340bf3f288ef8264dafc7d77403c5482f307a671
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 504ff6b7425c1a1d189e15141fa0843448948455)

Access clientActor exactly once

Make sure the null check is explicit to everyone.

Change-Id: I299728e411712ad73a83e78d66d078b0d79453b4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit dd91a3825bea25bb2919cc2ee63036f523083af2)

Bump Kryo to 4.0.3

https://github.com/EsotericSoftware/kryo/releases/tag/kryo-parent-4.0.3

Change-Id: Iebb8fb4a24c2406053a77fc141fc6ed8ad9b2462
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit c586853109faec36f9423dc65955ece72b9f3b5d)

Make LocalSnapshotStore final

We have a SpotBugs suppression which ends up being unnecessary if the
class is final.

Change-Id: If58497303871ae136875c80b8b094276ccda9d64
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit ddebb0ff6a098847b362d6f2bcf5d58f683714d3)

Turn LongSerdes into an enum

Use proper singleton for reuse.

JIRA: CONTROLLER-2089
Change-Id: Ia03b8610370521e97e45f67def07a32f1bf7d59b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit a879ebe46dc775b82e8b233117e16af2f1a27851)

Rename DataJournalEntrySerializer

DataJournalEntrySerdes is the proper name, use that.

JIRA: CONTROLLER-2089
Change-Id: I53247f7b226ed7ab0ff42ac24ad6f4d549e5db97
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 11483316054fd9862949e68f09326b90de1834b4)

Modernize sal-akka-segmented-journal

Use local variable type inference, instanceof patterns and seal
DataJournalEntry.

JIRA: CONTROLLER-2089
Change-Id: I3c2ee2de5eaae6874e2f75a44dfd6fce0942e8ae
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 7e629cabf95cf30562f10a658c8ebdd724d1b18d)

ServerInfo should be a plain record

This is a pure DTO, turn it into a record, eliminating quite a bit of
verbosity.

Also clean up RaftActorContextImpl.updatePeerIds() so we do not juggle
two booleans and set/reset our voting status.

Change-Id: Ifbb2ae0c8163472b8ef3b59210f12dccb1b8e30d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit 6213d7379b8c4d76a507ffd114a00ff9a8122287)

Bump versions to 8.0.5-SNAPSHOT

This starts the next development iteration.

Change-Id: Ib1a36a0278af6034f2378fa91a7e2103b9abf0ce
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Release controller

Clean up Shard a bit

Use instanceof patterns and local variable type inference to simplify
code a bit.

Change-Id: If437909b1a012a60fa5fd296dda6bffeb161fd25
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

{Start,Stop}DropMessages is generic

Add a diamond to constructor invocations to remove warnings.

Change-Id: I29f063eaf65656aef6715a8fe710e2afb35169e7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove setupDistributedDataStore()

These methods are no longer in use, remove them.

JIRA: CONTROLLER-2054
Change-Id: I8be5220eddb093e43ff3c71aa13fa40a60ddcc5a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Bump upstreams

Adopt:
- odlparent-13.0.10
- yangtools-11.0.5
- mdsal-12.0.4

Change-Id: Ia6f7420c5751436ec97d0a45187d79dadf9c3d94
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Clean up ClientBackedDataStoreTest

Use local variable type inference to simplify code a bit.

Change-Id: I5d6c116f40fb84250b3f775e92b98de558dde14d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Remove setupDistributedDataStoreWithoutConfig()

These legacy methods have been deprecated since previous major version,
remove them.

JIRA: CONTROLLER-2054
Change-Id: I13077de6ca505af6bf8627d7094e49a719636e32
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use constructor injection in OSGiDistributedDataStore

Upgraded SpotBugs does not like unitialized fields. Use constructor
injection to properly set them up.

Change-Id: I8d5226a8b0b6b2929eba3d4d9a73ac8e1c72613a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use VarHandles.fullFence() instead of an 'updated' field

Upgraded SpotBugs does not like an otherwise-unused field. Preempt
failures by using VarHandles.fullFence() instead.

Also modernize ActorUtils by using instanceof patterns and local
variable type inference.

Change-Id: Iec9b717eaf821657f3ced659e0c6e86f360b21c5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use constructor injection in OSGiActorSystemProvider

Upgraded SpotBugs does not like us not having initialized fields, use
contructor injection to side-step these warnings.

Change-Id: I3eef23b3cea88c97ccd8cb8794ec0587f07ccc77
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Do not allow global timer to be stopped

We have a globally-shared instance, it should never be stopped via
Timer.stop(). Log a warning whenever that is atttempted.

Change-Id: Id29f3799024b826d2278bf95f75cc661f85ab8e8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>

Use constructor injection for OSGiGlobalTimer

Upgraded SpotBugs is warning about field initialization, side-step it by
using constructor injection.

Change-Id: I1697459db8d72c956649f3c61f3d090322c3e9fb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>