Track skipped transactions We are allocating transaction identifiers which are shared across shards. As a transaction can touch only some shards, non-participating shards will never see the transaction ID, leading to holes in their UnsignedLongSets -- and those holes gradually eat up more and more memory. Track when we have such a hole and lazily forward a new request, which purges one or more such identifiers -- plugging the memory leak. The crux of the machinery is exchanging state between ProxyHistory and FrontendHistoryMetadataBuilder. JIRA: CONTROLLER-1991 Change-Id: I3817fa2841e5f9c405bb20ff1a104537ad459ce3 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Migrate nullness annotations This mass-migrates sal-distributed-datastore to use JDT nullness annotations instead of JSR305. Change-Id: I2cd739e3db8c074a55e335e17297a23065775d6a Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Fix sonar warnings in sal-distributed-datastore These come from squid: - String literals should not be duplicated - Modifiers should be declared in the correct order - Lambdas and anonymous classes should not have too many lines - Nested blocks of code should not be left empty - Local variables should not shadow class fields - Exception handlers should preserve the original exception - Utility classes should not have public constructors - Overriding methods should do more than simply call the same method in the super class - Unused private fields should be removed I fixed quite a few of them. Others we'd have to suppress or modify the sonar config to be more lenient. Change-Id: I7ce7b2a05feac9844fd9c37927de82b7b8b68ee5 Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
BUG-5280: add frontend state lifecycle When transitioning between roles we need to take care of proper handling of state known about frontend. This patch adds the leader/non-leader transitions, creating the state from FrontendMetadata and forgetting it when transactions are committed. Our replicated log needs to grow more entries to accurately replicate the state of the conversation between the frontend and backend, so if a member becomes the leader it has an understanding of which transactions and transaction chains have been completed (aborted, committed, purged). These are replicated before a response is sent to the frontend, so if a leader before they replicate successfully, the frontend will see them as a timeout and retry them (and be routed to the new leader). Both leader and followers are expected to keep the metadata handy: the leader keeps for the purpose of being able to generate a summarized snapshot. The followers keep it so their metadata view is consistent with the contents of the data tree. Change-Id: I72eea91ee84716cdd8a6a3521b42cca9a9393aff Signed-off-by: Robert Varga <rovarga@cisco.com>
BUG-5280: add {Create,Close,Purge}LocalHistoryPayload This patch introduces three new payloads which deal with replicating local histories to followers. These are persisted whenever a transaction chain (e.g. local history) is created or closed cleanly on the shard leader. Followers can use these to track transaction chains and pick up processing in case of a leader failover. Change-Id: I3fe5ac153c88f23f9b871bd23cb04a8e2410af91 Signed-off-by: Robert Varga <rovarga@cisco.com>
Fix warnings/javadocs in sal-distributed-datastore First off, I apologize for the size of this patch. There's a ton of classes in this project and I didn't even get to all of them (will follow-up). While a lot of files were touched, the changes were mostly small. Fixed a lot of checkstyle warnings and cleaned up javadocs. Most of the warnings/changes were for: - white space before if/for/while/catch - white space before beginning brace - line too long - illegal catching of Exception (suppressed) - variable name too short - indentation - missing period after first sentence in javadoc - missing first sentence in javadoc - missing <p/> in javadoc Change-Id: Id56d874a8fbcbbc9285279a71c0a5aba393653a9 Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
Address review comments For https://git.opendaylight.org/gerrit/#/c/42974/. Change-Id: If054abf5dce4207fe536beeef15ffaad61b63074 Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
BUG-5280: add FrontendMetadata This patch adds the frontend tracking abilities for followers. It also defines the corresponding ShardDataTreeSnapshotMetadata for use with persistence. Change-Id: I7e2c6755c3389dcb5284f17a9c6076fb9e7ac95e Signed-off-by: Robert Varga <rovarga@cisco.com> Signed-off-by: Vaclav Demcak <vdemcak@cisco.com>
BUG-5280: persist metadata in snaphots This patch adds the wiring in ShardDataTree to persist various pieces of metadata in a snapshot. It also includes metadata recovery from a snapshot. In order to make this work, this patch centralizes all actual payload and snapshot handling within the ShardDataTree by introducing explicit entrypoints for each avenue through which data can be introduced. Change-Id: Ibc15bd152bd44dd583d67bb7fc61bc8f3086df30 Signed-off-by: Robert Varga <rovarga@cisco.com>