Remove DistributedDataStoreTest Rehost the tests for waiting into ClientBackedDataStoreTest, leaving only limiter tests around -- and those are no longer useful. JIRA: CONTROLLER-2054 Change-Id: Idf908cd33110c610c8fe0c0b37cd69dabdd3f367 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Use SettableFuture<Empty> for readinessFuture java.lang.Void is promoting use of nulls, whereas Empty.value() is a non-null object. Migrate internal users. Change-Id: I6a8a85917d9c3a8c3d5a9aadf691c9d1a63a767b Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Refactor DataStore readiness tracking Using a CountDownLatch is not composable, which leads to current layout. Switch to using a SettableFuture, which can be accessed via AbstractDataStore.initialSettleFuture(). This allows us to externalize the settle policy, letting callers decide what to actually do. JIRA: CONTROLLER-1882 Change-Id: Iaf9a359cfc2507ae35688fca3673c13713c2b427 Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Allow shard settle timeout to be tuned When we are starting the datastore, we wait up to a fixed number of election timeout intervals (3) for shards to finish electing a leader. This is not always enough, especially if recovery takes a long time, hence introduce shardLeaderElectionTimeoutMultiplier to make this tunable. JIRA: CONTROLLER-1914 Change-Id: Iba1d116d0248fc6046aeeae3ec30ecac50f373c9 Signed-off-by: Tibor Král <tibor.kral@pantheon.tech> Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Rename ActorContext to ActorUtils ActorContext is overloaded name even within Akka, without us adding to it. Furthermore Akka's AbstractActor defines ActorContext as its nested class, which thoroughly breaks resolution priorities. Rename ActorContext to ActorUtils, reducing confusion around class uses. Change-Id: I140239a8f74ee7deecf9ee848df0cfbbb72f3c4d Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Speed up DistributedDataStoreTest Sharing SchemaContext shaves off about .5 seconds. Change-Id: I628af3b577ca53fc9f5ace05dc825f7324b71b8f Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Remove unused exceptions This drops exception declarations which are never used. Change-Id: Icc8938b9c3b437a0d5961ec1b481fd06c52d47f2 Signed-off-by: Stephen Kitt <skitt@redhat.com>
sal-distributed-datastore Checkstyle fixes (for next version) Even though sal-distributed-datastore currentla already has CS enforcement, these violations don't show up in the build, but I see them as red in Eclipse; I think this must be becaues the CS version used in the Eclipse plugin-in is more recent than the CS used in odlparent, and one of the existing activated checks got a little more stringent. Cleaning this up thus helps both to (a) not have read in Eclipse; (b) pave the way to upgrade Checkstyle in odlparent some day. Change-Id: Ib5649a95a1b26b5791f2c3f3f83924b569f965a2 Signed-off-by: Michael Vorburger <vorburger@redhat.com>
Fix unit test CS warnings in sal-distributed-datastore Fixed checkstyle warnings in unit tests. Most of the warnings/changes were for: - white space before if/for/while/catch - white space before beginning brace - line too long - illegal catching of Exception (suppressed) - illegal throwing of Throwable (changed to Exception) - variable name too short - indentation, mostly due to nested code inside anonymous JavaTestKit instances - separator wrapping: '.', '+', '&&' should be on a new line - local vars/params hiding a field - putting overloaded methods close to one another - remove unused vars - convert functional interfaces to lambdas (eclipse save action) - adding final for locals declared too far from first usage Also 3 classes are no longer used so I removed them rather than fix warnings. Change-Id: Ie1507e36c67a2b58f7efb62378212976b962f9fe Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
BUG-5280: introduce DistributedDataStoreClientActor This patch introduces a common ClientActor, which keeps track of frontend generations. Also introduce bind for DistributedDataStore, which uses this common infrastructure. Interface between the DistributedDataStore and the actor world is captured as DistributedDataStoreClient. Change-Id: I42c3281ca790fb5615a593740424ac494469e6a7 Signed-off-by: Robert Varga <rovarga@cisco.com>
Fix Eclipse compilation warnings. Fix compilation warnings in DistributedDataStoreTest that DistributedDataStores were never closed. Also fix NPEs on closing DistributedDataStores when the MXBeans are uninitialized. Change-Id: I5dcaa389e1e69f934e9016933b00be3adaf4529f Signed-off-by: Gary Wu <Gary.Wu1@huawei.com>
Fix license header violations in sal-distributed-datastore Change-Id: I519e80cb6329f752b6b533de067ea3fd6277bedf Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
BUG 2584 : Block Datastore creation till Shards are ready Change-Id: I4bef70ea9e6b0dd2cdb82344749abc5028cf3184 Signed-off-by: Moiz Raja <moraja@cisco.com>
Bug 2597: Batch modification operations in TransactionProxy Modified TransactionContextImpl to batch write, merge, delete modification operations into a single BatchedModifications message and send the batch when a count threshold is reached. Instead of using the current WriteData, MergeData, DeleteData message classes in BatchedModifications, I reused the Modification classes that are used as the payload for peristence and replication. BatchedModifications derives from MutableCompositeModification. The ShardWriteTransaction now simply transfers the Modification instances in the BatchedModifications instance to its internal MutableCompositeModification and applies the Modifications to its transaction. The Modification classes were refactored a little wrt to versioning. The Write/Merge/DeleteModifications no longer read/write the version from the stream. The version is read/written by MutableCompositeModification so it's redundant to also do so in the data Modification classes. The WriteData, MergeData, DeleteData and associated reply classes were deprecated. I did refactor them a little (along with ReadDataReply) as I was originally going to use them. The VersionedSerializableMessage interface was removed as I realized it makes more sense to pass the version in the WriteData, MergeData, DeleteData constructors instead of passing the version via toSerializable. This made it easier in TransactionContextImpl to transition it to use BatchedModifications. I created a VersionedExternalizableMessage base class that reads/write the version. To handle backwards compatibility with Helium, I derived a LegacyTransactionContextImpl class from TransactionContextImpl that overrides writeData, mergeData and deleteData to send WriteData, MergeData, DeleteData messages. TransactionProxy creates this instance if the remote tx versions is < Lithium version. Change-Id: I28df1f89e97667eaca114b991355a6e9d0160a59 Signed-off-by: tpantelis <tpanteli@brocade.com>
BUG 2676 : Apply backpressure when creating transactions This patch uses a rate limiter to limit the number of write transactions that can be created on the datastore at any given point of time (within a second). We also have added a metric that is trackable using JMX which will show exactly how much time we take in committing a transaction. Could this rate limiter have been on the DataBroker? Yes it could have been but I think we need to go granular on this one. Ideally the ratelimiter would be for every shard transaction but it's not trivial to do in a way that would actually block the consumer for now - this is because creating a transaction on Shard itself is an async operation. Change-Id: I3d8e3baeb892ba494c0ab0d13a0d6226c1516511 Signed-off-by: Moiz Raja <moraja@cisco.com>
Bug 2055: Handle shard not initialized resiliently Added a flag to the FindLocalShard and FindPrimaryShard messages to specify whether or not to wait for he shard to be initialized. Modified FindLocalShard and FindPrimaryShard message handling in the ShardManager to wait for the shard to be initialized before replying if the flag is set in the message. This is done async by caching the response info in the ShardInformation and sending the response when the ActorInitialized message for the shard is received. Modified DistributedDataStore#registerChangeListener to always create a DataChangeListenerRegistrationProxy and moved the code that performs the findLocalShard and RegisterChangeListener operations into DataChangeListenerRegistrationProxy. In DataChangeListenerRegistrationProxy, the findLocalShard operation is done async (with a long time out). On success, the RegisterChangeListener message is sent. Added change listener test to DistributedDataStoreIntegrationTest and removed the DistributedDataStoreTest since most of the tests tested registration and the bulk of that logic is now in DataChangeListenerRegistrationProxy with associated tests. The rest of the tests in DistributedDataStoreTest are covered by DistributedDataStoreIntegrationTest. Change-Id: I9f0d412801110c97eb48ecc4d0fd77ee081a7e81 Signed-off-by: tpantelis <tpanteli@brocade.com>
Bug 2038: Ensure only one concurrent 3-phase commit in Shard Added a ShardCommitCoordinator class that ensures there's only one concurrent 3-phase commit. The following outlines the new commit workflow: - On ready, the ShardTransaction creates the dom store cohort and forwards a new ForwardedReadyTransaction message to the shard. - The shard calls its ShardCommitCoordinator to add the cohort and modificaton to a cached keyed by transaction ID. - On CanCommitTransaction message, the ShardCommitCoordinator looks up and removes the cohort entry from the cache corresponding to the transaction ID passed via the CanCommit message. The ShardCommitCoordinator also caches the cohort entry for the current transaction in progress. If there's no transaction in progress, the committing transaction becomes the current transaction and canCommit is called on the cohort. Otherwise, the cohort entry is queued to be processed after the current tranaction completes. - On CommitTransaction message, if the transaction ID passed via the Commit message matches the currently cached cohort entry, the preCommit and commit phases are performed. When complete, the ShardCommitCoordinator dequeues the next waiting transaction cohort entry, if any, and process it. If a Tx is aborted and it is the current transaction, the ShardCommitCoordinator handles it as a completed Tx. Implemented a timeout mechanism using the akka scheduler such that if the commit message isn't received after a period of time (default 30 s) after the canCommit message, the transaction is aborted so that the next transaction can proceed. This is to handle remote node or network failures during a 3-phrase commit. The ThreePhaseCommitCohort actor was removed along with the ForwardedCommitTransaction. Change-Id: Iaa5692ca45cd7635d1a06a609f4bf98bec50df14 Signed-off-by: tpantelis <tpanteli@brocade.com>
Bug 1577: Gates access to Shard actor until its initialized 1. Shard manager creates Shard and mark them un-initialized. Shard completes recovery and onRecoveryComplete, sends a message to Shard manager to mark it initialized. If a request for Shard comes to Shard manager and the shard is not initialized, it sends ActorNotInitialized message. 2. Normalizes and refactors ActorContext. 3. Adds AbstractUntypedPersistentActorWithMetering to meter ShardManager. Change-Id: Ibf15a2ef56422bda53067039d2271a719b6b2ce3 Signed-off-by: Abhishek Kumar <abhishk2@cisco.com>
BUG 1735 Registering a data change listener should be asynchronous Change-Id: I1a819976afe6ca44ac811ee30d305fbe76bb8acf Signed-off-by: Moiz Raja <moraja@cisco.com>
Bug 1446: Add JMX stats for clustered data store Added stats, available via JMX, to track data for the various thread pool executors used by the clustered data store. Change-Id: Ifb8996075253e87b7fd4e7cb6cf876c3af80ff2a Signed-off-by: tpantelis <tpanteli@brocade.com>