Bug 1831 Batch messages on journal recovery 36/11136/11
authortpantelis <tpanteli@brocade.com>
Sun, 31 Aug 2014 15:15:18 +0000 (11:15 -0400)
committertpantelis <tpanteli@brocade.com>
Thu, 18 Sep 2014 14:25:34 +0000 (10:25 -0400)
commit9e59cc0d824e6752a7a3f3ba092abaaf3c1d4193
tree799a47bfa280f742ddf16bdef493f8bfec3838ca
parentf7f22a2284a7ec28c4a35518934e3f9783d503fa
Bug 1831 Batch messages on journal recovery

Added journal log recovery batching support in RaftActor along with
additonal abstract methods for granular recovery control.

It turns out that batching helps a little but the real performance hog
was deserialization. There was a couple inefficient PreConditions in the
NormalizedNodeSerializer (unwanted String concats in the fast path) that
accounted for significant performance degradation. I also made a few
other minor performance enhancements.

Although deserialization is much better with the fixes, I also
implemented some parallelization during shard recovery. I added a
ShardRecoveryCoordinator class that parallelizes deserialization
of journal log enry batches and snapshots for faster recovery time.
The resulting transactions are still committed to the data store in the
order the corresponding snapshot or log batch are received to preserve
data store integrity.

The journal recovery log batch size is configurable vai the config XML.
I also made the shard heartbeat interval and shard snapshot batch count
configurable.
`
Change-Id: I52ef1690bfb6cc486c329ee60f77c52720c24469
Signed-off-by: tpantelis <tpanteli@brocade.com>
24 files changed:
opendaylight/md-sal/sal-akka-raft/src/main/java/org/opendaylight/controller/cluster/example/ExampleActor.java
opendaylight/md-sal/sal-akka-raft/src/main/java/org/opendaylight/controller/cluster/raft/AbstractReplicatedLogImpl.java
opendaylight/md-sal/sal-akka-raft/src/main/java/org/opendaylight/controller/cluster/raft/ConfigParams.java
opendaylight/md-sal/sal-akka-raft/src/main/java/org/opendaylight/controller/cluster/raft/DefaultConfigParamsImpl.java
opendaylight/md-sal/sal-akka-raft/src/main/java/org/opendaylight/controller/cluster/raft/RaftActor.java
opendaylight/md-sal/sal-akka-raft/src/main/java/org/opendaylight/controller/cluster/raft/RaftActorContextImpl.java
opendaylight/md-sal/sal-akka-raft/src/main/java/org/opendaylight/controller/cluster/raft/ReplicatedLog.java
opendaylight/md-sal/sal-akka-raft/src/test/java/org/opendaylight/controller/cluster/raft/RaftActorTest.java
opendaylight/md-sal/sal-clustering-commons/src/main/java/org/opendaylight/controller/cluster/datastore/node/utils/serialization/NormalizedNodeSerializer.java
opendaylight/md-sal/sal-clustering-commons/src/main/java/org/opendaylight/controller/cluster/datastore/node/utils/serialization/PathArgumentSerializer.java
opendaylight/md-sal/sal-clustering-commons/src/main/java/org/opendaylight/controller/cluster/datastore/node/utils/serialization/ValueSerializer.java
opendaylight/md-sal/sal-clustering-commons/src/main/java/org/opendaylight/controller/cluster/datastore/node/utils/serialization/ValueType.java
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/DatastoreContext.java
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/Shard.java
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/ShardRecoveryCoordinator.java [new file with mode: 0644]
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/config/yang/config/distributed_datastore_provider/DistributedConfigDataStoreProviderModule.java
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/config/yang/config/distributed_datastore_provider/DistributedOperationalDataStoreProviderModule.java
opendaylight/md-sal/sal-distributed-datastore/src/main/yang/distributed-datastore-provider.yang
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/AbstractActorTest.java
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/ShardTest.java
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/ShardTransactionTest.java
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/utils/InMemoryJournal.java [new file with mode: 0644]
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/utils/InMemorySnapshotStore.java
opendaylight/md-sal/sal-distributed-datastore/src/test/resources/application.conf