Bug 2847: Recovery of a large journal runs out of memory 40/16540/6
authorTom Pantelis <tpanteli@brocade.com>
Fri, 13 Mar 2015 23:57:47 +0000 (19:57 -0400)
committerTom Pantelis <tpanteli@brocade.com>
Wed, 25 Mar 2015 22:47:32 +0000 (18:47 -0400)
commit07267ef7e2e6d46c3a514a978a8dbb266966176c
tree8cc3b8232148fbcb1d45b6522911a9e0e851623a
parente6e3ad00c95d72f731142e649f198c5ded9dde64
Bug 2847: Recovery of a large journal runs out of memory

Changed the ShardRecoveryCoordinator to commit each log entries batch
immediately instead of using an executor to prepare write transaction in
parallel and then commit them all on recovery complete. This resulted in
a lot of memory overhead for little to no gain. I also changed it to
cache the serialized ModificationPayload instances instead of
immediately de-serializing - this further reduces the memory footprint
as the serialized instances are much smaller

As a result, all of the recovery code is now in the
ShardRecoveryCoordinator - Shard is essentially a pass through.

I also reduced the shardJournalRecoveryLogBatchSize to 1000 to further
reduce the memory footprint.

Change-Id: I3aaabe52781bc0db14975e0a292ef9fd18aa3d7c
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
opendaylight/md-sal/sal-clustering-config/src/main/resources/initial/datastore.cfg
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/Shard.java
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/ShardRecoveryCoordinator.java
opendaylight/md-sal/sal-distributed-datastore/src/main/yang/distributed-datastore-provider.yang