BUG 3156 : Recreating CDS should not fail
When CDS is recreated it can fail because we try to
create actors on a datastore that is already terminated.
To address this issue I have enhanced DistributedDataStoreFactory
as follows,
- Simplified the actor system creation code by synchronizing
all static methods. Since these methods are anyway not used in
any fast paths - it is a fair enough solution.
- Added another static field in DistributedDataStoreFactory to
track the createdInstances. This is so that we can properly
cleanup the actor system as instances of the data store are destroyed.
- The actor system is now shutdown when there are no datastrore instances.
- Removed actor system shutdown from ActorContext to ensure that we
use the symmetric method destroyInstance
Now that we do not shutdown actor system till both the data stores
are destroyed it may so happen that we may have a situation where we are
trying to create an actor on the old actor system with an already used name
like shardmnanager-config or shardmanager-operational. To avoid this
I have added a loop when creating shardmanager which catches the thrown
exception and retries 100 times after waiting for 100 milliseconds. This
is to give some time for the actor to properly die.
Change-Id: I3c8b2b3b21a1519610bf2ed5e1af8be93f3120ce
Signed-off-by: Moiz Raja <moraja@cisco.com>
(cherry picked from commit
35f8cf2335afa8cf87ce565bbb974431ad5008a6)