CDS: Include CAN_COMMIT phase in rate limiter time period 88/23888/4
authorTom Pantelis <tpanteli@brocade.com>
Thu, 2 Jul 2015 02:53:56 +0000 (22:53 -0400)
committerGerrit Code Review <gerrit@opendaylight.org>
Thu, 13 Aug 2015 16:30:47 +0000 (16:30 +0000)
commit1783b656d3b1712c73c6596f3ae15e54e801d5e5
tree23eec857945151f5f4738c8431195fe2ad744400
parent26c4b15f6b55f012e7e7cf6e94d335386615af5c
CDS: Include CAN_COMMIT phase in rate limiter time period

I was testing with simulated latency in the followers. With a high enough
latency and tx thru-put, the pending commit queue in the
ShardCommitCoordinator got increasingly behind until latencies built up
enough to cause AskTimeoutExeption's on the front-end.

The rate limiter was throttling but not enough. I realized that the rate
limiter times the commit phase but not the canCommit phase. The latter is
what times out with pending tx's sitting in the queue waiting for canCommit.
So I changed ThreePhaseCommitCohortProxy to also time the canCommit
phase. This alleviated the timeouts - even with a really high max latency of
500 ms and 100 tx / sec client thru-put. The rate limiter thru-put
reduced it to about 3 / sec.

Change-Id: I6dc73d1d657519b9410ad034c69d26f19a0cb263
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/OperationCallback.java
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/ThreePhaseCommitCohortProxy.java
opendaylight/md-sal/sal-distributed-datastore/src/main/java/org/opendaylight/controller/cluster/datastore/TransactionRateLimitingCallback.java
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/TransactionRateLimitingCallbackTest.java [new file with mode: 0644]
opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/TransactionRateLimitingCommitCallbackTest.java [deleted file]