Bug 2187: Timeout the Persist state

Bug 2187: Timeout the Persist state

On add/remove server, if replication consensus isn't reached in the
Persist state, any pending or future operations will remain queued
until they eventually timeout on the caller side with no response. If
consensus is eventually reached, the pending operations would get
processed even though the caller is gone.

To alleviate this, I added a timer to the Persist state (2 * election
timeout). If it times out, pending operations are failed with a
PRIOR_REQUEST_CONSENSUS_TIMEOUT response. Also future operations are
failed if the timeout occurred.

Change-Id: I83ae528d6bec3fb8f8e3da7c5fd4ca75cfeeb4d5
Signed-off-by: Tom Pantelis <tpanteli@brocade.com>