Fix intermittent failures in FollowerTest