Do not bump follower term while it is isolated