Currently in testing: the GitHub code will be opened once complete.

Rolling Restart Procedure

There's one core principle: a majority must always stay alive. With 3 nodes, you can only take down 1 at a time; with 5 nodes, up to 2.

  1. Restart followers (non-leader nodes) first, one at a time — the leader keeps serving throughout. Confirm each restarted node has caught up via peer log replication/ snapshot before moving to the next.
  2. Finally, bring down the leader. The remaining nodes detect the lost heartbeat and elect a new leader within 0.5-1 second. Client requests during that window get E no_leader, but the official clients retry transparently, so the application only sees a brief delay.
  3. Bring the old leader back up — it rejoins as a follower and catches up (it doesn't automatically reclaim leadership).

Lock state is preserved throughout as long as a majority stays alive — locks granted or released during the restart also go through commit, so nothing is lost.

Flow (3 Nodes)

Leader
Follower 1
Follower 2
① restart follower 1 — the leader keeps serving
catches up via log replication/snapshot
② once caught up, restart follower 2
catches up via log replication
③ restart the leader last
Vote → new leader elected within 0.5-1s
the old leader rejoins as a follower and catches up
consensus RPC (Raft)

During the brief window while the leader is down, clients may receive E no_leader, but the official clients retry transparently, so it only appears as a short delay to the application.