main #5

Merged
ferociousbyte merged 14 commits from main into stable 2026-11-18 06:17:41 +00:00
No description provided.
An audit of the code (2026-11-16) found three ways to harm a cluster, each reproduced on real hosts before the fix:

- A term of long.MaxValue from any peer with a certificate was taken over and made durable; no election could follow.
  An entry out of place, or a configuration entry without a member list, made the protocol throw, and a host stops on
  that. A message now says whether a correct node could have sent it (Message.IsWellFormed); the codec reads no
  other, so its sender is cut off, and the protocol drops one before it changes anything.
- A sender of a later term whose entries differed from what a follower had committed had the follower cut its log
  back below its commit index (state machine safety). Such a sender is the leader of no term; nothing of it is taken.
- A certificate that names where its issuer is to be had sent the node there, in the handshake's own chain build and
  in the check after it, before either had refused it. Chains are now built from what the peer presented alone.

The simulation checks in every scenario that a node sends only messages it would take itself. What stays open is in
plan 05: a holder of a cluster certificate that is no member is still heard like one (open 8), and the accept loop
ends on a socket error (open 9).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
What the audit of 2026-11-16 left in the transport and the host:

- The loop that accepts connections ended on a socket error that was no stop, such as no descriptor left; the node
  sent on and heard nobody, and nothing said so. It now reports the error once and tries again, and connections that
  have not proven who they are hold at most 256 places. By hand, with 400 descriptors and 5,000 silent connections:
  the task runs on and a member is answered afterwards, where it had faulted after 145 before.
- A task that sends to a peer ended for good on an exception outside its write; it now loses the round and reports it.
- Certificate files that changed and do not load were kept quiet; they are reported once per change.
- A certificate that says what it is for has to be for servers and clients to be a node's; a name given before its
  certificate is checked is logged shortened; a directory the log creates is its owner's alone; a start that cut
  bytes off the log says so.
- In the protocol, what a leader holds about a peer is listed once (Forget), where two lists had to be kept alike,
  and the two calls of fsync share the one that repeats an interrupted call.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
From the audit of 2026-11-16, nothing that changes what a node does:

- Removed: RaftNode.VotedFor, the tests' view into the host's internals (it has none they use), two members of the
  simulated disk, a counter of the faulty file system and the asynchronous disposal of the test certificates that
  nothing read or called, a local of the log's writer program, ten using directives, an output of the workflow.
- Comments corrected where they said something else than the code: what RaftLog.Syncs counts, when StatusChanged is
  raised, what a refusal answers, which tests the workflow runs, how many repositories hold ci-version.sh, plan
  numbers that had moved, and the lines the two scripts print as their usage.
- One helper each for what the tests had several times: waiting for a condition on real hosts (five copies), and
  running seeds with every failing one named (five copies in three forms - the suites that named only the first now
  name all). Percentile once in the driver.
- The workflow's check for vulnerable packages fails when "dotnet list" fails, where it had passed.

257 tests pass. Whether version 1 of the messages counts as released (MessageCodec's comment says not) is the
owner's to say and left as it is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Three decisions of the owner (2026-11-17) on what the audit left open:

- Clusters run on version 1 of the wire format, so it stays as it is; a message that changes is version 2. The
  codec's comment and plan 06 said it was not released.
- MaximumFrameBytes is 8 MiB, where it was 64: a frame's buffer is allocated before its bytes arrive, batches stay
  near 1 MiB, and a snapshot is meant to say where the state is. A message beyond the limit would end the connection
  at the peer; it is not sent, and that is reported once. The test cluster's node, whose snapshot is its whole state,
  sets 256 MiB.
- The README names who holds the copyright.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Plan 11 says what "only members are heard" can mean, and which choice is left
All checks were successful
/ build (push) Successful in 1m9s
ac3c0ca016
The owner decided (2026-11-17) that certificates name their node and that a node hears members only. The second is
not one check: a follower that is behind would drop a member it has not heard of, once that member leads, and never
catch up - and unilayer adds nodes without telling the others. The plan sets out certificates that carry the member's
address, and three answers to who is heard beside the members: not those known to be removed (recommended, no change
of messages), a member that vouches (new messages, version 2), or certificates alone.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The default rule took a certificate only if its first DNS name was the whole member name, which no certificate of a
member named host:port can meet; an application that names members by address had to accept every certificate of its
authority (audit of 2026-11-16, S5). Now any subject alternative name counts, or the common name without one, against
the member's name or its host, as a DNS name or an IP address. ClusterCertificates.Names is public, for an application
that keeps its own rule.

The other half of the decision of 2026-11-17, variant A of plan 11, was built and failed in the simulation, in tests
that existed: a follower that knows a name was removed and missed its return drops the leader of that name for good
(liveness, seeds 31, 67, 167, 225, 13, 55), and letting it depose that leader ends in elections without end where
only that name can win (seed 143). Nothing of it is in the code. Plan 11 says why no rule without a new message
repairs it, and what is left to choose from; a list of revoked certificates in the transport is recommended.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A node that was removed keeps a certificate of the authority under a member's name, and a node added again under
that name differs from it in nothing but its certificate. So what the protocol could not tell apart (plan 11, A,
which failed in the simulation), the transport can: ClusterCertificates reads a list of revoked serial numbers
beside the authority, again when it changes; a certificate on it is refused in both directions, and connections
that stand are asked about again every second and closed. No state, no message, version 1 unchanged.

A list that is missing lets nobody in; one that changed and does not load is not taken in part, and is reported.
Revoking is the operator's step when a node is removed, and unilayer's half is not built.

281 tests pass; the cluster test with a member revoked while it runs passed 8 times in a row.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
RevokedCertificates is public: one place says what a line of the list may look like, for the file ClusterCertificates
reads and for an application's own (unilayer, plan 11). 281 tests pass.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
unilayer 0c11928. Its cluster tests pass on both engines (161 each), its end-to-end suite on the raft engine with
certificates of the new script passed 155 of 155 - with neither setting on, which no scenario of that suite uses.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A bad record anywhere in the last segment was taken for a write the last stop tore and cut off, entries the node
may have acknowledged among them (audit of 2026-11-16, S10; plan 04, open 3). The log now keeps how far the last
segment was made durable, in a file of its own written after every sync and not synced itself: exact after a
process that died, behind after a power cut, and behind is the safe side. Before the log is shortened the file is
lowered and made durable first, so it never names more than was durable. A bad record before that point, or a
segment shorter than it, does not open.

The last segment is synced on opening: what a process that died left in it is taken for durable from then on.

Left: a bit flip behind the point after a power cut still looks torn. No crash makes the log refuse itself: the
crash test passes with 20,000 seeds.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The smaller findings of the audit of 2026-11-16:

S6: a peer can send faster than the node steps, and what waited was unbounded. Of one peer 1024 messages or 16 MiB
wait, of all eight times that; more is dropped as a network drops it and counted.

S7: anyone who reaches the port could have a line logged per refused connection. One per source address and ten
seconds now, with how many were left out.

S8: a connection lasted past the end of the certificate it was made with. Connections that stand are looked at
every second, for a certificate that ran out as for one revoked.

S4: the retry of the task that accepts has a test.

288 tests pass. The test of S6 fails with the bound off.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Names and revoked certificates are proven end to end in unilayer's suite
All checks were successful
/ build (push) Successful in 1m50s
76abb49856
Scenario 19 of unilayer's end-to-end suite: three nodes in containers on the raft engine take certificates with
their addresses and both settings, node by node; a certificate of the CA for another address and one revoked while
its node runs are not heard, and each node is heard again with a new one. 174 of 174 begun on the raft engine, 194
of 194 begun on DotNext.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The audit's tooling findings T1 to T3, T5 and T6, and D7 with them.

Packages (T5, the owner's decision of 2026-11-18): nuget.config names nuget.org as the only source, every project
has a packages.lock.json, and CI restores with --locked-mode. A build for one runtime keeps its lock file with its
own output, so publishing the node program for the machines, or an application above these projects, changes no
committed file. Checked in the official SDK image: the locked restore passes, and a publish for linux-x64 leaves
the committed lock files as they are.

The workflow (T3): actions/checkout by the commit of v4.4.0, looked up at its source, and a permissions block that
says what the token is for. Every branch is still built; only "stable" reaches the steps that tag.

The test cluster (T1, T2, T6): keys of the throw-away authority are written with mode 0600, also over a file that
others could read (a test). The node's client port listens on one address, this machine by default and the
management address on the virtual machines, and an entry a client asks for is 1 MiB at most. vm.sh checks the
image against the SHA512SUMS beside it and says why host keys are not checked. Its verb "reset" and clock.py's
"now" went: the driver called neither (X15).

One source file issues the certificates for the cluster across machines and for the tests (D7); the test project
links it, as neither project can reference the other.

Run on three virtual machines (the owner's machine, 2026-11-18): deploy, the client port bound to 10.0.2.15 and
refused from another machine over the shared network, keys 0600, and 90 s of the driver with every fault and
clock drift (seed 7, 20 faults): every check holds. CI not checked.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The audit's D3, D5 and D6, the cosmetic findings left over, and the owner's decisions of 2026-11-18.

D3: ProtocolTesting holds the three nodes, the moment, the options, an entry and the election of the protocol
tests; a test class says only what it changes. D5: RaftHost asks its loop through one AskAsync, answers what the
protocol declined through one Declined, and cancels what waits through one interface of the requests. D6:
LogStore replaces the snapshot and the term-and-vote file through one Replace, in the order both copies had;
the crash tests pass with 20,000 seeds.

Decided by the owner: ProposalAccepted.Count is gone (set, never read), the simulation's Observer is gone (never
set), a transport event's peer is a field "Peer" of the log entry's scope (it was read nowhere), the licence is
said where it is said today (L3), and cluster:tlsVerifyNodeName stays off by default in unilayer (plan 11).

Cosmetic: the unread half of a deconstruction in RaftNode (X3), a counter where a queue was only counted (X12),
two methods that only forwarded (C12), and a .gitignore of what this repository leaves (C14); what is ignored in
the tree is the same before and after. The check in Transfer() that a handover's target is still a voter (X5)
stays, with why.

289 tests pass (Release, the owner's machine, 62 s). CI not checked.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
unilayer/raft!5
No description provided.