Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions CLUSTER_MEMBERSHIP_GOSSIP.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,26 @@ flowchart LR
`GossipRequest`; the default `GossipTransport` simply forwards the request
through `IContext.RequestReenter`.

### Type relationships

```mermaid
classDiagram
class Gossiper
class GossipActor
class Gossip
class MemberStateDeltaBuilder
class GossipSender
class IGossipTransport
class GossipTransport
Gossiper o-- Gossip
Gossiper --> GossipActor : commands
Gossiper --> GossipSender : uses
GossipActor --> Gossip : merges
Gossip --> MemberStateDeltaBuilder : uses
GossipSender ..> IGossipTransport
IGossipTransport <|-- GossipTransport
```

## Detecting members

Cluster providers (e.g., Kubernetes) watch the environment for running nodes and
Expand All @@ -51,6 +71,41 @@ for inclusion in the gossip state ([Gossiper.cs](src/Proto.Cluster/Gossip/Gossip
The gossip implementation stores the full membership under the `cluster:topology`
key and tracks active member IDs for later consensus checks ([Gossip.cs](src/Proto.Cluster/Gossip/Gossip.cs#L85-L92)).

## Gossip state structure

```mermaid
classDiagram
class GossipState {
+members : map<string, GossipMemberState>
}
class GossipMemberState {
+values : map<string, GossipKeyValue>
}
class GossipKeyValue {
+sequence_number : long
+value : Any
+local_timestamp_unix_milliseconds : long
}
GossipState --> GossipMemberState : members
GossipMemberState --> GossipKeyValue : values
```

Each cluster node keeps a `GossipState` containing a map of member IDs to
`GossipMemberState` entries. A `GossipMemberState` holds a map of keys such as
`cluster:topology` or `cluster:heartbeat` to `GossipKeyValue` records. Every
`GossipKeyValue` carries a monotonically increasing `sequence_number` and a
payload packed as `google.protobuf.Any`【F:src/Proto.Cluster/GossipContracts.proto†L21-L28】【F:src/Proto.Cluster/GossipContracts.proto†L40-L43】.

When a node updates one of its keys, the sequence number is incremented before
the value is stored, ensuring later updates supersede earlier ones【F:src/Proto.Cluster/Gossip/GossipStateManagement.cs†L120-L132】.
To avoid resending data, `MemberStateDeltaBuilder` tracks a per-target
high-water mark ("watermark") for each `{target}.{member}` pair. Only values with
sequence numbers above this watermark are included in a delta, and the watermark
is advanced to the highest sent sequence number【F:src/Proto.Cluster/Gossip/MemberStateDeltaBuilder.cs†L51-L74】.
`Gossip` stores these watermarks as committed offsets and updates them when a
peer acknowledges receipt, guaranteeing that each node receives monotonically
ordered state without duplicates【F:src/Proto.Cluster/Gossip/Gossip.cs†L69-L71】【F:src/Proto.Cluster/Gossip/Gossip.cs†L300-L307】.

## Gossip dissemination

The gossip loop periodically updates heartbeat information and sends the current
Expand Down
4 changes: 4 additions & 0 deletions logs/exceptions.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,12 @@

### Proto.Tests.SupervisionTestsAllForOne.AllForOneStrategy_Should_PassExceptionOnRestart
`System.InvalidOperationException: Collection was modified; enumeration operation may not execute.` during enumeration of mailbox statistics.

### Proto.Cluster.Tests.GossipCoreTests.Large_cluster_should_get_topology_consensus
`Test failure: output indicated [FAIL] but passed on rerun; no stack trace captured.`

### Proto.Tests.SupervisionTestsAlwaysRestart.AlwaysRestartStrategy_Should_RestartFailingChildOnly
`Assert.Equal() Failure: Values differ. Expected: 1 Actual: 0` in `SupervisionTests_AlwaysRestart.cs:line 43`.

### Proto.Cluster.Tests.ClusterTopologyBuilderTests.Compute_FiltersBlockedAndDuplicates
`Assert.Equal() Failure: Strings differ Expected: "4" Actual: "3"`
7 changes: 7 additions & 0 deletions logs/log1756018838.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Log 1756018838
- Added class diagrams detailing relationships among Gossip types for clearer navigation of the subsystem.
- Documented the structure of `GossipState`, explaining sequence numbers and high-water marks to aid maintainers.
- Verified .NET 8.0.100 environment and executed core test suites.
- `Proto.Actor.Tests` (initial flake in Supervision test, passed on rerun)
- `Proto.Remote.Tests`
- `Proto.Cluster.Tests`
Loading