Transport and networking internals

Orleans separates message routing from transport management. MessageCenter decides where a message should go; ConnectionManager obtains a usable connection to the target silo; the connection implementation frames, queues, and writes messages to a socket. This separation lets routing repair activation addresses while transport focuses on connection management.

Connection lifecycle

For each remote SiloAddress, ConnectionManager keeps a ConnectionEntry containing active connections, a pending connection attempt, and the last failure time. GetConnection reuses a suitable active connection. When active connections are exhausted, it starts one shared pending attempt per endpoint, and concurrent senders await that attempt.

Connection establishment has a bounded OpenConnectionTimeout. A failed attempt clears the pending task, removes defunct connections, records the failure, and applies the configured retry delay before another attempt. A timeout classifies the transport attempt as failed; application processing outcome requires separate reconciliation.

Source: ConnectionManager, Connection, and SiloConnection.

Message path and framing

When MessageCenter.SendMessage has a target silo, it reuses an existing connection or obtains one from ConnectionManager. A local target uses receive processing directly. A known-dead target causes a transient rejection for request and one-way messages, and expired messages are dropped before consuming transport work.

The connection pipeline performs the protocol preamble and then exchanges framed payloads. MessageSerializer encodes the message header and body using Orleans serialization; the frame helper validates lengths and rejects malformed input before dispatch. TLS wraps the same Orleans framing and message protocol as connection middleware.

The connection accepts outgoing messages through an unbounded channel. Its socket writer observes transport flow control while concurrent senders can continue adding messages to that channel, so a slow connection can accumulate queued messages until it recovers or closes. A successful socket write means the frame was handed to the transport; the grain result confirms request completion. Disconnects remove the connection from the endpoint entry and cause later sends to establish a replacement.

Failure and shutdown

When a connection terminates, the base connection sends each in-flight message to the connection-specific OnSendMessageFailure implementation and calls RetryMessage for messages which remain queued. Silo and gateway inbound connections fail in-flight messages; client outbound connections return them to MessageCenter for routing. Application-level response timeouts remain ambiguous because a request can have reached the target before the connection failed.

Shutdown first blocks new application traffic while allowing responses and membership traffic needed to complete the stop protocol. MessageCenter rejects or drops blocked messages, stops accepting client messages, and then closes connections. Inbound requests arriving at a stopping silo are rejected or dropped according to message direction, so callers must still handle a rejection or timeout.

Design trade-offs

• Per-endpoint connection state avoids global coordination but means every silo must observe and repair its own broken paths.
• Reusing connections reduces handshake and allocation cost, while multiple connections can improve throughput and avoid head-of-line blocking.
• Dropping expired messages protects a saturated runtime from work whose callback deadline has elapsed. Applications use operation IDs and durable state to reconcile the operation’s outcome.

For end-to-end semantics, see messaging and delivery semantics. Network endpoint selection and firewall requirements belong in topology, networking, and clustering.