The relationship between ManagedChannel, InternalSubchannel and netty channel

[hlx502]

A ManagedChannel contains multiple InternalSubchannels, and an InternalSubchannel contains multiple TCP connections (ConnectionClientTransport). I have the following questions, can you help me answer them? Thanks.

I understand that InternalSubchannel is generally a connection between a specific service instance, why does it need EquivalentAddressGroups as a parameter, and why does InternalSubchannel contain multiple TCP connections?

[kannanjgithub]

Transport connection is only established to one of addresses in an EquivalentAddressGroup as gRPC considers them all equivalent.
For DNS based name resolution for example, there is only one entry created in the EquivalentAddressGroup per i.p. Address.
In the case of XDS name resolution of an endpoint, an endpoint can have additional i.p. Addresses as defined by this envoy proto for an endpoint in xds. The XdsEndpointResource creates an EquivalentAddressGroup with more than one address when there is dual stack enabled, in which case it adds i.p.v6 address from the endpoint's additional addresses into the EAG.

An InternalSubChannel holds only one ConnectionClientTransport denoting a transport connection, not multiple as you mentioned.

gRPC's per-rpc load balancing happens across all the subchannels in a managed channel.

[ejona86]

why does it need EquivalentAddressGroups as a parameter

Long, long ago gRPC Java supported dual-stack (IPv4 and IPv6 addresses for the same server). We knew each LB policy, even things like round-robin, would need pick-first behavior to choose the working version of IP. So we built pick-first into the Subchannel. We allow multiple EquivalentAddressGroups because the Attributes can be significant, so it prevents needing to merge multiple EAGs together (which we used to do, but stopped doing once Attributes mattered more).

However, with xDS we started using hierarchical LBs, where one LB policy delegates to another. And they've worked out pretty well. And thus with the cross-language dual-stack design subchannels cross-language won't have multiple addresses and instead we're delegating to pick first itself. So we're in the middle of migrating from the old style and in the future subchannels will have only one address.

and why does InternalSubchannel contain multiple TCP connections?

A subchannel can have up to one of (pending, active) transport. "pending" being connecting and "active" being connected. However, it can have many non-active/pending transports (stored in transports field). These are old transports that still have RPCs running on them and haven't yet stopped, but can't be used for new RPCs. For example, when we receive a GOAWAY the activeTransport will be cleared but it stays in transports until the transport terminates. This is necessary for managedChannel.awaitTermination() and even managedChannel.shutdownNow(), as shutdownNow() will cancel those old RPCs.

[ejona86]

Seems like this is resolved. If you have more questions, comment, and it can be reopened.