Support using the ring hash balancer without xDS

[atollena]

It is currently not possible to use the ring_hash balancer without xDS. As far as I can tell (from the Go code and the A42 gRFC), there is no strong ties between xDS and the ring_hash balancer: the only missing piece to use it directly via service config is that there is no public interface to set the hash code in the request context (or equivalent, e.g. CallConfig in c-core and CallOption in Java).

So the main thing to define would be the interface to set the hash on a per-request basis, across language. That may warrant another gRFC.

The alternative for gRPC users that want to use consistent hashing is to either build an xDS infrastructure, or copy paste the ring_hash balancer code just to expose a way to set the hash, which is not ideal.

[veblush]

Mark, would you take a look at this?

[markdroth]

I've discussed this briefly with @ejona86 and @dfawley, and I think we'd be open to changing the ring_hash policy itself to provide a way to compute the hash for a request. We don't think it really makes sense to have a separate mechanism to define the hash in gRPC the way that it's done in xDS.

I don't think we have time to work on this right now, but if you want to put together a gRFC and are willing to contribute an implementation, we'd be open to reviewing it.

[atollena]

@markdroth I have a question related to this. A lot of our consistent hashing use cases make use of additional address metadata, instead of only the endpoint IP address as currently supported by gRPC. Envoy supports customizing the host hash key with the lb.envoy hash_key host metadata (see the ring hash docs in envoy documentation). This is useful to minimize churn in cases where logical replicas of a service do not change, but the IP address may change, such as with Kubernetes Statefulsets.

So in order to make the ring hash balancer to be truly useful to us, we would need:

• A way to set the request hash programatically instead of via RDS, as originally described in this issue
• A way to override the endpoint hash key in non-xDS cases, by feeding the endpoint attribute from a custom resolver
• Ideally, also a way to set the hash key tolb.envoy hash_key field (or another LbEndpoint metadata field) when using xDS.

I'm thinking that all of those may make sense to bundle as a single gRFC. Do you have a preference for the granularity of gRFCs? I see that they tend to bundle multiple related but somewhat independent features together.

[markdroth]

Bundling all of this into a single gRFC seems fine.

I think the mechanism for setting the endpoint's hash key will be fairly straightforward; we can basically just define a resolver attribute for that. And I also don't see any problem with supporting an endpoint metadata key in EDS to set that attribute.

However, I am a little concerned about the complexity involved in exposing an interface to programmatically set the request's hash. When we discussed this earlier, we were thinking that this would probably just be something simple like "use a specific request header", which could just be built into the ring_hash policy. Exposing an API to allow applications to do this programmatically is a much broader API surface, and before going down that road, I'd like to be convinced that there's no simpler way to do this. Is there a way that we can handle your use-case without that complexity?

[atollena]

However, I am a little concerned about the complexity involved in exposing an interface to programmatically set the request's hash. ...

I think that your suggestion of extracting the key from a header covers our use case, which is to programatically set the hash key in the application, which can be done through the metadata interface available in all languages. So we could use a pre-defined grpc- prefixed metadata for the hash key, or make the header name configurable in the ring hash policy config. This is very simple and I think would cover most use cases.

We may also want to define hashing filter_state io.grpc.channel.id key when using service config, I guess because if it's useful for xDS users, it's probably also useful for others. I don't have a use case for this, though.

[markdroth]

Sounds like we're aligned on using a header for the request hash. I'd be fine with adding a field to the ring_hash config to configure the header name to use.

The io.grpc.channel.id filter state thing was added for a fairly ugly internal use-case that we'd like to eventually eliminate, so I'd prefer not to duplicate this in the ring_hash config.

I'm looking forward to seeing the gRFC for this!