Fractional evaluation - Sticky Bucketing

[pmarchini]

Hey everyone,

I'm opening this discussion asking for help 😁

I'm trying to understand if there's a way to achieve sticky bucketing without extra tools during the fractional evaluation.
In our use case, we're splitting different experiments and trying to "balance" the load into different experiments with a configuration similar to the following one:

"experiments-coordinator": {
    "state": "ENABLED",
    "variants": {
        "EXP_1": "EXP_1",
        "EXP_2": "EXP_2",
        "NONE": "NONE"
    },
    "defaultVariant": "NONE",
    "targeting": {
        "fractional": [
            {
                "cat": [
                    { "var": "$flagd.flagKey" },
                    { "var": "targetingKey" }
                ]
            },
            ["EXP_1", 40],
            ["EXP_2", 50],
            ["NONE", 10]
        ]
    }
}

One of our requirements is the ability to change the percentages without reassigning the variants on subsequent calls.

After checking the implementation and the documentation, I suppose this is not the case by design.

If I'm correct, then there's no way, at the moment, to achieve sticky bucketing using fraction.
Is this the case?

---

Note: while reading through the ADRs, I've noticed this:
https://github.com/open-feature/flagd/pull/1800/files#diff-844f86b50491b140561b7e6db409ea1b0e06e34f6f40049c4b0027ed3849e1d5R40

So I'm wondering: is it something you're planning to add?

cc @beeme1mr , sorry for the tag but I've seen that the ADR is yours 😁

[beeme1mr]

Hey @pmarchini, the fractional operator already performs deterministic bucketing. The first element in the array determines the evaluation context attributes that are used to determine the bucket to use (targetingKey will be used if this config is omitted). The ADR I have open focuses on expanding the number of buckets from 100 to 100,000 to allow for more fine-grained releases.

You can play around with the sticky bucketing configuration in the playground and read about the technical implementation here.

[pmarchini]

Hi @beeme1mr, thank you so much for your reply!
I just have one doubt at this point:

Is it expected that when the weights change, a flag that has already been evaluated (and returned a variant) could return a different variant?

Note: just to be clear

First evaluate:

Second evaluate:

[beeme1mr]

Yes, that's expected because the overall size of the buckets changes based on the updated weight. In the example you posted above, you're changing four bucket sizes at once, which will cause some thrashing. However, if you only had two variants, say "on" and "off", then slowly increasing the bucket size of "on", requests that evaluate to "on" will always stay on as long as you either keep the weight the same or increase the weight. If you decrease the weight, you run the risk of reverting to the "off" variant.

Here's a visual representation of what you're seeing above.

[pmarchini]

Thank you for the explanation and confirmation!
I must have misinterpreted the concept of "sticky bucketing"; I would have expected an already assigned value to never change the returned variant.

I understand that such behavior requires at least a persistence layer somewhere, which would invalidate the design of flagd (if I got that right).

[beeme1mr]

As far as I know, this is how most feature flagging SDKs work to remain deterministic and stateless. It's theoretically possible to do the calculation yourself in a before hook and include that value in context. You may also be able to store information on a session or cookie to prevent unexpected variant thrashing. It's an interesting use case, but I don't think we'll be able to support it natively in flagd. If you do find a solution, please let us know so we can include a guide in our docs. Thanks!