Add fairness to matching RPS rate limiter#10370
Draft
prathyushpv wants to merge 3 commits into
Draft
Conversation
Each namespace gets a per-host fair-share quota: share(ns) = MatchingRPS * MatchingNamespaceFairShare(ns). Requests from namespaces exceeding their share are deprioritized so they're throttled first when matching is at its host RPS limit, mirroring the history fairness PR (#10134). The single namespace-level float config also acts as the on/off switch — values outside (0, 1) disable fairness for that namespace; default 0 = off. Also fixes the host RPS interceptor to forward the CallerName header into quotas.Request, which the priority fairness functions read to route to per-namespace buckets. Without this, both this PR and #10134 would be inert in production.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
Add fairness mechanism to matching rps rate limiter. Each namespace will have a matching RPS quota per matching host based on this formula
NamespaceQuota = MatchingRPS * MatchingNamespaceFairShare.
MatchingNamespaceFairShareis a namespace-level float in[0, 1], configurable through dynamic config. If a namespace's request rate exceeds this quota, those extra requests will be deprioritized. If matching rps rate limiter starts to throttle requests, these deprioritized requests will be throttled first. This will prevent unbalanced load from a namespace affecting other namespaces in a matching host. Values outside the open interval(0, 1)disable the mechanism for that namespace (default 0 = off).Also fixes the host RPS interceptor (
common/rpc/interceptor/rate_limit.go) to forward theCallerNameheader intoquotas.Request. The fairness priority function readsreq.Callerto route to the right per-namespace bucket; without this fix the mechanism (and the equivalent one in #10134) is inert.Why?
To prevent noisy neighbour issues in matching rps rate limiter.
How did you test it?
Potential risks
The interceptor fix activates the history fairness mechanism in #10134 for the first time. Both flags default off, so no behavior change unless an operator explicitly opts in per namespace.