Scaling your API with rate limiters

https://stripe.com/blog/rate-limiters code

Rate limiting can help make your API more reliable in the following scenarios: 下面几个场景需要 rate limiting 支持


At Stripe, we operate 4 different types of limiters in production. The first one, the Request Rate Limiter, is by far the most important one. We recommend you start here if you want to improve the robustness of your API.

Request rate limiter. 限制每秒请求数量. HTTP 403 响应

Pasted-Image-20231225105548.svg

Concurrent requests limiter. 限制并发请求数量. HTTP 429 响应

Pasted-Image-20231225105549.svg

Fleet usage load shedder. 按照预估容量进行配置,对关键请求和非关键请求分离,

We always reserve a fraction of our infrastructure for critical requests. If our reservation number is 20%, then any non-critical request over their 80% allocation would be rejected with status code 503.

Pasted-Image-20231225105545.svg

Worker utilization load shedder. 按照当前可用资源配置,对不同优先级请求分离。这个需要系统自身能发现当前资源使用状态,以决定是否和如何做load shedder.

Pasted-Image-20231225105539.svg


Building rate limiters in practice