API Throttling using Redis and FastAPI (Dockerized)

API Throttling is when you limit the number of calls on your api endpoint such that your valuable or heavy services do not get overwhelmed by a single user or a group of user. It is absolutely necessary when you are trying to sell APIs or trying to provide your endpoint to public.

In this tutorial we are gonna achieve this by protecting a FastAPI endpoint using redis. Redis is a super fast in-memory datastore structure, it follows NoSQl architecture meaning in consists of key-value pairs. Thus making it very fast to access and is widely used for server-side caching of data, however it should not be used as a primary memory store as it is very volatile.

So how are going to achieve this. We are going to use the request ip address to rate limit the user. This can also be achieved by using a token system(Ex: JWT, etc) but that would be a bigger topic.

One of the features of redis is that we can create a key-value pair and set to expire, it stay for a certain amount of time called Time To Live (ttl from no onwards). So using the user’s ip address we are going a create a key-value pair which will expire in 60 seconds. We are going to increase the value of this everytime there is a call to our endpoint and check if it exceeds the necessary number of calls, at which point we will raise an exception. The key-value pair will expire in 60 seconds thus reseting the value. Thus effectively you are controlling the number of calls a user makes in a minute.

Lets get coding now… I am doing this with docker so you need to have docker in your system before attempting this.

The whole code repo is available here

The file structure will look like this:

Technically you don’t need a virtual env but this was a converted project so I have it.

Now run

pip install fastapi uvicorn redis

Lets the dependencies install and then create the requirements.txt file by running:

pip freeze > requirements.txt

Lets make the main file next:

from fastapi import FastAPI, Request, HTTPException, status
app = FastAPI()
@app.get("/")
def test(request: Request):
clientIp = request.client.host
return {
    "message": "Hello world",
}

This will return a simple hello world message on localhost:8000

Lets setup docker…

Code for Dockerfile:

FROM python:3.8.3
WORKDIR '/backend'
COPY /requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

Code for docker-compose.yml :

version: "3"
services:
    redis-server:
      image: redis:latest
    fapi:
      image: python:3.8.3
      build: .
      restart: always
      depends_on:
        - redis-server
      volumes:
        - .:/backend
      ports:
       - "8000:8000"

Be careful if you are copying code from here are yml files are notorious for there spacing syntax. We have two services here 1) The redis-server which we will use for the objective 2) The main fastapi app which will user the Dockerfile me just made to build it’s image.

(PS: We are setting up volumes here because we are in a development environment, you may not do that in a production environment)

Now if you use the command:

docker-compose up

Your docker image should build up on it’s own and serve you fastapi app on post 8000. IMP: To see the endpoint you need to go to 127.0.0.1:8000 and not localhost.

You should see something like this. May take some time to pull images for the first time.

Lets get redis ready now, make a redis.py file and use this code:

import redis
client = redis.Redis(host="redis-server")

def limiter(key, limit):
    req = client.incr(key)
    if req == 1:
        client.expire(key, 60)
        ttl = 60
    else:
        ttl = client.ttl(key)
    if req > limit:
        return {
            "call": False,
            "ttl": ttl
        }
    else:
        return {
            "call": True,
            "ttl": ttl
        }

This code performs the before explained logic and returns 2 values: mentioning if the call is possible and the ttl. IMP: When connecting to the redis service I am using “redis-server” as mentioned in the service in the docker-compose.yml file

Now lastly lets implement this logic in the main.py file, the new code should look like this:

from fastapi import FastAPI, Request, HTTPException, status
from .redis import limiter
app = FastAPI()
@app.get("/")
def test(request: Request):
    clientIp = request.client.host
    res = limiter(clientIp, 5)
    if res["call"]:
        return {
            "message": "Hello world",
            "ttl": res["ttl"]
        }
    else:
       raise      HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE,      detail={
       "message": "call limit reached",
       "ttl": res["ttl"]
    })

Pretty straightforward here, we extract the ip of the request and send it to our function with number of calls we are going to allow per minute.

If we get a favorable return from the function, we return the desired result along with the ttl, or else we raise a 503 exception and return an error message.

I suggest restarting your docker network once after this. Do that by CTRL+C in the terminal and then run

docker-compose up --build

You should see results in your browser. When you are within limit you see a 200.

Click execute again and you will see that the ttl decrease.

Once you reach the limit(5 here) you will see a 503 error.

And Voila! your rate limiter works.

If this helped you make sure to tell me in the comments. Also mention any doubts or if you want the JWT version of it.

Github

LinkedIn

Thanks.