Unable to get request body in custom request formatter when using FastAPI

[vstrimaitis]

I'm interested in getting the request body in a custom request formatter when using FastAPI. However, request.body() returns a coroutine and therefore requires the await keyword, but the _format_log_object method is not async. My custom formatter looks something like this:

class _RequestFormatter(json_logging.JSONRequestLogFormatter):
    def _format_log_object(
        self, record, request_util: json_logging.util.RequestUtil
    ):
        request: Request = record.request_response_data._request

        request_body_bytes = await request.body()  # <--- can't do this!
        request_body = bytes.decode(request_body_bytes)

        log_obj = super()._format_log_object(record, request_util)
        log_obj.update({"request_body": request_body})
        return log_obj

Such an approach works in frameworks like Flask, where you don't have to use await to get the body, but fails for FastAPI. Do you have any suggestions on how to work around this?

[bobbui]

i would suggest u extend logging.Formatter for maximum flexibility, something like that

json-logging-python/json_logging/__init__.py

Line 267 in 8d6c1dd

class BaseJSONFormatter(logging.Formatter):

[vstrimaitis]

Yes, I'm aware of them (and I've used them before with Flask). But the methods inside of this class (format and _format_log_object) are not async, so I can't write something like request_body = await request.body(), which is the only way to get the request body in FastAPI.

[bobbui]

@vstrimaitis is not really familiar with async await syntax. But will do some experiment once have time

[stephane-klein]

@vstrimaitis @bobbui I have the same issue. Have you found a solution?

[stephane-klein]

My solution:

with this patch I can execute:

from json_logging.dto import DefaultRequestResponseDTO

...

class async_iterator_wrapper:
    """Class to create async iterator"""

    def __init__(self, obj):
        self._it = iter(obj)

    def __aiter__(self):
        return self

    async def __anext__(self):
        try:
            value = next(self._it)
        except StopIteration:
            raise StopAsyncIteration
        return value


class CustomRequestResponseDTO(DefaultRequestResponseDTO):
    async def async_on_request_complete(self, response):
        super(CustomRequestResponseDTO, self).on_request_complete(response)
        self['request'] = '{} {}'.format(self._request.method, self._request.url)
        resp_body = [section async for section in self._response.body_iterator]
        self['response_body'] = b'\n'.join(resp_body).decode()
        self._response.body_iterator = async_iterator_wrapper(resp_body)

[vstrimaitis]

Hi @stephane-klein! I ended up making a custom middleware class which extends BaseHTTPMiddleware and monkeypatched json_logging.framework.fastapi.implementation.JSONLoggingASGIMiddleware to point to this new class. The actual implementation of the class is exactly like JSONLoggingASGIMiddleware except it additionally extracts the request body from the request object and makes it available as an additional entry in the extra dict. This allowed me to then define the custom request formatter class in a standard way and just extract the request body from the record object.

This is obviously not a clean solution in any sense, but it ended up working for me back when I needed it and the implementation has stayed the same since then 😅