Skip to content

Stream video uploads to disk instead of buffering whole files #12

@luojiyin1987

Description

@luojiyin1987

Summary

The video upload endpoint currently calls await file.read() and only checks the size after the entire upload has been buffered in memory. With a 500MB limit, one request can consume hundreds of MB in the app process before rejection.

Impact

  • High peak memory usage per upload request
  • Concurrent uploads can exhaust FastAPI worker memory
  • Oversized uploads are rejected too late, after the buffering cost has already been paid

Proposed fix

  • Write uploads to disk in chunks
  • Count bytes while streaming and reject as soon as the request exceeds MAX_FILE_SIZE
  • Clean up the temp session directory on size-limit or write failures
  • Add regression coverage for successful streamed uploads and oversize rejection

Notes

A fix branch already exists in my fork: luojiyin1987:fix/stream-upload.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions