Skip to content

openai sse: cap the pending-line buffer in SseState #152

Open

Description

@senamakel

SseState (crates/tinyinference-llm/src/providers/openai/sse.rs) appends every received chunk to buf and never limits the unterminated line it holds. An endpoint that sends a long data: line without a newline can grow the buffer until the stream ends or the request times out.

This affects every streaming caller of the OpenAI-compatible adapter and predates #151, which only routes local-openai invokes through the same parser. Raised in review there (CodeRabbit).

Suggested fix: a fixed maximum for a pending line (for example 1 MiB), checked after each chunk is drained; when it is exceeded, end the stream with a non-retryable ProviderFailed, as the out-of-range tool-call index guard does. Add a test with a synthetic oversized line.

Activity

  1. tinysweeper commented on Nov 11, 2026

    @tinysweeper

    Unbounded pending-line buffer in SseState can grow without limit on a long data: line lacking a newline; affects all streaming callers of the OpenAI-compatible adapter. Suggested fix is a capped buffer (e.g. 1 MiB) that ends the stream with a non-retryable ProviderFailed, plus a test. No candidate duplicates or fixes available.

    Labelled priority: p2.

  2. added theissue type on Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions