DocSense

A self-hosted document question-answering API: upload PDFs or images, then ask questions and get streamed answers with page-level citations, using a local LLM through Ollama.

Demo

No live demo.

What it does

• Upload and parse documents. POST /ingest accepts PDF, PNG and JPEG (the type is detected from the file's magic bytes, not the client header), stores the file in MinIO and queues a Celery job. The worker parses it with Docling, chunks it with Docling's HybridChunker (512 tokens), embeds the chunks with BAAI/bge-m3 and writes them to Postgres with page number and bounding box.
• Hybrid retrieval. Each query runs a pgvector HNSW cosine search and a Postgres full-text search (plainto_tsquery ranked by ts_rank_cd) in parallel, merges them with Reciprocal Rank Fusion (k=60), then reranks the top results with the BAAI/bge-reranker-v2-m3 cross-encoder.
• Streamed, cited answers. POST /query streams tokens as Server-Sent Events from a local Ollama model (llama3.1 by default). The model is told to cite chunk IDs inline; only IDs that match retrieved chunks are returned as citations (document, filename, page).
• Semantic cache. Answers are cached per user in Redis and reused when a new question's embedding has cosine similarity of at least 0.95 with a cached one over the same document set (7-day TTL, last 100 entries per user).
• Per-user isolation. Documents, chunks and cache entries are scoped by user ID. Auth verifies Clerk RS256 JWTs against a JWKS URL; when CLERK_JWKS_URL is unset, every request runs as a dev user.
• Rate limits and upload limits. Redis-backed per-user limits (10 ingests per minute by default) and a 50 MB upload cap.
• Run logging. Each answered query is written to a runs table with the question, answer, cited chunk IDs and latency.
• Offline evaluation. A RAGAS script scores retrieval and answers on a small fixed dataset (10 question and answer pairs over 2 sample PDFs).

Architecture

flowchart LR
    client[Client] -->|POST /ingest| api[FastAPI app<br/>apps/api]
    client -->|POST /query, SSE| api
    api -->|store file| minio[(MinIO)]
    api -->|doc row, status| pg[(Postgres 16<br/>+ pgvector)]
    api -->|process_document task| redis[(Redis)]
    redis --> worker[Celery worker<br/>apps/workers]
    worker -->|download| minio
    worker -->|Docling parse + HybridChunker<br/>bge-m3 embeddings| worker
    worker -->|chunks + vectors| pg

    subgraph query[Query pipeline: packages/rag/pipeline.py]
        embed[Embed question<br/>bge-m3] --> cache{Semantic cache hit?}
        cache -->|no| search[Dense HNSW search<br/>+ full-text search]
        search --> rrf[RRF fusion]
        rrf --> rerank[Cross-encoder rerank<br/>bge-reranker-v2-m3]
        rerank --> llm[Ollama chat, streamed<br/>packages/agents/chain.py]
    end

    api --> embed
    cache -->|yes| api
    cache <--> redis
    search <--> pg
    llm --> ollama[Ollama server]
    llm -->|tokens, citations| api
    llm -.->|optional trace| langfuse[Langfuse]

Loading

Ingestion is asynchronous: the API commits a pending document row before uploading to MinIO and only enqueues the Celery task after the upload succeeds, so a stuck pending row marks a failed upload. The worker retries up to 3 times with backoff and deletes a document's chunks before inserting, so a retry after a partial write does not collide on primary keys. Vector search and keyword search both live in Postgres, which keeps the stack to one database at the cost of using ts_rank_cd rather than true BM25. Dense search sets hnsw.iterative_scan = 'relaxed_order' so per-user filtering does not starve the HNSW result set when one user owns a small share of the rows (this needs pgvector 0.8 or later). Embedding and reranking run locally on CPU through sentence-transformers, and reranking uses its own thread pool so it does not block other requests' embedding calls. Generation uses a local Ollama model, so there are no API costs, but answer speed depends on the host machine.

Stack

• API: FastAPI, Uvicorn, slowapi (rate limiting), python-jose (Clerk JWT verification), python-magic
• Jobs: Celery with Redis as broker and result backend
• Data: Postgres 16 with pgvector, SQLAlchemy 2 (asyncpg and psycopg2), Alembic
• Storage: MinIO through boto3 and aiobotocore
• Parsing: Docling
• Retrieval models: sentence-transformers with BAAI/bge-m3 (1024-dim embeddings) and BAAI/bge-reranker-v2-m3
• Generation: Ollama Python client, llama3.1 by default
• Observability: structlog request logging, optional Langfuse tracing (see Limitations)
• Evaluation: RAGAS with langchain-ollama as the judge model (optional evals extra)
• Tooling: uv, Ruff, mypy (strict), pytest, GitHub Actions

Run it locally

Requirements: Docker, uv, Python 3.11, Ollama, and libmagic (brew install libmagic or apt-get install libmagic1). No paid API keys are needed. The first run downloads the bge-m3 and reranker models from Hugging Face; HF_TOKEN is optional.

# 1. Install dependencies
uv sync --extra dev

# 2. Configure. .env.example does not include the MinIO settings, and the code's
#    defaults do not match the compose credentials, so add them.
cp .env.example .env
cat >> .env <<'EOF'
MINIO_ENDPOINT=localhost:9000
MINIO_ACCESS_KEY=docsense
MINIO_SECRET_KEY=docsense123
MINIO_BUCKET=docsense-docs
EOF

# 3. Alembic and Celery read some settings from the process environment, not .env
set -a; . ./.env; set +a

# 4. Start Postgres, Redis, MinIO and Langfuse, then run migrations
make dev

# 5. Start the local model
ollama pull llama3.1
ollama serve   # skip if Ollama already runs as a background service

# 6. In separate terminals (each with the env exported as in step 3)
make api      # http://localhost:8000, Swagger UI at /api-docs
make worker

Try it:

curl -F "file=@evals/dataset/docs/technical_overview.pdf" http://localhost:8000/ingest
curl http://localhost:8000/docs/<doc_id>/status       # wait for "done"
curl -N -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "Which model is used for text generation?"}'

Without CLERK_JWKS_URL set, auth is off and requests run as dev_user (override with DEV_USER_ID).

Endpoints

Method	Path	Purpose
POST	/ingest	Upload a file (multipart file), returns doc_id
GET	/docs	List your documents
GET	/docs/{doc_id}	Document detail
GET	/docs/{doc_id}/status	Processing status: pending, processing, done, failed
DELETE	/docs/{doc_id}	Delete a document, its chunks and the stored file
POST	/query	{question, doc_ids?, top_k?}, streams SSE events (token, done, error)
GET	/health	Postgres and Redis status
GET	/auth/me	Current user ID

Configuration

Settings live in packages/models/config.py and are read from .env. The ones you are most likely to change:

Variable	Default	Notes
DATABASE_URL, DATABASE_URL_SYNC	required	async (asyncpg) and sync (psycopg2) URLs
REDIS_URL	redis://localhost:6379/0
MINIO_ENDPOINT, MINIO_ACCESS_KEY, MINIO_SECRET_KEY, MINIO_BUCKET	localhost:9000, minioadmin, minioadmin, docsense-docs	compose uses docsense / docsense123
OLLAMA_BASE_URL, OLLAMA_MODEL, OLLAMA_TIMEOUT	http://localhost:11434, llama3.1, 120
CLERK_JWKS_URL	empty	empty disables auth
LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST	empty, empty, http://localhost:3001	empty keys disable tracing
RATE_LIMIT_INGEST, RATE_LIMIT_QUERY	10/minute, 30/minute
MAX_UPLOAD_BYTES	52428800	50 MB

Tests and evaluation

# Fast suite, as run in CI (needs Postgres, Redis and MinIO from `make dev`)
uv run pytest -m "not slow" -q

# Slow tests: load the real embedding/reranker models, and run RAGAS checks
# (faithfulness >= 0.85, answer relevancy >= 0.80) against a running Ollama
uv sync --extra dev --extra evals
uv run pytest -m slow -v

There are 134 test functions across 17 test files (counted with grep -hE "^\s*(async )?def test_" tests/test_*.py | wc -l). 13 of them are marked slow and are skipped in CI.

CI (.github/workflows/ci.yml) runs Ruff lint and format checks, mypy, and the non-slow pytest suite against Postgres (pgvector), Redis and MinIO service containers. The RAGAS evaluation does not run in CI.

To run the full RAGAS evaluation (Faithfulness, Answer Relevancy, Context Precision, Context Recall, with the Ollama model as judge) over evals/dataset/qa_pairs.json:

uv sync --extra evals
uv run python evals/run_evals.py   # writes evals/results/eval_<timestamp>.json

The script ingests the two sample PDFs directly (no API server) and needs Postgres, Redis and Ollama running.

Project layout

apps/api/        FastAPI app: routes (ingest, docs, query, health, auth) and dependencies
apps/workers/    Celery app and the document processing task
packages/rag/    Embedder, retrieval (dense, full-text, RRF), reranker, semantic cache, query pipeline
packages/agents/ Streaming Ollama generation and citation parsing
packages/db/     SQLAlchemy tables and sessions
packages/ingest/ MinIO storage client and MIME mapping
packages/models/ Settings and Pydantic API models
packages/obs/    structlog middleware and Langfuse tracing
alembic/         Database migrations
evals/           RAGAS runner, QA dataset and sample PDFs
infra/           docker-compose for Postgres, Redis, MinIO and Langfuse
docs/decisions/  Architecture decision records
tests/           pytest suite

Limitations

• DOCX and PPTX are mapped in the code but rejected at upload, because magic-byte detection reports them as ZIP files.
• Audio is not supported.
• Langfuse tracing is written against the v2 SDK (Langfuse.trace()), but uv.lock resolves langfuse 4.5.1, which has no trace() method. Leave the Langfuse keys empty; with them set, queries fail when the trace is created.
• There is no frontend; CORS allows http://localhost:3000 only.
• cost_usd is always 0, since generation runs locally.