A LLM semantic caching system aiming to enhance user experience by reducing response time via cached query-result pairs.
-
Updated
Jun 30, 2025 - Python
#
semantic-cache
Here are 47 public repositories matching this topic...
Redis Vector Library (RedisVL) -- the AI-native Python client for Redis.
python search redis openai embedding redis-search vector-search huggingface vector-database large-language-models llm anthropic semantic-cache retrieval-augmented-generation llmcache
-
Updated
Mar 26, 2026 - Python
mimir is a drop-in proxy that caches LLM API responses using semantic similarity, reducing costs and latency for repeated or similar queries.
-
Updated
Dec 24, 2025 - Go
SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features semantic caching, model profiling, and automatic failover for local AI labs.
docker self-hosted model-serving gpu-monitoring fastapi llm openai-proxy semantic-cache local-llm ollama llm-proxy ollama-api ai-gateway llm-router self-hosted-ai ai-cache
-
Updated
Mar 23, 2026 - Python
Reliable and Efficient Semantic Prompt Caching with vCache
machine-learning consistency cache chatbot openai llama gpt verified memcache correctness semantic-search similarity-search rag vector-search guarantees vector-database llm vllm semantic-cache llm-memory
-
Updated
Dec 17, 2025 - Python
One API for 25+ LLMs, OpenAI, Anthropic, Bedrock, Azure. Caching, guardrails & cost controls. Go-native LiteLLM & Kong AI Gateway alternative.
azure mcp gateway bedrock kong guardrails pii-detection llm llmops anthropic semantic-cache qwen llm-proxy litellm ai-gateway deepinfra moonshot-ai ai-infrastructure llm-cost llm-gateway
-
Updated
Mar 27, 2026 - Go
Redis Vector Library (RedisVL) -- the AI-native Java client for Redis.
java redis ai embeddings vectors rag vector-search vector-database llm generative-ai semantic-cache llm-cache rag-chatbot semantic-routing agentic-ai
-
Updated
Mar 17, 2026 - Java
This is a RAG based chatbot in which semantic cache and guardrails have been incorporated.
-
Updated
Nov 11, 2024 - HTML
This repository contains sample code demonstrating how to implement a verified semantic cache using Amazon Bedrock Knowledge Bases to prevent hallucinations in Large Language Model (LLM) responses while improving latency and reducing costs.
agent aws demo bedrock rag aws-blog llm semantic-cache llm-agent amazon-bedrock amazon-bedrock-agents amazon-bedrock-knowledge-bases
-
Updated
Apr 3, 2025 - Jupyter Notebook
High-performance LLM query cache with semantic search. Reduce API costs 80% and latency from 8.5s to 1ms using Redis + Qdrant vector DB. Multi-provider support (OpenAI, Anthropic).
redis embeddings openai cost-optimization rag fastapi vector-database qdrant semantic-cache llm-caching
-
Updated
Dec 2, 2025 - Python
Enhance LLM retrieval performance with Azure Cosmos DB Semantic Cache. Learn how to integrate and optimize caching strategies in real-world web applications.
-
Updated
Mar 22, 2024 - Python
Ultra-fast Semantic Cache Proxy written in pure C
-
Updated
Jan 28, 2026 - C
Redis Vector Similarity Search, Semantic Caching, Recommendation Systems and RAG
redis vector vector-search llm semantic-cache redis-vector-search retrieval-augmented-generation dalle-3
-
Updated
Apr 3, 2024 - Python
VCAL Core — high-performance semantic cache and vector cache library for LLM applications.
in-memory semantic-search similarity-search hnsw vector-search semantic-cache llm-cache ai-infrastructure ai-cache vector-cache
-
Updated
Feb 22, 2026 - Rust
A ChatBot using Redis Vector Similarity Search, which can recommend blogs based on user prompt
python redis chatbot vector-search vector-database sentence-transformers huggingface-transformers large-language-models llm generative-ai semantic-cache redis-vector-search llmcache
-
Updated
Sep 30, 2023 - Python
Optimized RAG Retrieval with Indexing, Quantization, Hybrid Search and Caching
quantization hnsw huggingface hybrid-search large-language-models splade qdrant-vector-database semantic-cache retrieval-augmented-generation
-
Updated
Nov 6, 2024 - Python
Adaptive semantic cache for LLMs with streaming support, ML-based thresholds, and real-time cost tracking. Built in Rust for sub-millisecond performance.
infrastructure rust redis middleware machine-learning embeddings tokio observability prometheus-metrics ai-agents cost-optimization terminal-ui adaptive-thresholding vector-database qdrant semantic-cache llm-ops llm-optimization llm-cache agent-memory
-
Updated
Jan 14, 2026
An operating system for autonomous AI agents — 5-tier cache-first routing (97.5% cost reduction), Ed25519 constitution enforcement, 130 agents, 106 plugins. Rust.
nlp rust machine-learning automation privacy chatbot wasm self-hosted multi-agent cybersecurity personal-assistant ai-safety ai-agents intent-classification onnx agent-framework llm prompt-injection semantic-cache agentic-ai
-
Updated
Mar 19, 2026 - Rust
Semantic cache layer for LLM APIs — embed prompts locally, find near-matches, skip redundant LLM calls.
-
Updated
Mar 10, 2026 - Python
Improve this page
Add a description, image, and links to the semantic-cache topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the semantic-cache topic, visit your repo's landing page and select "manage topics."