GH Repository · zilliztech
GPTCache
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
- stars
- 8,192
- 30-day movement
- starts with the next reading
- Related entries
- 60
- Connections
- 1
agent-frameworkPythonsemantic-searchllamallama-indexredischatbotpythonllmchatgpt-apimilvussimilarity-searchvector-searchbabyagiaigcchatgptautogptopenaidollymemcachelangchainmakegpt
GPTCache is a semantic cache for LLMs, storing and retrieving responses by similarity search rather than exact match. It is a Python library that integrates with LangChain and llama_index and can use vector stores such as Milvus or Redis as backends.
You reach for it to avoid paying for and waiting on repeated LLM calls by serving semantically similar queries from cache.
Use it to
- Cache LLM responses by semantic similarity
- Integrate caching into LangChain pipelines
- Integrate caching into llama_index workflows
- Back the cache with Milvus or Redis
- Reduce OpenAI API costs for chatbots
For Developers building LLM apps who want response caching
- Role
- agent-framework
- Language
- Python
- Licence
- MIT
- Forks
- 596
- Open issues
- 76
- Last push
- 2025-07-11
- Latest release
- 0.1.5 · 2023-04-06
topicsllmsemantic-cachesimilarity-searchlangchainllama-indexvector-search