BigHugger
sk Skill · majiayu000

caching-architecture

LiteLLM-RS response caching architecture. Covers the two-tier deterministic cache (L1 in-memory + optional L2 Redis) behind LLMCache and DualCache, SHA-256 cache key generation with schema versioning, TTL and eviction policy, request-path wiring for chat completions and embeddings, cache statistics, and admin endpoints. Use when adding or tuning gateway response caching — cache keys, tiers, TTLs or eviction, cache…

installs 8w
0
30-day movement
starts with the next reading
Related entries
1
Connections
0
yamlrustRust
Host repository
majiayu000/litellm-rs
Host stars
113
Host language
Rust