BigHugger
sk Skill · agentsope

agentsop-vllm

Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.

installs 8w
0
30-day movement
starts with the next reading
Related entries
1
Connections
0
bashPython
Host repository
agentsope/SkillAlchemy
Version
1.0
Host stars
420
Host language
Python