GH Repository · EricLBuehler
mistral.rs
Fast, flexible LLM inference
- stars
- 7,693
- 30-day movement
- +155/day
- Related entries
- 0
- Connections
- 3
makedockerotherRustuqffrustllm
mistral.rs is an LLM inference engine written in Rust, described by its author as fast and flexible. The topics list also references UQFF, which by name suggests a quantization-related format used by the project.
Reach for it when you want a Rust-based engine for running large language model inference locally.
Use it to
- Run LLM inference locally
- Serve models from a Rust toolchain
- Experiment with quantized model formats
- Build applications on a fast inference backend
For Developers running LLM inference, especially in Rust environments
- Role
- other
- Language
- Rust
- Licence
- MIT
- Forks
- 702
- Open issues
- 240
- Last push
- 2026-09-08
- Latest release
- v0.1.0 · 2024-04-27
topicsllminferencerustquantizationopen-source