AlexsJones / llmfit

llmfit English · 中文 **Hundreds of models & providers. One command to find what runs on your hardware.** A terminal tool that right-sizes LLM models to your system's RAM, CPU, and GPU. Detects your hardware, scores each model across quality, speed, fit, and context dimensions, and tells you which ones will actually run well on your machine. Ships with an interactive TUI (default) and a classic CLI mode. Supports multi-GPU setups, MoE architectures, dynamic quantization selection, speed estimation, and local runtime providers (Ollama, llama.cpp, MLX, Docker Model Runner). > **Sister project:** Check out sympozium for managing agents in Kubernetes. --- Install Windows If Scoop is not installed, follow the Scoop installation guide. macOS / Linux Homebrew Quick install Downloads the latest release binary from GitHub and installs it to (or if no sudo). **Install to without sudo:** Docker / Podman This prints JSON from command. The JSON could be further queried with . From source --- Usage TUI (default) Launches the interactive terminal UI. Your system specs (CPU, RAM, GPU name, VRAM, backend) are shown at the top. Models are listed in a scrollable table sorted by composite score. Each row shows the model's score, estimated tok/s, best quantization for your hardware, run mode, memory usage, and use-case category. | Key | Action | |---|---| | / or / | Navigate models | | | Enter search mode (partial match on name, provider, params, use case) | | or | Exit search mode | | | Clear search | | | Cycle fit filter: All, Runnable, Perfect, Good, Marginal | | | Cycle availability filter: All, GGUF Avail, Installed | | | Cycle sort column: Score, Params, Mem%, Ctx, Date, Use Case | | | Enter Visual mode (select multiple models) | | | Enter Select mode (column-based filtering) | | | Cycle color theme (saved automatically) | | | Open Plan mode for selected model (hardware planning) | | | Open provider filter popup | | | Open use-case filter popup | | | Open capability filter popup | | | Mark selected model for compare | | | Open compare view (marked vs selected) | | | Clear compare mark | | | Toggle installed-first sorting (any detected runtime provider) | | | Download selected model (provider picker when multiple are available) | | | Refresh installed models from runtime providers | | | Toggle detail view for selected model | | / | Scroll by 10 | | / | Jump to top / bottom | | | Quit | Vim-like modes The TUI uses Vim-inspired modes shown in the bottom-left status bar. The current mode determines which keys are active. Normal mode The default mode. Navigate, search, filter, and open views. All keys in the table above apply here. Visual mode ( ) Select a contiguous range of models for bulk comparison. Press to anchor at the current row, then navigate with / or arrow keys to extend the selection. Selected rows are highlighted. | Key | Action | |---|---| | / or arrows | Extend selection up/down | | | Compare all selected models (opens multi-compare view) | | | Mark current model for two-model compare | | or | Exit Visual mode | The multi-compare view displays a table where rows are attributes (Score, tok/s, Fit, Mem%, Params, Mode, Context, Quant, etc.) and columns are models. Best values are highlighted. Use / or arrow keys to scroll horizontally if more models are selected than fit on screen. Select mode ( ) Column-based filtering. Press (shift-v) to enter Select mode, then use / or arrow keys to move between column headers. The active column is visually highlighted. Press or to activate the appropriate filter for that column: | Column | Filter action | |---|---| | Inst | Cycle availability filter | | Model | Enter search mode | | Provider | Open provider popup | | Params | Open parameter-size bucket popup (<3B, 3-7B, 7-14B, 14-30B, 30-70B, 70B+) | | Score, tok/s, Mem%, Ctx, Date | Sort by that column | | Quant | Open quantization popup | | Mode | Open run-mode popup (GPU, MoE, CPU+GPU, CPU) | | Fit | Cycle fit filter | | Use Case | Open use-case popup | Row navigation ( / ) still works in Select mode so you can see the effect of filters as you apply them. Press to return to Normal mode. TUI Plan mode ( ) Plan mode inverts normal fit analysis: instead of asking "what fits my hardware?", it estimates "what hardware is needed for this model config?". Use on a selected row, then: | Key | Action | |---|---| | / / | Move between editable fields (Context, Quant, Target TPS) | | / | Move cursor in current field | | Type | Edit current field | | / | Remove characters | | | Clear current field | | or | Exit Plan mode | Plan mode shows estimates for: • minimum and recommended VRAM/RAM/CPU cores • feasible run paths (GPU, CPU offload, CPU-only) • upgrade deltas to reach better fit targets Themes Press to cycle through 6 built-in color themes. Your selection is saved automatically to and restored on next launch. | Theme | Description | |---|---| | **Default** | Original llmfit colors | | **Dracula** | Dark purple background with pastel accents | | **Solarized** | Ethan Schoonover's Solarized Dark palette | | **Nord** | Arctic, cool blue-gray tones | | **Monokai** | Monokai Pro warm syntax colors | | **Gruvbox** | Retro groove palette with warm earth tones | CLI mode Use or any subcommand to get classic table output: REST API ( ) starts an HTTP API that exposes the same fit/scoring data used by TUI/CLI, including filtering and top-model selection for a node. Supported query params for / : • (or ): max number of rows returned • : (forces perfect-only when ) • : • : • : • : provider text filter (substring) • : free-text filter across name/provider/size/use-case • : • : include non-runnable rows (default on , on ) • : per-request context cap for memory estimation Validate API behavior locally: GPU memory override GPU VRAM autodetection can fail on some systems (e.g. broken , VMs, passthrough setups). Use to manually specify your GPU's VRAM: Accepted suffixes: / / (gigab…