llmfit: What LLMs run on my PC?

Tue, 21 Apr 2026 00:00:00 +0000

Hi! LinkedIn told me there is a repository that shows you the models (LLMs, VLMs) that better fit your GPU and PC. It is called llmfit (link to the repo in the resources section). Let’s take a look!

Installation

Couldn’t be easier. Run on your linux terminal, no sudo required:

curl -fsSL https://llmfit.axjns.dev/install.sh | sh -s -- --local

Then, type llmfit on your terminal and that’s it!

Execution and Filtering

As soon as you run it, you find a list of 900+ models under the main hardware specs of your PC (CPU, RAM, GPU and VRAM), with 72 models hidden by incompatible backend though…

My list was comprised of 913 models, ordered by score. In addition to the name and score columns, we have: provider, number of parameters in billions, throughput in tokens per second, quantization type, size in disk, mode, memory percentage, context window, date, fit and use case. I couldn’t find the models from the Swallow LLM team in Japan, finetuned for Japanese proficiency (no nitpicking intended!).

Alibaba

If someone is committed to the local/open-AI-model scene is Alibaba from China, with their famous Qwen models. Qwen2.5-Coder-14B-Instruct-AWQ scored 94 on my hardware. Could be a nice prospect for a coding agent of some sort.

Google

Being as renowned as the bigger sibling Gemini, the Gemma family of models is great for general and multimodal local AI. gemma-3-12b-it scored 90 on my hardware. I’ve used it before and got good results on OCR tasks, even though I got better results with Mistral-Small3.2 (not in my GPU but in a more capable RTX 5090).

Mistral

And speaking of Mistral AI… Vive la France! I am very fond of the Mistral models. While the spotlight is elsewhere, they have built great local models, specially in terms of text extraction from images. That’s why I immediately noticed the absence of Mistral-Small3.1:24b and Mistral-Small3.2:24b on my list. However, checking the official llmfit repo, I found that might have been due to an updating problem on my end, because Mistral-Small3.1:24b is considered by the llmfit team.

Microsoft

Microsoft also has interesting local models. I’m very fond of phi-4:14b, which scored 90 on my PC. It’s meant for coding according to llmfit, but in practice it’s good for chat and multi-lingual NLP too. It’s a 14B powerhouse in my opinion, after using it for almost a year at work. I need to try their newer Phi-4-reasoning-vision-15B, released on March 4, 2026.

NVIDIA

Even though I own 3 NVidia GPUs, I’ve never used an LLM released by them. Thanks to llmfit, I noticed a model that could be useful for my recent Japanese endeavours: NVIDIA-Nemotron-Nano-9B-v2-Japanese. Fits perfectly on my system, I should try it with

OpenAI

Who would have thought OpenAI is not that open. Just kidding. With only two models on the list, gpt-oss-20b is my best option, although the 120B variant apparently runs on my system (Mixture of Experts Mode). I know I need to take advantage of my RAM somehow, by loading some parameters there, just have to learn how.

Perfect Fit

By pressing f, you can filter by fit. It turns out 657 out of 913 models fit my system perfectly. The best one comes from a provider called huihui-ai, their version of Llama-3.2-11B-Vision-Instruct. What might be the differences with the one made by Meta?

Takeaways

llmfit is easy to install, run and navigate.
Beautiful, clean CLI.
I could not find some models, e.g. the whole Swallow-LLM family of LLMs from Japan.

Thank you for reading!

Local AI |