What AI model can my computer run?

How to read it

A local model runs fastest when the whole file sits in your graphics card's memory (VRAM), or on a Mac, in the part of unified memory its graphics are allowed to use. Each model comes in several quantizations. Of the ones here, Q8_0 is the largest and closest to the original (bartowski: "extremely high quality, generally unneeded"), and Q4_K_M a little over half its size, which he recommends as the default for most uses. A bigger quantization is a little better and needs more memory. For every model the table shows each quantization's file size and whether it fits:

• fits the file plus 2 GB to spare fits in the memory. The table's "best fit" is the largest of these.
• tight only 1 to 2 GB to spare. It may load, but a long conversation can run out of room.
• split too big for the card alone, but it fits in the card and your system RAM together, with the same room to spare. It runs, much slower, with part of the model on the processor. Enter your system RAM to see these.
• too big it doesn't fit.

Memory is counted cautiously. A card sold as 16 GB is taken as 16 billion bytes, a little less than it really has, and each file at the size Hugging Face lists for it. So a model this tool says fits does fit by the rule. Longer conversations (a bigger context) use more memory on top of the file, which is what the room to spare is for.

Why a Mac gets less than its full memory

On a Mac the processor and graphics share one pool of memory, but macOS lets the graphics use only part of it by default. The rule in Apple's driver, as decompiled and posted in llama.cpp's discussion, holds back 33% of memory, or 25% on a Mac with more than 32 GB. A 16 GB Mac's graphics can therefore use about 10.7 GB, and a 64 GB Mac's about 48 GB. The tool checks against that share. The same discussion shows the limit can be raised with sudo sysctl iogpu.wired_limit_mb, at the cost of memory for everything else, and it resets on restart. The figure dates from 2023, and Apple has not published it, so newer macOS versions may differ.

Speeds

A speed is shown only where a publication measured that model, at that quantization, on that graphics card or Mac, generating text with llama.cpp or a tool built on it (Ollama, LM Studio). It is tokens of text a second. Where no one has published a figure for the pair, the table says so and shows no number. We never work one out from memory bandwidth or another card's result. Generation speed rises with memory bandwidth, and a model split with system RAM runs far slower than one held fully on the card.

The models and their sources

The file sizes are from bartowski's quantizations on Hugging Face, in GGUF, the format llama.cpp, Ollama and LM Studio load, each read from the file's own page:

Rules quoted: bartowski, "Which file should I choose?" on his GGUF model cards ("Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM", and the same with system RAM added for maximum quality), read 2026-10-19; macOS's GPU memory share from llama.cpp discussion #2182, posted 2023-07-13. File sizes read from Hugging Face on the dates linked. Speeds: each from the page it links to.