Quantization Explained: GGUF, GPTQ, AWQ, and How They Affect Local Model Quality

Quantization stores a model’s weights at lower numerical precision than the FP16 or BF16 checkpoint it was trained in, trading a small amount of accuracy for a much smaller memory footprint and, usually, faster inference. For most people running models locally in 2026, the practical answer is short: a 4-bit build is the production sweet spot, and the bit width you choose matters more than the three-letter label on the file. The rest of this article explains what each label means, where the quality actually goes, and how to decide when the difference is worth paying for.

It is written for readers who have downloaded a GGUF file, seen suffixes such as Q4_K_M, and wondered whether they are giving up quality, or spending memory on a quantization that buys them nothing measurable.

What quantization changes, and what it leaves alone

Quantization replaces the high-precision numbers that make up a model’s weights with a much smaller set of representable values. Rather than storing every weight as a 16-bit float, the quantizer groups weights together, rounds each group to a low-bit representation, and stores scale factors so the runtime can reconstruct an approximation of the original values. The model architecture, tokenizer, context handling, and prompt format are unchanged. Nothing is deleted and nothing is retrained, which is why quantization is not the same thing as pruning or distillation.

The consequence is a controlled error. Every weight is now slightly wrong, and those small errors accumulate through the network’s layers. How well they are controlled depends on the method used to choose the approximate values, which is where GPTQ, AWQ, and calibration-based approaches differ.

Algorithm, format, and runtime are three different layers

The most common source of confusion in this area is that the names people compare are not the same kind of thing. As llmdeepdive’s lesson on quantization methods puts it, community quantizations mix three layers that must be named separately to be chosen well:

• Algorithm – the method that decides which approximate values to use, such as GPTQ, AWQ, or quantization-aware training (QAT).
• Format – the file layout that stores the quantized tensors and their metadata, such as GGUF.
• Runtime – the kernels, on CPU or GPU, that actually consume the result.

GPTQ, AWQ, GGUF, and QAT each live at a different layer. GGUF is a format used by the llama.cpp ecosystem, not an algorithm. AWQ is an algorithm, but it is also commonly used as shorthand for a family of GPU-oriented model files and runtimes. Comparing “GGUF versus AWQ” as if they were two competitors at the same level produces conclusions that do not survive contact with a real deployment.

The algorithms: GPTQ, AWQ, QAT, and imatrix

GPTQ is a post-training weight-quantization method. It starts from the trained BF16 checkpoint and a set of calibration inputs – sample prompts run through the model to observe how it behaves. It then quantizes the weights column by column, using approximate second-order information to compensate for reconstruction error: the gap between a layer’s outputs before and after quantization. Later weights are adjusted to absorb the damage done by earlier rounding. That preserves layer outputs considerably better than rounding each weight independently to its nearest value. Implementations expose bit width, group size, and ordering choices.

AWQ, or activation-aware weight quantization, takes a different signal. It observes activations on calibration data and uses that information to protect the weights that matter most to the layer’s output, rather than treating all weights as equally important.

QAT, quantization-aware training, moves the process earlier: the model is trained with quantization in the loop instead of being quantized after the fact. It is the most expensive option and the least common in community releases.

imatrix calibration belongs to the same family of post-training, calibration-driven approaches and is widely used in the llama.cpp ecosystem to produce better low-bit GGUF files. The details of any given quantizer matter less to the end user than the resulting artifact: what bit width it targets, and how much quality it retains.

The formats: GGUF, MLX, AWQ, and GPTQ files

GGUF is the llama.cpp file format. It runs on CPU and GPU, has the widest tooling and the largest number of pre-quantized community variants, and is what most local chat applications load by default.

MLX targets Apple Silicon. According to Presenc AI’s 2026 quantization benchmark compilation, MLX 4-bit quantization delivers quality comparable to GGUF Q4 at slightly faster inference, but with a smaller community ecosystem and fewer pre-quantized model variants.

AWQ and GPTQ files are generally consumed by GPU inference stacks rather than by llama.cpp. The format must match the runtime: a quantized file is not portable across every loader just because the underlying algorithm is the same.

What the published quality numbers actually say

Presenc AI’s 2026 compilation of published benchmarks (perplexity, MMLU, HumanEval, GSM8K) across GGUF, MLX, AWQ, and GPTQ reaches five findings that are worth internalizing before choosing a file:

• Q4 is the sweet spot. Four-bit weights with mixed-precision activations typically degrade perplexity by 1-3 percent versus FP16.
• Q3 costs real quality. Three-bit quantization shows 3-8 percent perplexity degradation, plus visible regression on reasoning benchmarks such as GSM8K and HumanEval.
• Q5 and Q6 rarely pay off. They recover under 1 percent of perplexity over Q4 while consuming meaningfully more memory.
• AWQ edges out GPTQ on modern models. On most Llama 3+ and Qwen 2+ models, AWQ is roughly 0.5-1.0 percent better on perplexity at the same bit width, attributed to activation-aware scaling.
• MLX 4-bit is competitive on Apple Silicon, matching GGUF Q4 quality with slightly faster inference and fewer available model variants.

The source table illustrates how flat the high end of the curve is. Perplexity here is measured on Wikitext-2, where lower is better:

The pattern is clear: going from 16 bits to 8 bits costs almost nothing measurable, and the meaningful decision happens further down the ladder, between 3-bit and 4-bit builds.

One caveat applies to every memory estimate you will read, including the ones above. Weight size is not total memory use. The KV cache grows with context length, and the runtime adds its own overhead. A 4-bit model that fits comfortably at a short context can still run out of memory at a long one.

Choosing by scenario

If the model fits in memory at Q4 or higher: take the 4-bit build. The 1-3 percent perplexity cost is the price of admission for local inference, and the memory you save buys you longer context or a larger model.

If you are squeezing a model onto limited hardware: a 3-bit build may be the only thing that runs, but go in expecting reasoning and code tasks to degrade before general chat does. GSM8K and HumanEval regressions show up earlier than a casual conversation would reveal.

If you are choosing between Q5 and Q6: the published data says the extra memory buys under 1 percent perplexity recovery over Q4. Spend that memory on context instead.

If you are on an NVIDIA GPU with a serving stack: AWQ has a small but consistent edge over GPTQ at the same bit width on recent models. If both are available for your model, that is a reasonable tiebreaker.

If you are on Apple Silicon: MLX 4-bit is competitive with GGUF Q4 and slightly faster. The trade-off is ecosystem breadth – fewer pre-quantized variants exist, so you may need to convert models yourself.

How to check quality on your own hardware

The following is a suggested verification protocol, not a set of results produced for this article. No measurements were run for this piece, and the numbers cited above come from the published sources referenced at the end.

1. Pick a fixed prompt set that reflects your actual work: a few reasoning questions, a code task, a summarization task, and a long-context task.
2. Fix sampling settings – temperature, top-p, and seed – so that differences reflect the model rather than randomness.
3. Run the same model at a high-precision baseline (FP16, or Q8_0 if FP16 does not fit) and at the quantization you are evaluating.
4. Compare outputs side by side on your tasks. Perplexity is a useful aggregate signal but it is not a substitute for looking at the answers you care about.
5. Repeat with more than one prompt per task type. Single-prompt comparisons are dominated by noise.
6. Record the model version, quantization label, runtime, context length, and hardware alongside your notes, so the result is reproducible later.

If you cannot measure a difference on your tasks, you are probably not paying one. That is the useful outcome of the exercise.

Limitations and open uncertainty

Quality depends on the calibration data used to produce a given quantization, not only on the algorithm and bit width. Two files labeled the same way from different sources can behave differently. Community quantizations of popular models are maintained by those communities rather than by the model’s authors, so version drift is real: a benchmark run against one artifact does not automatically transfer to another.

Published benchmark tables also compress a lot of variation. Perplexity on Wikitext-2 is one narrow measure, and reasoning-benchmark regressions can appear at bit widths where perplexity looks acceptable. Treat the numbers as a map of the terrain, not as a guarantee about your specific model, runtime, and workload.

The practical next step is simple: start from a 4-bit build of the model you actually want to run, verify it on a handful of tasks you care about, and only move to a different bit width when you can point at a specific failure. If you have a result worth comparing – model, hardware, quantization label, and the task where you saw a difference – share it with the community so the next person has one more data point.