If you are choosing between GGUF, Safetensors and GPTQ, the first useful step is to notice that the comparison is slightly malformed. GGUF and Safetensors are containers — ways of laying out numbers and metadata inside a file. GPTQ is a quantisation method, and its weights are usually distributed inside a Safetensors container rather than in a format of their own. Once you separate those two ideas, most of the confusion around local model files disappears.
This article is written for people who are technically curious but not necessarily deep in the weeds: you have downloaded a model, or you are about to, and you want to know what the file extension actually commits you to. The short version is that you should pick your runtime first, then pick the format it accepts.
Two questions hiding inside one comparison
Model files answer two independent questions. The first is how the numbers are stored: what the header looks like, whether tensors are memory-mappable, whether any code runs when the file is opened. The second is how precisely the weights are stored: full precision, or reduced to 8-bit, 4-bit, or something else. GGUF and Safetensors are answers to the first question. GPTQ and AWQ are answers to the second.
Conflating them leads to statements like “GGUF is a 4-bit format,” which is not accurate. A GGUF file can hold tensors at several different quantisation levels, and a Safetensors file can hold full-precision weights or quantised ones. The container tells you how to read the bytes; the quantisation tells you how much information survived.
GGUF: the local-inference single file
GGUF was developed for the llama.cpp project, and it remains the format most associated with running models on your own machine. It is a binary container built for fast loading and saving, and for readability of its metadata. A GGUF file has three parts: a metadata section of key-value pairs describing the architecture, version and hyperparameters; a tensor metadata section giving each tensor’s shape, data type and name; and then the tensor data itself (Common AI Model Formats).
The practical consequence of that design is that a single GGUF file is self-describing enough that it needs no accompanying tokenizer or config file. That is why it dominates the local-inference world: you download one file, point a runtime at it, and it works. GGUF is supported by llama.cpp, ollama and vLLM, among other runtimes. It is primarily used for language models; using it for diffusion models through stable-diffusion.cpp is possible but less common.
If your target is a laptop, a CPU-only box, or an Apple silicon machine, GGUF is usually the format you will be working with — not because it is faster in the abstract, but because that is the format the runtimes on those machines consume.
Safetensors: a header, then raw bytes
Safetensors is deliberately simple. The file begins with a JSON header that gives each tensor a name, a dtype, a shape and a byte range, and then the raw bytes follow. There is no code and nothing is executed when the file is loaded. Because the layout is fixed, the file is memory-mappable, so a large model can be loaded without being copied through RAM first (GGUF, Safetensors and Model File Formats).
That combination — no execution, predictable layout — made Safetensors the default for Hugging Face releases. It is also the carrier format for a lot of quantisation work. When you download a GPTQ or AWQ model, what you usually receive is a Safetensors file with extra metadata describing how the weights were quantised.
One caveat worth stating plainly: “Safetensors” describes the container, not the contents. A Safetensors file is not automatically a good model, a small model, or a safe-to-run-anywhere model. It simply means the loader will not run code from the file.
GPTQ: a quantisation method that travels inside Safetensors
GPTQ is one of the GPU-native quantised weight formats, alongside AWQ and similar schemes. These are usually distributed as Safetensors with extra metadata, and they are aimed at batching servers on datacentre GPUs rather than at laptops. Crucially, they are not interchangeable with GGUF.
That last point causes a lot of wasted downloads. A GPTQ checkpoint and a GGUF checkpoint of the same base model are not two encodings of the same artefact that any runtime can read. They target different execution stacks. If your runtime expects GGUF, a GPTQ directory will not substitute for it, and vice versa.
The legacy format worth avoiding: .bin, .pt and .pth
The older PyTorch checkpoint formats — .bin, .pt and .pth — are a zip archive containing tensor data plus a Python pickle describing how to rebuild the object. This matters because unpickling executes code by design. A checkpoint can therefore contain instructions that run when the file is deserialised, before any inference happens and before you see a token, with the privileges of whoever ran the loader. Nothing about this requires a sophisticated attack.
These files still appear on older repositories and in research artefacts. When a modern alternative exists, there is little reason to prefer them.
A like-for-like comparison
| Format | What it actually is | Typical home | Runs code on load? |
|---|---|---|---|
| GGUF | Single-file container: key-value metadata, tensor metadata, quantised tensors | Local inference on CPU, Apple silicon, single machines | No |
| Safetensors | JSON header (name, dtype, shape, byte range) followed by raw bytes | Hugging Face releases; carrier for quantised weights | No |
| GPTQ / AWQ | Quantisation schemes, usually shipped as Safetensors with extra metadata | Batching servers on datacentre GPUs | No (container is Safetensors) |
| .bin / .pt / .pth | Zip archive plus a Python pickle | Older repositories and research artefacts | Yes |
| ONNX / MLX / CoreML | Runtime-specific packagings | Conversion targets for particular runtimes and Apple hardware | Depends on runtime |
The table compares documented characteristics, not measured performance. No speed or quality claims are implied by the ordering.
Choosing by scenario
You are running on a laptop, a CPU-only machine, or Apple silicon. Look for GGUF. The single-file design removes the tokenizer and config juggling that other formats require, and it is the format the local runtimes consume.
You are publishing or archiving a model. Safetensors is the default on Hugging Face, and its lack of executable content is a genuine advantage when other people will load your files.
You are serving batches on datacentre GPUs. GPTQ and AWQ are the formats built for that environment. They are not the right choice for a laptop, and a laptop-oriented GGUF file is not the right choice for a batching server.
Your deployment target demands a specific runtime. ONNX, MLX and CoreML are conversion targets rather than distribution formats — you meet them when a particular runtime requires one, not because they are a better way to store weights.
What the file format does not tell you
Three things are routinely mistaken for format properties and are not.
- Quantisation level. The container does not fix the precision. A Safetensors file can hold full-precision or 4-bit weights; a GGUF file can hold several quantisation levels. Read the model card, not the extension.
- Memory use. The size of the weights file is not the amount of memory inference will consume. Context length and the KV cache, plus runtime overhead, add on top, and they grow with how much you ask the model to hold in context.
- Licence. Open weights, open source and commercial-use permission are three different things. The file format says nothing about any of them; check the applicable licence for the specific model.
A pre-flight checklist before you download
- Name the runtime you intend to use, and confirm which formats it accepts.
- Check the repository for the file extension you need. If it only offers Safetensors and you need GGUF, you are looking at the wrong repository or a conversion step you have not planned for.
- Read the quantisation level from the model card, not from the filename alone.
- Confirm the licence for the specific weights you are downloading.
- If the repository only offers .bin, .pt or .pth files, treat that as a signal to check whether a Safetensors or GGUF release exists elsewhere.
A useful verification step with Safetensors: because the header is JSON, it can be read without loading the tensors. Inspecting the header tells you the tensor names, dtypes and shapes before you commit memory to the model.
Limits of this comparison
This article describes what the formats are and where they are used. It does not provide speed, throughput or quality comparisons, because no reproducible measurements were available for it. If you are choosing between two quantisation levels of the same model, that is a separate question from choosing a container format, and it needs its own testing on your own hardware.
It is also worth noting that the ecosystem moves. Runtime support for a given format is a moving target, so the authoritative answer for your setup is the documentation of the runtime you actually run.
Your next step
Decide on the runtime first. If it is llama.cpp, ollama or a similar local runner, go looking for GGUF. If you are loading weights into a Python stack or publishing a model, Safetensors is the default and the safer container. If you are batching on datacentre GPUs, GPTQ or AWQ is the family to investigate. Then check the quantisation level and the licence separately, because neither one is encoded in the file extension.
If you have hit a specific format mismatch — a runtime refusing a file, or a repository that only ships one container — share your runtime, model and the exact error in the community so others can compare against their own setup.
FAQ
Is GGUF the same thing as a 4-bit quantisation?
No. GGUF is a container format. It can hold tensors at different quantisation levels, so the extension alone does not tell you the precision. Read the model card for the quantisation level.
Can I use a GPTQ model with llama.cpp or ollama?
GPTQ and similar GPU-native quantised weights are described as not interchangeable with GGUF. Whether a conversion path exists depends on the model and the tooling, so verify against the documentation of the runtime you intend to use rather than assuming the file will load.
Does Safetensors mean the model is safe?
It means the container does not execute code when loaded, which removes a specific class of risk associated with pickle-based checkpoints. It says nothing about the quality of the weights, the quantisation level, or the licence.
Why do some repositories still ship .bin or .pt files?
These are legacy PyTorch checkpoints, still found on older repositories and in research artefacts. When a Safetensors or GGUF release exists for the same model, there is little reason to prefer them.
Does the file size tell me how much RAM I need?
Not on its own. Weights size is only part of the picture; context length and the KV cache, plus runtime overhead, add further memory use during inference.