FP8 Storage
Esta página aún no está disponible en tu idioma.
FP8 Storage cuts a model’s VRAM roughly in half by keeping its weights on the GPU in 8-bit floating point
(float8_e4m3fn). Before each layer runs, its weights are cast up to the compute precision (FP16 or BF16), and the
stored FP8 weights are put back afterwards, so quality is largely preserved. Measured on Wan 2.2 TI2V-5B (fp16 single
file, 512², 20 steps, RTX 4090), the transformer drops from 9.5GB to 4.9GB in VRAM, at 21.4s per video instead of
18.4s.
It is a per-model setting for full-precision models, and it pairs with Low-VRAM mode: low-VRAM mode streams layers between RAM and VRAM, FP8 Storage shrinks the layers themselves. How it compares with quantized downloads is in the format comparison. To also run the math in FP8 on the GPU’s FP8 hardware, see FP8 Compute.
What it applies to
Section titled “What it applies to”The Model Manager shows the FP8 storage setting only for models where it changes something. A model installed
before this rule may still have it stored as on; it then does nothing, and the log says so once when the model loads:
FP8 Storage is set for '<model>' but does nothing here: <reason>.
By file format
Section titled “By file format”| Model file | FP8 Storage |
|---|---|
| Full precision (FP16, BF16, FP32) | Yes: what it is for. |
| fp8 or scaled fp8 checkpoint | On after install. The weights stay in fp8 with their own scale; without the setting they would be unpacked to BF16 at twice the size. See Checkpoints that are already FP8. |
| MXFP8 checkpoint | Turn it off. It stores one scale per block of 32 weights, which InvokeAI does not keep on any GPU: the layers are unpacked at load, and FP8 Storage then re-encodes them as fp8 without a scale: blocks whose scale is far below 1 lose their weights to underflow. If the file also has per-tensor fp8 layers, those stay fp8 and FP8 Storage changes nothing. |
| int8 and nvfp4 single files | Partly. Their quantized layers stay as they are; only the layers the build ships in full precision are stored in FP8. On an Ideogram 4, Qwen-Image-2.1 or FLUX.3 int8 file, and on a FLUX.3 nvfp4 file, nothing at all. |
| GGUF, NF4 and SDNQ | No: already quantized. The setting is not shown. |
If your model is quantized, its full-precision variant gets the full effect of FP8 Storage.
By model
Section titled “By model”| Model | FP8 Storage |
|---|---|
| Main models: SD1, SD2, SDXL, SDXL Refiner, SD3, CogView4 | Yes |
| FLUX.1, FLUX.2 (all variants) | Yes |
| FLUX.3 | Yes — except int8 and nvfp4 single files, where it does nothing. The toggle can still show as on for an nvfp4 build, since installing switches it on for its fp8 block scales |
| Z-Image, Anima, Krea-2, Qwen-Image, Qwen-Image-2.1, ERNIE-Image, MOSS-Image | Yes |
| Ideogram 4, single-file safetensors | Yes, for each transformer branch’s file |
| LTX-2 | Yes |
| Wan (checkpoint and Diffusers) | Yes, except ComfyUI fp8_scaled files, which load at BF16 |
| ControlNet (SD1, SD2, SDXL), T2I-Adapter | Yes |
| VAE | No: visible decode-quality regression |
| Text encoders, tokenizers | No: small models, and FP8 rounding costs text quality. The FLUX.2 Mistral encoder keeps its scaled fp8 weights packed on any GPU that passes the FP8 test, with no setting. |
| Anima ControlNet-LLLite | No: the adapter is only tens of MB |
| FLUX.1, FLUX.2, Z-Image and Qwen-Image ControlNets | No: not implemented yet |
| MiniMax H3 | No: not implemented yet |
| Ideogram 4, Diffusers folder | No: published only as nf4 or fp8 |
| LoRA, ControlLoRA | No: patched into the base model, not run alone |
Which layers inside a model stay at full precision
Norm layers, position and patch embeddings, and proj_in/proj_out are skipped, so precision-sensitive tiny
learned scalars (such as FLUX’s RMSNorm.scale) aren’t crushed to FP8. This mirrors the diffusers default skip list.
Each model’s own declared exclusions are honored too. Architectures list their precision-sensitive modules
themselves (diffusers calls these _skip_layerwise_casting_patterns and _keep_in_fp32_modules), and those layers
stay at compute precision. This matters where the generic patterns don’t fit the architecture’s naming: Z-Image and
Anima both name their timestep-embedding MLP t_embedder, which no generic pattern matches. Cast to FP8, Z-Image fails
outright and Anima renders a heavily dithered image with no fine detail.
The cost is a slightly smaller saving on the models that declare a lot. Measured against the generic defaults alone:
| Model | Weights kept at compute precision | Saving given up |
|---|---|---|
| Krea-2 | 39 M (time_embed) | ~38 MiB |
| Anima | 19 M (t_embedder, x_embedder, final_layer) | ~18 MiB |
| Wan 14B | 232 M (condition_embedder, patch_embedding) | ~221 MiB |
| MOSS-Image | 486 M (every block’s proj.q, the embedders, modulations and output head) | ~464 MiB |
| FLUX.1, Qwen-Image | 0 | 0 |
MOSS-Image keeps each block’s query projection at compute precision because its forward pass casts the block’s input to that layer’s dtype; the rest of the block is stored in FP8. Measured on an RTX 4090, its transformer takes 5.3GB with FP8 Storage instead of 10.2GB.
InvokeAI applies the cast with its own forward hooks rather than diffusers’ apply_layerwise_casting, on every
nn.Module including diffusers ModelMixin subclasses, so it works with InvokeAI’s CustomLinear layers and with
partial loading. The FP8 cast happens on the CPU, before the weights move to VRAM.
By GPU
Section titled “By GPU”FP8 Storage needs no FP8 hardware: the math still runs in FP16 or BF16. What your GPU adds is speed, through the separate FP8 Compute.
VRAM win only. The transformer or UNet drops by roughly 50% in VRAM. Each step is the same speed or marginally slower,
because every layer casts its FP8 weights up on entry; putting the stored weights back on exit costs nothing. This
covers the RTX 30 series and Ampere workstation cards, and older CUDA cards such as the RTX 20 and GTX 16 series:
float8_e4m3fn is a pure storage type that any CUDA device can hold. RTX 3090 owners fitting FLUX into 24GB benefit
the most.
The same ~50% VRAM win. These GPUs have FP8 tensor cores, but FP8 Storage alone does not use them: the weights are cast up to BF16 before each layer. To use them for checkpoints published in FP8, turn on FP8 Compute.
A ROCm build of PyTorch reports an AMD GPU as a CUDA device, so FP8 Storage works on AMD as it does on NVIDIA. Intel
GPUs (XPU) are supported too. Before the first use on a device, InvokeAI runs a small FP8 test on it; if the device,
driver or PyTorch build cannot store and upcast float8_e4m3fn (some older AMD GPUs, for example), the model loads
without FP8 and the log warns once: FP8 storage probe failed on <device> (…); not using FP8.
Not used. PyTorch can hold FP8 on the CPU, but casting is emulated in software and costs more than the memory saves. With the setting on, the loader returns the model unchanged, without a log line. Checkpoints that store FP8 weights are unpacked to BF16 here.
Turning it on
Section titled “Turning it on”- Open the Model Manager.
- Select a model (Main, ControlNet or T2I-Adapter). The setting only appears where it has an effect.
- Under Default Settings, turn on FP8 storage.
- Click Save Defaults.
It applies on the next load. If the model is already cached, InvokeAI evicts the cached copy so the new setting takes effect; if a generation is using the model, the eviction waits until it finishes.
Enable it on large models that don’t fit comfortably in VRAM: FLUX dev or Klein, large SDXL checkpoints, ControlNet-XL adapters. For small SD1 and SD2 models the savings are negligible and not worth the small precision trade-off.
When installing
Section titled “When installing”| What you install | FP8 Storage |
|---|---|
| A checkpoint whose transformer or UNet stores FP8 weights | On automatically. InvokeAI reads the weight types from the file, not its name, so a full-precision file with “fp8” in its name is not affected. Only the denoiser counts: an all-in-one checkpoint with an FP8 text encoder beside a full-precision UNet or transformer is left alone. The log says denoiser weights are stored in float8, enabling FP8 Storage by default. |
| A scaled fp8 checkpoint | On automatically, and it costs no precision: the weights stay exactly as published, with their scale. |
| An MXFP8 checkpoint | On automatically, but turn it off: it costs quality (see By file format). |
A Wan checkpoint in ComfyUI’s fp8_scaled format | Shows as on, but the Wan loader cannot keep these packed yet: it declines, logs FP8 Storage not applied to …, and loads the model at BF16. |
| Anything else, from a file path, URL or Hugging Face repo, or from folder scan results | Tick FP8 storage next to the install source. It applies to the models from there that support it; for the others the request is dropped, so no model is recorded with a setting that does nothing. Starter models don’t offer it. |
Either way the setting stays editable under Default Settings, and re-identifying a model keeps it.
Checkpoints that are already FP8
Section titled “Checkpoints that are already FP8”For these, FP8 Storage re-encodes nothing: it stops InvokeAI from unpacking them. That matters because the file stores
a scale beside each weight, and re-encoding the unpacked result would throw that scale away: on FLUX.2 Klein 4B roughly
3% of the weights would round to zero, for a file that was already one byte per weight. The cost is a little speed,
since each layer is unpacked on use. FLUX.3’s fp8r build from Black Forest Labs is a scaled fp8 file too, so the setting keeps it in fp8. Measured on an RTX 4090, the Ideogram 4 fp8 pair generates 1024px in 40.6s, against
35.6s with FP8 Compute, and holds 8.7GB per branch instead of the 17.3GB the
unpacked weights need: the difference between fitting on a 24GB card and running out of memory in the first step.
The small, precision-sensitive parts of the model are always unpacked. They are a tiny share of the weights, so you still get nearly the full saving.
What the log shows for these
The loader reports what it kept and which setting kept it, for example
Ideogram 4: kept 208 layer(s) in fp8 (scaled fp8 checkpoint, kept for FP8 Storage, dequantized per forward). On
FLUX.1, FLUX.2, FLUX.3, Z-Image and Anima the model then goes to the layer cast, which declines it with
FP8 storage skipped for <model>: … weight(s) are already fp8, …, so you see both lines for the same model. Both are
the intended outcome, not a failure. Qwen-Image keeps its fp8 layers without the first line.
Quality
Section titled “Quality”FP8 Storage is near-lossless for most workloads:
- Norms and embeddings, the precision-sensitive layers, are skipped.
- The math still runs in FP16 or BF16; FP8 is only how the weights are stored on the GPU.
- On a checkpoint that is already FP8, the weights are not re-encoded at all.
VAEs are never cast. Very precision-sensitive LoRAs may drift slightly: patching works as usual, but compare side by side if your workflow depends on subtle LoRA behavior. If you see a regression, turn FP8 Storage off for that model and run it again.
FP8 Storage and partial loading work together: use both when VRAM is tight. FP8 Compute with partial loading also works, but costs speed and reproducibility.
Troubleshooting
Section titled “Troubleshooting”| Symptom | What to do |
|---|---|
| VRAM didn’t drop after turning it on | A model in use is evicted only when its generation finishes: start a new generation. If it still hasn’t dropped, look in the log for FP8 layerwise casting enabled for <model>. Without it, the model is one FP8 Storage skips (What it applies to), a Wan fp8_scaled checkpoint (the log says why), or already FP8 (FP8 storage skipped … already fp8: nothing needed re-encoding). |
FP8 storage probe failed in the log | Your GPU, driver or PyTorch build can’t store FP8; the model loads without it. FP8 Storage is never used on the CPU or Apple silicon. |
| Quality regression on one model | Turn FP8 Storage off for that model and reload. If quality comes back, the model has FP8-sensitive layers outside the skip list: please report it (below). |
RuntimeError: … float8_e4m3fn … | Your PyTorch predates FP8 support (added in 2.1). Reinstall with the official launcher, whose PyTorch supports it. |
Reporting an issue
Section titled “Reporting an issue”If FP8 Storage misbehaves (a crash, a quality regression, an out-of-memory error that shouldn’t happen), open a GitHub issue with:
- What you did: the workflow or generation step, and whether it happens every time.
- Model: exact name and variant (such as “FLUX.2 Klein 9B Diffusers” or “SDXL Base 1.0 single-file”), and whether the file is full precision or quantized (GGUF, NF4, SDNQ, int8, nvfp4, scaled fp8, MXFP8).
- LoRAs: whether any LoRAs or ControlLoRAs are stacked on it, and how many.
- Other settings: low-VRAM mode on or off, any
cpu_onlytext encoder, a configured VRAM limit. - GPU: model and VRAM (such as “RTX 3090 24GB”).
- OS: Windows or Linux, and the driver or CUDA version if you know it.
- Logs: the InvokeAI log around the failure, especially the
FP8 layerwise casting enabled for <model>line (or its absence) and any traceback. - For a quality regression: a side-by-side image, FP8 Storage on and off, same seed.