# FP8 Storage import { Steps, Tabs, TabItem } from '@astrojs/starlight/components'; FP8 Storage cuts a model's VRAM roughly in half by keeping its weights on the GPU in 8-bit floating point (`float8_e4m3fn`). Before each layer runs, its weights are cast up to the compute precision (FP16 or BF16), and the stored FP8 weights are put back afterwards, so quality is largely preserved. Measured on Wan 2.2 TI2V-5B (fp16 single file, 512², 20 steps, RTX 4090), the transformer drops from 9.5GB to 4.9GB in VRAM, at 21.4s per video instead of 18.4s. It is a per-model setting for full-precision models, and it pairs with [Low-VRAM mode](/configuration/optimization/low-vram-mode/): low-VRAM mode streams layers between RAM and VRAM, FP8 Storage shrinks the layers themselves. How it compares with quantized downloads is in the [format comparison](/configuration/optimization/memory-options/#format-comparison). To also run the math in FP8 on the GPU's FP8 hardware, see [FP8 Compute](/configuration/optimization/fp8-compute/). :::caution[Turn it off for MXFP8 checkpoints] Installing switches FP8 Storage on for every checkpoint that stores FP8 weights, MXFP8 included. For MXFP8 it costs quality: see [By file format](#by-file-format). Turn it off under **Default Settings** after installing one. ::: ## What it applies to The Model Manager shows the **FP8 storage** setting only for models where it changes something. A model installed before this rule may still have it stored as on; it then does nothing, and the log says so once when the model loads: `FP8 Storage is set for '' but does nothing here: `. ### By file format | Model file | FP8 Storage | | --- | --- | | Full precision (FP16, BF16, FP32) | **Yes**: what it is for. | | fp8 or scaled fp8 checkpoint | **On after install.** The weights stay in fp8 with their own scale; without the setting they would be unpacked to BF16 at twice the size. See [Checkpoints that are already FP8](#checkpoints-that-are-already-fp8). | | MXFP8 checkpoint | **Turn it off.** It stores one scale per block of 32 weights, which InvokeAI does not keep on any GPU: the layers are unpacked at load, and FP8 Storage then re-encodes them as fp8 *without* a scale: blocks whose scale is far below 1 lose their weights to underflow. If the file also has per-tensor fp8 layers, those stay fp8 and FP8 Storage changes nothing. | | int8 and nvfp4 single files | **Partly.** Their quantized layers stay as they are; only the layers the build ships in full precision are stored in FP8. On an Ideogram 4, Qwen-Image-2.1 or FLUX.3 int8 file, and on a FLUX.3 nvfp4 file, nothing at all. | | GGUF, NF4 and [SDNQ](/configuration/optimization/sdnq-quantization/) | **No**: already quantized. The setting is not shown. | If your model is quantized, its full-precision variant gets the full effect of FP8 Storage. ### By model | Model | FP8 Storage | | --- | --- | | Main models: SD1, SD2, SDXL, SDXL Refiner, SD3, CogView4 | Yes | | FLUX.1, FLUX.2 (all variants) | Yes | | FLUX.3 | Yes — except int8 and nvfp4 single files, where it does nothing. The toggle can still show as on for an nvfp4 build, since installing switches it on for its fp8 block scales | | Z-Image, Anima, Krea-2, Qwen-Image, Qwen-Image-2.1, ERNIE-Image, MOSS-Image | Yes | | Ideogram 4, single-file safetensors | Yes, for each transformer branch's file | | LTX-2 | Yes | | Wan (checkpoint and Diffusers) | Yes, except ComfyUI `fp8_scaled` files, which load at BF16 | | ControlNet (SD1, SD2, SDXL), T2I-Adapter | Yes | | VAE | No: visible decode-quality regression | | Text encoders, tokenizers | No: small models, and FP8 rounding costs text quality. The FLUX.2 Mistral encoder keeps its scaled fp8 weights packed on any GPU that passes the FP8 test, with no setting. | | Anima ControlNet-LLLite | No: the adapter is only tens of MB | | FLUX.1, FLUX.2, Z-Image and Qwen-Image ControlNets | No: not implemented yet | | MiniMax H3 | No: not implemented yet | | Ideogram 4, Diffusers folder | No: published only as nf4 or fp8 | | LoRA, ControlLoRA | No: patched into the base model, not run alone |
Which layers inside a model stay at full precision **Norm layers, position and patch embeddings, and `proj_in`/`proj_out` are skipped**, so precision-sensitive tiny learned scalars (such as FLUX's `RMSNorm.scale`) aren't crushed to FP8. This mirrors the diffusers default skip list. **Each model's own declared exclusions are honored too.** Architectures list their precision-sensitive modules themselves (diffusers calls these `_skip_layerwise_casting_patterns` and `_keep_in_fp32_modules`), and those layers stay at compute precision. This matters where the generic patterns don't fit the architecture's naming: Z-Image and Anima both name their timestep-embedding MLP `t_embedder`, which no generic pattern matches. Cast to FP8, Z-Image fails outright and Anima renders a heavily dithered image with no fine detail. The cost is a slightly smaller saving on the models that declare a lot. Measured against the generic defaults alone: | Model | Weights kept at compute precision | Saving given up | | --- | --- | --- | | Krea-2 | 39 M (`time_embed`) | ~38 MiB | | Anima | 19 M (`t_embedder`, `x_embedder`, `final_layer`) | ~18 MiB | | Wan 14B | 232 M (`condition_embedder`, `patch_embedding`) | ~221 MiB | | MOSS-Image | 486 M (every block's `proj.q`, the embedders, modulations and output head) | ~464 MiB | | FLUX.1, Qwen-Image | 0 | 0 | MOSS-Image keeps each block's query projection at compute precision because its forward pass casts the block's input to that layer's dtype; the rest of the block is stored in FP8. Measured on an RTX 4090, its transformer takes 5.3GB with FP8 Storage instead of 10.2GB. InvokeAI applies the cast with its own forward hooks rather than diffusers' `apply_layerwise_casting`, on every `nn.Module` including diffusers `ModelMixin` subclasses, so it works with InvokeAI's `CustomLinear` layers and with partial loading. The FP8 cast happens on the CPU, before the weights move to VRAM.
## By GPU FP8 Storage needs no FP8 hardware: the math still runs in FP16 or BF16. What your GPU adds is speed, through the separate [FP8 Compute](/configuration/optimization/fp8-compute/). VRAM win only. The transformer or UNet drops by roughly 50% in VRAM. Each step is the same speed or marginally slower, because every layer casts its FP8 weights up on entry; putting the stored weights back on exit costs nothing. This covers the RTX 30 series and Ampere workstation cards, and older CUDA cards such as the RTX 20 and GTX 16 series: `float8_e4m3fn` is a pure storage type that any CUDA device can hold. RTX 3090 owners fitting FLUX into 24GB benefit the most. The same ~50% VRAM win. These GPUs have FP8 tensor cores, but FP8 Storage alone does not use them: the weights are cast up to BF16 before each layer. To use them for checkpoints published in FP8, turn on [FP8 Compute](/configuration/optimization/fp8-compute/). A ROCm build of PyTorch reports an AMD GPU as a CUDA device, so FP8 Storage works on AMD as it does on NVIDIA. Intel GPUs (XPU) are supported too. Before the first use on a device, InvokeAI runs a small FP8 test on it; if the device, driver or PyTorch build cannot store and upcast `float8_e4m3fn` (some older AMD GPUs, for example), the model loads without FP8 and the log warns once: `FP8 storage probe failed on (…); not using FP8.` Not used. PyTorch can hold FP8 on the CPU, but casting is emulated in software and costs more than the memory saves. With the setting on, the loader returns the model unchanged, without a log line. Checkpoints that store FP8 weights are unpacked to BF16 here. ## Turning it on 1. Open the **Model Manager**. 2. Select a model (Main, ControlNet or T2I-Adapter). The setting only appears where it has an effect. 3. Under **Default Settings**, turn on **FP8 storage**. 4. Click **Save Defaults**. It applies on the next load. If the model is already cached, InvokeAI evicts the cached copy so the new setting takes effect; if a generation is using the model, the eviction waits until it finishes. Enable it on large models that don't fit comfortably in VRAM: FLUX dev or Klein, large SDXL checkpoints, ControlNet-XL adapters. For small SD1 and SD2 models the savings are negligible and not worth the small precision trade-off. ### When installing | What you install | FP8 Storage | | --- | --- | | A checkpoint whose transformer or UNet stores FP8 weights | **On automatically.** InvokeAI reads the weight types from the file, not its name, so a full-precision file with "fp8" in its name is not affected. Only the denoiser counts: an all-in-one checkpoint with an FP8 text encoder beside a full-precision UNet or transformer is left alone. The log says `denoiser weights are stored in float8, enabling FP8 Storage by default`. | | A scaled fp8 checkpoint | **On automatically**, and it costs no precision: the weights stay exactly as published, with their scale. | | An MXFP8 checkpoint | **On automatically, but turn it off**: it costs quality (see [By file format](#by-file-format)). | | A Wan checkpoint in ComfyUI's `fp8_scaled` format | Shows as on, but the Wan loader cannot keep these packed yet: it declines, logs `FP8 Storage not applied to …`, and loads the model at BF16. | | Anything else, from a file path, URL or Hugging Face repo, or from folder scan results | **Tick FP8 storage** next to the install source. It applies to the models from there that support it; for the others the request is dropped, so no model is recorded with a setting that does nothing. Starter models don't offer it. | Either way the setting stays editable under **Default Settings**, and re-identifying a model keeps it. ## Checkpoints that are already FP8 For these, FP8 Storage re-encodes nothing: it stops InvokeAI from unpacking them. That matters because the file stores a scale beside each weight, and re-encoding the unpacked result would throw that scale away: on FLUX.2 Klein 4B roughly 3% of the weights would round to zero, for a file that was already one byte per weight. The cost is a little speed, since each layer is unpacked on use. FLUX.3's `fp8r` build from Black Forest Labs is a scaled fp8 file too, so the setting keeps it in fp8. Measured on an RTX 4090, the Ideogram 4 fp8 pair generates 1024px in 40.6s, against 35.6s with [FP8 Compute](/configuration/optimization/fp8-compute/), and holds 8.7GB per branch instead of the 17.3GB the unpacked weights need: the difference between fitting on a 24GB card and running out of memory in the first step. The small, precision-sensitive parts of the model are always unpacked. They are a tiny share of the weights, so you still get nearly the full saving.
What the log shows for these The loader reports what it kept and which setting kept it, for example `Ideogram 4: kept 208 layer(s) in fp8 (scaled fp8 checkpoint, kept for FP8 Storage, dequantized per forward)`. On FLUX.1, FLUX.2, FLUX.3, Z-Image and Anima the model then goes to the layer cast, which declines it with `FP8 storage skipped for : … weight(s) are already fp8, …`, so you see both lines for the same model. Both are the intended outcome, not a failure. Qwen-Image keeps its fp8 layers without the first line.
## Quality FP8 Storage is near-lossless for most workloads: - Norms and embeddings, the precision-sensitive layers, are skipped. - The math still runs in FP16 or BF16; FP8 is only how the weights are stored on the GPU. - On a checkpoint that is already FP8, the weights are not re-encoded at all. VAEs are never cast. Very precision-sensitive LoRAs may drift slightly: patching works as usual, but compare side by side if your workflow depends on subtle LoRA behavior. If you see a regression, turn FP8 Storage off for that model and run it again. FP8 Storage and partial loading work together: use both when VRAM is tight. [FP8 Compute](/configuration/optimization/fp8-compute/) with partial loading also works, but costs speed and reproducibility. ## Troubleshooting | Symptom | What to do | | --- | --- | | VRAM didn't drop after turning it on | A model in use is evicted only when its generation finishes: start a new generation. If it still hasn't dropped, look in the log for `FP8 layerwise casting enabled for `. Without it, the model is one FP8 Storage skips ([What it applies to](#what-it-applies-to)), a Wan `fp8_scaled` checkpoint (the log says why), or already FP8 (`FP8 storage skipped … already fp8`: nothing needed re-encoding). | | `FP8 storage probe failed` in the log | Your GPU, driver or PyTorch build can't store FP8; the model loads without it. FP8 Storage is never used on the CPU or Apple silicon. | | Quality regression on one model | Turn FP8 Storage off for that model and reload. If quality comes back, the model has FP8-sensitive layers outside the skip list: please report it (below). | | `RuntimeError: … float8_e4m3fn …` | Your PyTorch predates FP8 support (added in 2.1). Reinstall with the official launcher, whose PyTorch supports it. | ### Reporting an issue If FP8 Storage misbehaves (a crash, a quality regression, an out-of-memory error that shouldn't happen), [open a GitHub issue](https://github.com/invoke-ai/InvokeAI/issues/new/choose) with: - **What you did**: the workflow or generation step, and whether it happens every time. - **Model**: exact name and variant (such as "FLUX.2 Klein 9B Diffusers" or "SDXL Base 1.0 single-file"), and whether the file is full precision or quantized (GGUF, NF4, SDNQ, int8, nvfp4, scaled fp8, MXFP8). - **LoRAs**: whether any LoRAs or ControlLoRAs are stacked on it, and how many. - **Other settings**: low-VRAM mode on or off, any `cpu_only` text encoder, a configured VRAM limit. - **GPU**: model and VRAM (such as "RTX 3090 24GB"). - **OS**: Windows or Linux, and the driver or CUDA version if you know it. - **Logs**: the InvokeAI log around the failure, especially the `FP8 layerwise casting enabled for ` line (or its absence) and any traceback. - For a quality regression: a side-by-side image, FP8 Storage on and off, same seed.