---

license: other license_name: circlestone-labs-non-commercial-license license_link: https://huggingface.co/circlestone-labs/Anima/blob/main/LICENSE.md base_model:

• circlestone-labs/Anima

basemodelrelation: quantized

---

GGUF models of ANIMA

How to use

• Update ComfyUI to v0.14.1 or above. (You don't need custom script.)
• Download GGUF you want. (Recommend at least Q5KM)
• Use ComfyUI-GGUF custom node to load the model.

Generation speed

Tested on

• RTX5090(400W), ComfyUI with --fast option and Patch Sage Attention KJ node(AUTO).
• 832x1216, cfg 5.0, 50steps

Quant	it/s	Time (s)	Speed vs BF16 (%)
BF16	4.65	11.70	0.00%
Q8_0	4.46	12.07	-4.09%
Q6_K	3.60	14.91	-22.58%
Q5KS	3.35	15.94	-28.03%
Q5KM	3.41	15.67	-26.67%
Q5_1	3.42	15.24	-26.45%
Q5_0	3.40	15.73	-26.88%
Q4KS	3.55	15.12	-23.66%
Q4KM	3.59	14.98	-22.80%
Q4_1	4.01	13.46	-13.76%
Q4_0	3.97	13.50	-14.62%

Sample

!AnimaGGUF_Comparison

How to reproduce

1. Convert BF16 model to FP32

import torch
import safetensors.torch
import os
import sys

def converttofp32(inputpath, outputpath):
    statedict = safetensors.torch.loadfile(input_path)

newstatedict = {}
    for key, tensor in state_dict.items():
        print(f"{key} ({tensor.dtype}) -> torch.float32")
        new_tensor = tensor.to(torch.float32)
        newstatedict[key] = new_tensor

safetensors.torch.savefile(newstatedict, outputpath)
    print(f"outputpath: {outputpath}")

if name == "main":
    assert len(sys.argv) == 3, f"usage: {sys.argv[0]} SOURCE TARGET"
    inputpath, outputpath = sys.argv[1:3]

converttofp32(inputpath, outputpath)

1. Read this manual.
2. make F32 GGUF using https://github.com/city96/ComfyUI-GGUF/blob/main/tools/convert.py#L258
3. Run llama-quantize.