π Model Description
pipeline_tag: image-text-to-text base_model:
- huihui-ai/Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated
These are quantizations of the model Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated
The imatrix from unsloth has been used for these.
Original model: https://huggingface.co/huihui-ai/Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated
Download the latest llama.cpp to use them.
Try to use the best quality you can run.
For the mmproj, try to use the F32 version as it will produce the best results.
F32 > BF16 > F16.
π GGUF File List
| π Filename | π¦ Size | β‘ Download |
|---|---|---|
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ2_M.gguf
LFS
Q2
|
9.47 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ2_S.gguf
LFS
Q2
|
8.65 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ2_XS.gguf
LFS
Q2
|
8.45 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ3_M.gguf
LFS
Q3
|
12.59 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ3_S.gguf
LFS
Q3
|
12.39 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ3_XS.gguf
LFS
Q3
|
11.73 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ3_XXS.gguf
LFS
Q3
|
11.04 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ4_NL.gguf
LFS
Q4
|
16.12 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-IQ4_XS.gguf
LFS
Q4
|
15.24 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-MXFP4_MOE.gguf
LFS
|
15.91 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q2_K.gguf
LFS
Q2
|
10.49 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q2_K_S.gguf
LFS
Q2
|
9.8 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q3_K_L.gguf
LFS
Q3
|
14.81 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q3_K_M.gguf
LFS
Q3
|
13.7 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q3_K_S.gguf
LFS
Q3
|
12.38 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q4_K_M.gguf
Recommended
LFS
Q4
|
17.28 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q4_K_S.gguf
LFS
Q4
|
16.26 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q5_K_M.gguf
LFS
Q5
|
20.23 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q5_K_S.gguf
LFS
Q5
|
19.63 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q6_K.gguf
LFS
Q6
|
23.37 GB | Download |
|
Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-Q8_0.gguf
LFS
Q8
|
30.25 GB | Download |
|
mmproj-BF16.gguf
LFS
FP16
|
1.01 GB | Download |
|
mmproj-F16.gguf
LFS
FP16
|
1.01 GB | Download |
|
mmproj-F32.gguf
LFS
|
2.01 GB | Download |
|
mmproj-Q8_0.gguf
LFS
Q8
|
679.16 MB | Download |