π Model Description
license: apache-2.0 language:
- en
- DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
- GLM 4.7 Flash
- conversational
- thinking
- reasoning
- uncensored
- swearing
- nsfw
- not-for-all-audiences
- mistral
- dolphin
- venice
- dphn
- gguf
- based
- quantized
About
Original model: DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored
Fully uncensored thinking/reasoning model quantized using llama.cpp. Use at your own risk.
This model was finetuned using GLM 4.7 Flash to convert it from standard instruct to a thinking/reasoning model. Thinking is enabled by default and requires no special system prompt.
32k context.
Recommended settings
- Temp: 0.15
- Top K: 40
- Repeat Pen: 1.1
- Top P: 0.95
- Min P: 0.05
Optional optimizations:
- Enable Flash Attention for faster inference
- KV Cache Quantization at Q8_0 for memory savings with minimal quality impact
Download
Sorted by recommended. Bigger size does NOT mean higher quality.
| Link/Type | Size (GB) | Notes |
|---|---|---|
| bf16 | 47.2 | Perfect, full-precision but overkill |
| Q8_0 | 25.1 | Near-perfect, max quality |
| Q6_K | 19.3 | Excellent, most recommended |
| Q5K_M | 16.8 | Very high quality, best 5-bit |
| Q5K_S | 16.3 | Very high quality |
| Q5_1 | 17.7 | Legacy |
| Q5_0 | 16.3 | Legacy |
| Q4K_M | 14.3 | High quality, most popular/efficient, best 4-bit |
| Q4K_S | 13.5 | High quality, most most popular/efficient |
| Q4_1 | 14.9 | Legacy |
| Q4_0 | 13.4 | Legacy |
| Q3K_L | 12.4 | Mid quality, best 3-bit |
| Q3K_M | 11.5 | Mid quality |
| Q3K_S | 10.4 | Mid quality |
| Q2_K | 8.9 | Low quality |
Credits
Thanks to:
π GGUF File List
| π Filename | π¦ Size | β‘ Download |
|---|---|---|
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q2_K.gguf
LFS
Q2
|
8.28 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q3_K_L.gguf
LFS
Q3
|
11.55 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q3_K_M.gguf
LFS
Q3
|
10.69 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q3_K_S.gguf
LFS
Q3
|
9.69 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q4_0.gguf
Recommended
LFS
Q4
|
12.52 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q4_1.gguf
LFS
Q4
|
13.85 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q4_K.gguf
LFS
Q4
|
13.35 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q4_K_M.gguf
LFS
Q4
|
13.35 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q4_K_S.gguf
LFS
Q4
|
12.62 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q5_0.gguf
LFS
Q5
|
15.18 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q5_1.gguf
LFS
Q5
|
16.52 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q5_K.gguf
LFS
Q5
|
15.61 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q5_K_M.gguf
LFS
Q5
|
15.61 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q5_K_S.gguf
LFS
Q5
|
15.18 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q6_K.gguf
LFS
Q6
|
18.02 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-Q8_0.gguf
LFS
Q8
|
23.33 GB | Download |
|
Dolphin-Mistral-24B-Venice-Edition-Thinking-bf16.gguf
LFS
FP16
|
43.92 GB | Download |