πŸ“‹ Model Description

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Qwen2.5-1.5B-bnb-4bit_math - GGUF

  • Model creator: https://huggingface.co/raiyan007/
  • Original model: https://huggingface.co/raiyan007/Qwen2.5-1.5B-bnb-4bit_math/

NameQuant methodSize
Qwen2.5-1.5B-bnb-4bitmath.Q2K.ggufQ2_K0.63GB
Qwen2.5-1.5B-bnb-4bitmath.IQ3XS.ggufIQ3_XS0.68GB
Qwen2.5-1.5B-bnb-4bitmath.IQ3S.ggufIQ3_S0.71GB
Qwen2.5-1.5B-bnb-4bitmath.Q3KS.ggufQ3KS0.71GB
Qwen2.5-1.5B-bnb-4bitmath.IQ3M.ggufIQ3_M0.72GB
Qwen2.5-1.5B-bnb-4bitmath.Q3K.ggufQ3_K0.77GB
Qwen2.5-1.5B-bnb-4bitmath.Q3KM.ggufQ3KM0.77GB
Qwen2.5-1.5B-bnb-4bitmath.Q3KL.ggufQ3KL0.82GB
Qwen2.5-1.5B-bnb-4bitmath.IQ4XS.ggufIQ4_XS0.84GB
Qwen2.5-1.5B-bnb-4bitmath.Q40.ggufQ4_00.87GB
Qwen2.5-1.5B-bnb-4bitmath.IQ4NL.ggufIQ4_NL0.88GB
Qwen2.5-1.5B-bnb-4bitmath.Q4KS.ggufQ4KS0.88GB
Qwen2.5-1.5B-bnb-4bitmath.Q4K.ggufQ4_K0.92GB
Qwen2.5-1.5B-bnb-4bitmath.Q4KM.ggufQ4KM0.92GB
Qwen2.5-1.5B-bnb-4bitmath.Q41.ggufQ4_10.95GB
Qwen2.5-1.5B-bnb-4bitmath.Q50.ggufQ5_01.02GB
Qwen2.5-1.5B-bnb-4bitmath.Q5KS.ggufQ5KS1.02GB
Qwen2.5-1.5B-bnb-4bitmath.Q5K.ggufQ5_K1.05GB
Qwen2.5-1.5B-bnb-4bitmath.Q5KM.ggufQ5KM1.05GB
Qwen2.5-1.5B-bnb-4bitmath.Q51.ggufQ5_11.1GB
Qwen2.5-1.5B-bnb-4bitmath.Q6K.ggufQ6_K1.19GB
Qwen2.5-1.5B-bnb-4bitmath.Q80.ggufQ8_01.53GB

Original model description:


base_model: unsloth/qwen2.5-1.5b-bnb-4bit
tags:
  • text-generation-inference
  • transformers
  • unsloth
  • qwen2
  • trl

license: apache-2.0
language:
  • en

Uploaded model

  • Developed by: raiyan007
  • License: apache-2.0
  • Finetuned from model : unsloth/qwen2.5-1.5b-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

πŸ“‚ GGUF File List

πŸ“ Filename πŸ“¦ Size ⚑ Download
Qwen2.5-1.5B-bnb-4bit_math.IQ3_M.gguf
LFS Q3
740.68 MB Download
Qwen2.5-1.5B-bnb-4bit_math.IQ3_S.gguf
LFS Q3
727.09 MB Download
Qwen2.5-1.5B-bnb-4bit_math.IQ3_XS.gguf
LFS Q3
697.8 MB Download
Qwen2.5-1.5B-bnb-4bit_math.IQ4_NL.gguf
LFS Q4
897.87 MB Download
Qwen2.5-1.5B-bnb-4bit_math.IQ4_XS.gguf
LFS Q4
860.39 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q2_K.gguf
LFS Q2
644.97 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q3_K.gguf
LFS Q3
786 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q3_K_L.gguf
LFS Q3
839.39 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q3_K_M.gguf
LFS Q3
786 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q3_K_S.gguf
LFS Q3
725.69 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q4_0.gguf
Recommended LFS Q4
891.64 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q4_1.gguf
LFS Q4
969.73 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q4_K.gguf
LFS Q4
940.37 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q4_K_M.gguf
LFS Q4
940.37 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q4_K_S.gguf
LFS Q4
896.75 MB Download
Qwen2.5-1.5B-bnb-4bit_math.Q5_0.gguf
LFS Q5
1.02 GB Download
Qwen2.5-1.5B-bnb-4bit_math.Q5_1.gguf
LFS Q5
1.1 GB Download
Qwen2.5-1.5B-bnb-4bit_math.Q5_K.gguf
LFS Q5
1.05 GB Download
Qwen2.5-1.5B-bnb-4bit_math.Q5_K_M.gguf
LFS Q5
1.05 GB Download
Qwen2.5-1.5B-bnb-4bit_math.Q5_K_S.gguf
LFS Q5
1.02 GB Download
Qwen2.5-1.5B-bnb-4bit_math.Q6_K.gguf
LFS Q6
1.19 GB Download
Qwen2.5-1.5B-bnb-4bit_math.Q8_0.gguf
LFS Q8
1.53 GB Download