---

base_model: glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think library_name: gguf license: apache-2.0 language:

• en
• fr

tags:

• granite
• gguf
• quantized
• llama.cpp
• ollama

quantized_by: llama.cpp pipeline_tag: text-generation

---

granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF

GGUF quantized versions of granite-4.0-h-tiny-DISTILL-OPUS-4.5-think

Available Formats

Filename	Size	Quant Type	Description
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-f16.gguf	13.22 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-F16
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q2k.gguf	2.46 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q2K
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q3kl.gguf	3.42 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q3K_L
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q3km.gguf	3.18 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q3K_M
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q3ks.gguf	2.91 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q3K_S
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q40.gguf	3.77 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q40
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q41.gguf	4.18 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q41
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q4km.gguf	4.02 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q4K_M
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q4ks.gguf	3.80 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q4K_S
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q50.gguf	4.58 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q50
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q51.gguf	4.98 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q51
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q5km.gguf	4.71 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q5K_M
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q5ks.gguf	4.58 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q5K_S
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q6k.gguf	5.44 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q6K
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q80.gguf	7.04 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q80

Quick Start

Ollama

# Use Q4KM (recommended)
ollama run hf.co/glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF:Q4KM

Or other quantizations
ollama run hf.co/glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF:Q8_0
ollama run hf.co/glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF:Q2_K

llama.cpp

# Download and run
llama-cli --hf-repo glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF --hf-file granite-4.0-h-tiny-distill-opus-4.5-think-q4km.gguf -p "Hello, how are you?"

With server
llama-server --hf-repo glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF --hf-file granite-4.0-h-tiny-distill-opus-4.5-think-q4km.gguf -c 2048

LM Studio / GPT4All

Download the .gguf file of your choice and load it in your application.

Quantization Details

Type	Bits	Use Case
Q2_K	2	Extreme compression, low quality
Q3KM	3	Very compressed
Q4KM	4	Recommended - Best size/quality
Q5KM	5	High quality
Q6_K	6	Very high quality
Q8_0	8	Near lossless
F16	16	Original precision

Original Model

This is the quantized version of granite-4.0-h-tiny-DISTILL-OPUS-4.5-think

• Base Model: ibm-granite/granite-4.0-h-tiny
• Fine-tuning Dataset: TeichAI/claude-4.5-opus-high-reasoning-250x
• Special Feature: Thinking/Reasoning with tags