Model Highlights

• PRISM Ablation — State-of-the-art technique that removes over-refusal behaviors while preserving model capabilities
• 30B-A3B MoE Architecture — 30 billion total parameters with ~3 billion active per token for fast, efficient inference
• 128K Context Window — Extended context for complex tasks and large codebases
• Interleaved Thinking — Multi-turn reasoning that persists across conversations with per-turn thinking control

Benchmarks

Usage

Transformers

Install the latest transformers from source:

pip install git+https://github.com/huggingface/transformers.git

Run inference:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "Ex0bit/GLM-4.7-Flash-PRISM"

tokenizer = AutoTokenizer.frompretrained(MODELPATH)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Hello!"}]
inputs = tokenizer.applychattemplate(
    messages,
    tokenize=True,
    addgenerationprompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

generatedids = model.generate(inputs, maxnewtokens=128, dosample=False)
outputtext = tokenizer.decode(generatedids[0][inputs.input_ids.shape[1]:])
print(output_text)

vLLM

Install vLLM nightly:

pip install -U vllm --pre --index-url https://pypi.org/simple --extra-index-url https://wheels.vllm.ai/nightly
pip install git+https://github.com/huggingface/transformers.git

Serve the model:

vllm serve Ex0bit/GLM-4.7-Flash-PRISM \
     --tensor-parallel-size 4 \
     --speculative-config.method mtp \
     --speculative-config.numspeculativetokens 1 \
     --tool-call-parser glm47 \
     --reasoning-parser glm45 \
     --enable-auto-tool-choice \
     --served-model-name glm-4.7-flash-prism

SGLang

Install SGLang:

uv pip install sglang==0.3.2.dev9039+pr-17247.g90c446848 --extra-index-url https://sgl-project.github.io/whl/pr/
uv pip install git+https://github.com/huggingface/transformers.git@76732b4e7120808ff989edbd16401f61fa6a0afa

Launch the server:

python3 -m sglang.launch_server \
  --model-path Ex0bit/GLM-4.7-Flash-PRISM \
  --tp-size 4 \
  --tool-call-parser glm47  \
  --reasoning-parser glm45 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --mem-fraction-static 0.8 \
  --served-model-name glm-4.7-flash-prism \
  --host 0.0.0.0 \
  --port 8000

Note: For Blackwell GPUs, add --attention-backend triton --speculative-draft-attention-backend triton to your SGLang launch command.

Recommended Parameters

License