lovedheart/Qwen3-Coder-Next-REAP-40B-A3B-GGUF

• Qwen/Qwen3-Coder-Next

• text-generation-inference

!Qwen3-coder-next-reap

Qwen3-Coder-Next-REAP-40B-A3B has the following specifications:

• Type: Causal Language Models
• Number of Parameters: 40B in total and 3B activated
• Hidden Dimension: 2048
• Number of Layers: 48
• Hybrid Layout: 12 (3 (Gated DeltaNet -> MoE) -> 1 * (Gated Attention -> MoE))
• Gated Attention:
• Number of Attention Heads: 16 for Q and 2 for KV
• Head Dimension: 256
• Rotary Position Embedding Dimension: 64
• Gated DeltaNet:

• Mixture of Experts:
• Number of Experts: 256 (uniformly pruned from 512)
• Number of Activated Experts: 10
• Number of Shared Experts: 1
• Context Length: 262,144 natively
• Compression Method: REAP (Router-weighted Expert Activation Pruning)
• Compression Ratio: 50% expert pruning

Test video (writing tiny video game @ Q4KXL ): https://bilibili.com/video/BV1k6fZB7EQz/