πŸ“‹ Model Description

license: apache-2.0 datasets:
  • HuggingFaceH4/ultrachat_200k
language:
  • en
base_model: Locutusque/TinyMistral-248M-v2-Instruct

Description

GGUF version of Locutusque/TinyMistral-248M-v2-Instruct.

Recommended inference parameters

do_sample: true
temperature: 0.1
top_p: 0.14
top_k: 12
repetition_penalty: 1.1

Recommended prompt template

<|imstart|>user\n{user message}<|imend|>\n<|im_start|>assistant\n{assistant message}<|endoftext|>

πŸ“‚ GGUF File List

πŸ“ Filename πŸ“¦ Size ⚑ Download
TinyMistral-248M-v2-Instruct.Q2_K.gguf
LFS Q2
100.57 MB Download
TinyMistral-248M-v2-Instruct.Q3_K_M.gguf
LFS Q3
123.05 MB Download
TinyMistral-248M-v2-Instruct.Q4_K_M.gguf
Recommended LFS Q4
148.46 MB Download
TinyMistral-248M-v2-Instruct.Q5_K_M.gguf
LFS Q5
170.93 MB Download
TinyMistral-248M-v2-Instruct.Q6_K.gguf
LFS Q6
194.8 MB Download
TinyMistral-248M-v2-Instruct.Q8_0.gguf
LFS Q8
252.08 MB Download