πŸ“‹ Model Description

datasets:
  • IlyaGusev/saigascored
  • IlyaGusev/saigapreferences
language:
  • ru
inference: false license: other license_name: yandexgpt-5-lite-8b-pretrain license_link: LICENSE

Llama.cpp compatible versions of an original 8B model.

Download one of the versions, for example saigayandexgpt8b.Q4KM.gguf.

wget https://huggingface.co/IlyaGusev/saigayandexgpt8bgguf/resolve/main/saigayandexgpt8b.Q4K_M.gguf

Download interact_gguf.py

https://raw.githubusercontent.com/IlyaGusev/saiga/refs/heads/main/scripts/interact_gguf.py

How to run:

pip install llama-cpp-python fire



python3 interactgguf.py saigayandexgpt8b.Q4K_M.gguf

System requirements:

  • 9GB RAM for q8_0 and less for smaller quantizations

πŸ“‚ GGUF File List

πŸ“ Filename πŸ“¦ Size ⚑ Download
saiga_yandexgpt_8b.BF16.gguf
LFS FP16
14.98 GB Download
saiga_yandexgpt_8b.Q2_K.gguf
LFS Q2
2.97 GB Download
saiga_yandexgpt_8b.Q3_K_M.gguf
LFS Q3
3.75 GB Download
saiga_yandexgpt_8b.Q3_K_S.gguf
LFS Q3
3.42 GB Download
saiga_yandexgpt_8b.Q4_0.gguf
Recommended LFS Q4
4.35 GB Download
saiga_yandexgpt_8b.Q4_K_M.gguf
LFS Q4
4.59 GB Download
saiga_yandexgpt_8b.Q4_K_S.gguf
LFS Q4
4.38 GB Download
saiga_yandexgpt_8b.Q5_K_M.gguf
LFS Q5
5.34 GB Download
saiga_yandexgpt_8b.Q5_K_S.gguf
LFS Q5
5.22 GB Download
saiga_yandexgpt_8b.Q6_K.gguf
LFS Q6
6.15 GB Download
saiga_yandexgpt_8b.Q8_0.gguf
LFS Q8
7.96 GB Download