πŸ“‹ Model Description

datasets:
  • IlyaGusev/saiga_scored
language:
  • ru
inference: false pipeline_tag: text-generation license: other license_name: llama3 license_link: https://llama.meta.com/llama3/license/

Llama.cpp compatible versions of an original 8B model.

Download one of the versions, for example model-q4_K.gguf.

wget https://huggingface.co/IlyaGusev/saigallama38bgguf/resolve/main/model-q4K.gguf

Download interactllama3llamacpp.py

wget https://raw.githubusercontent.com/IlyaGusev/rulm/master/selfinstruct/src/interactllama3_llamacpp.py

How to run:

pip install llama-cpp-python fire



python3 interactllama3llamacpp.py model-q4_K.gguf

System requirements:

  • 10GB RAM for q8_0 and less for smaller quantizations

πŸ“‚ GGUF File List

πŸ“ Filename πŸ“¦ Size ⚑ Download
model-f16.gguf
Recommended LFS FP16
14.97 GB Download
model-q2_K.gguf
LFS Q2
2.96 GB Download
model-q4_K.gguf
LFS Q4
4.58 GB Download
model-q8_0.gguf
LFS Q8
7.95 GB Download