Quantization made by Richard Erkhov.

Github

Discord

Request more models

llama-3-8b-samantha - GGUF

• Model creator: https://huggingface.co/ruslandev/
• Original model: https://huggingface.co/ruslandev/llama-3-8b-samantha/

Name	Quant method	Size
llama-3-8b-samantha.Q2K.gguf	Q2K	2.96GB
llama-3-8b-samantha.IQ3XS.gguf	IQ3XS	3.28GB
llama-3-8b-samantha.IQ3S.gguf	IQ3S	3.43GB
llama-3-8b-samantha.Q3KS.gguf	Q3K_S	3.41GB
llama-3-8b-samantha.IQ3M.gguf	IQ3M	3.52GB
llama-3-8b-samantha.Q3K.gguf	Q3K	3.74GB
llama-3-8b-samantha.Q3KM.gguf	Q3K_M	3.74GB
llama-3-8b-samantha.Q3KL.gguf	Q3K_L	4.03GB
llama-3-8b-samantha.IQ4XS.gguf	IQ4XS	4.18GB
llama-3-8b-samantha.Q40.gguf	Q40	4.34GB
llama-3-8b-samantha.IQ4NL.gguf	IQ4NL	4.38GB
llama-3-8b-samantha.Q4KS.gguf	Q4K_S	4.37GB
llama-3-8b-samantha.Q4K.gguf	Q4K	4.58GB
llama-3-8b-samantha.Q4KM.gguf	Q4K_M	4.58GB
llama-3-8b-samantha.Q41.gguf	Q41	4.78GB
llama-3-8b-samantha.Q50.gguf	Q50	5.21GB
llama-3-8b-samantha.Q5KS.gguf	Q5K_S	5.21GB
llama-3-8b-samantha.Q5K.gguf	Q5K	5.34GB
llama-3-8b-samantha.Q5KM.gguf	Q5K_M	5.34GB
llama-3-8b-samantha.Q51.gguf	Q51	5.65GB
llama-3-8b-samantha.Q6K.gguf	Q6K	6.14GB
llama-3-8b-samantha.Q80.gguf	Q80	7.95GB

Original model description:

---

language:

• en

license: apache-2.0
tags:

• text-generation-inference
• transformers
• unsloth
• llama
• trl

base_model: unsloth/llama-3-8b-bnb-4bit
datasets:

• cognitivecomputations/samantha-data

---

Uploaded model

• Developed by: ruslandev
• License: apache-2.0
• Finetuned from model : unsloth/llama-3-8b-bnb-4bit

This model is finetuned on the data of Samantha.
Prompt format is Alpaca. I used the same system prompt as the original Samantha.

"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.


Instruction:
{SYSTEM_PROMPT}

Input:
{QUESTION}

Response:
"""

Training

gptchain framework has been used for training.

Training hyperparameters

• learningrate: 2e-4
• seed: 3407
• gradientaccumulationsteps: 4
• perdevicetrainbatchsize: 2
• optimizer: adamw8bit
• lrschedulertype: linear
• warmupsteps: 5
• numepochs: 2
• weight_decay: 0.01

Training results

Training Loss	Epoch	Step
2.0778	0.0	1
0.6255	0.18	120
0.6208	0.94	620
0.6244	2.0	1306

2 epoch finetuning from llama-3-8b took 1 hour on a single A100 with Unsloth and Huggingface's TRL library.