Quantization made by Richard Erkhov.

Github

Discord

Request more models

gpt2o-chatbot-10 - GGUF

• Model creator: https://huggingface.co/styalai/
• Original model: https://huggingface.co/styalai/gpt2o-chatbot-10/

Name	Quant method	Size
gpt2o-chatbot-10.Q2K.gguf	Q2K	0.84GB
gpt2o-chatbot-10.IQ3XS.gguf	IQ3XS	0.84GB
gpt2o-chatbot-10.IQ3S.gguf	IQ3S	0.84GB
gpt2o-chatbot-10.Q3KS.gguf	Q3K_S	0.84GB
gpt2o-chatbot-10.IQ3M.gguf	IQ3M	0.91GB
gpt2o-chatbot-10.Q3K.gguf	Q3K	0.97GB
gpt2o-chatbot-10.Q3KM.gguf	Q3K_M	0.97GB
gpt2o-chatbot-10.Q3KL.gguf	Q3K_L	1.03GB
gpt2o-chatbot-10.IQ4XS.gguf	IQ4XS	0.9GB
gpt2o-chatbot-10.Q40.gguf	Q40	0.91GB
gpt2o-chatbot-10.IQ4NL.gguf	IQ4NL	0.91GB
gpt2o-chatbot-10.Q4KS.gguf	Q4K_S	1.04GB
gpt2o-chatbot-10.Q4K.gguf	Q4K	1.11GB
gpt2o-chatbot-10.Q4KM.gguf	Q4K_M	1.11GB
gpt2o-chatbot-10.Q41.gguf	Q41	1.0GB
gpt2o-chatbot-10.Q50.gguf	Q50	1.09GB
gpt2o-chatbot-10.Q5KS.gguf	Q5K_S	1.15GB
gpt2o-chatbot-10.Q5K.gguf	Q5K	1.29GB
gpt2o-chatbot-10.Q5KM.gguf	Q5K_M	1.29GB
gpt2o-chatbot-10.Q51.gguf	Q51	1.18GB
gpt2o-chatbot-10.Q6K.gguf	Q6K	1.52GB
gpt2o-chatbot-10.Q80.gguf	Q80	1.63GB

Original model description:

---

tags:

• autotrain
• text-generation-inference
• text-generation
• peft

library_name: transformers
base_model: styalai/gpt2o-chatbot-09
widget:
- messages:
- role: user
content: What is your favorite condiment?
license: other

---

Model Trained Using AutoTrain

This model was trained using AutoTrain. For more information, please visit AutoTrain.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

modelpath = "PATHTOTHISREPO"

tokenizer = AutoTokenizer.frompretrained(modelpath)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    torch_dtype='auto'
).eval()

Prompt content: "hi"
messages = [
    {"role": "user", "content": "hi"}
]

inputids = tokenizer.applychattemplate(conversation=messages, tokenize=True, addgenerationprompt=True, returntensors='pt')
outputids = model.generate(inputids.to('cuda'))
response = tokenizer.decode(outputids[0][inputids.shape[1]:], skipspecialtokens=True)

Model response: "Hello! How can I assist you today?"
print(response)