• gguf
• embedding
• qwen3
• llama-cpp
• jina-embeddings-v5

• multilingual

jina-embeddings-v5-text-small-retrieval-GGUF

GGUF quantizations of jina-embeddings-v5-text-small-retrieval using llama.cpp. A 677M parameter multilingual embedding model quantized for efficient inference.

Elastic Inference Service | ArXiv | Blog

[!IMPORTANT]

We highly recommend to first read this blog post for more technical details and customized llama.cpp build.

Overview

jina-embeddings-v5-text-small-retrieval is a task-specific embedding model for retrieval, part of the jina-embeddings-v5-text model family.

Usage with llama.cpp

PUT inference/textembedding/jina-v5
{
  "service": "elastic",
  "service_settings": {
    "model_id": "jina-embeddings-v5-text-small"
  }
}

# Build llama.cpp (upstream)
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build --config Release

Run embedding
./build/bin/llama-embedding -m jina-embeddings-v5-text-small-retrieval-Q8_0.gguf \
  --pooling last -p "Your text here"

License

CC-BY-NC-4.0. For commercial use, please contact us.