The following Python packages are required:

• transformers>=5.1.0
• torch>=2.8.0
• peft>=0.15.2
• vllm==0.15.1

Optional / Recommended

• flash-attention: Installing flash-attention is recommended for improved inference speed and efficiency, but not mandatory.
• sentence-transformers: If you want to use the model via the sentence-transformers interface, install this package as well.

The fastest way to use v5-text in production. Elastic Inference Service (EIS) provides managed embedding inference with built-in scaling, so you can generate embeddings directly within your Elastic deployment.

PUT inference/textembedding/jina-v5
{
  "service": "elastic",
  "service_settings": {
    "model_id": "jina-embeddings-v5-text-nano"
  }
}

See the Elastic Inference Service documentation for setup details.

from sentence_transformers import SentenceTransformer
import torch

model = SentenceTransformer(
    "jinaai/jina-embeddings-v5-text-nano-clustering",
    trustremotecode=True,
    model_kwargs={"dtype": torch.bfloat16},  # Recommended for GPUs
    configkwargs={"attnimplementation": "flashattention_2"},  # Recommended but optional
)
Optional: set truncate_dim in encode() to control embedding size


texts = [
    "We propose a novel neural network architecture for image segmentation.",
    "This paper analyzes the effects of monetary policy on inflation.",
    "Our method achieves state-of-the-art results on object detection benchmarks.",
    "We study the relationship between interest rates and housing prices.",
    "A new attention mechanism is introduced for visual recognition tasks.",
]

Encode texts
embeddings = model.encode(texts)
print(embeddings.shape)
(5, 768)

similarity = model.similarity(embeddings, embeddings)
print(similarity)
tensor([[1.0000, 0.2933, 0.9304, 0.2928, 0.8635],

        [0.2933, 1.0000, 0.3062, 0.8083, 0.3035],

        [0.9304, 0.3062, 1.0000, 0.2943, 0.8651],

        [0.2928, 0.8083, 0.2943, 1.0000, 0.2827],

        [0.8635, 0.3035, 0.8651, 0.2827, 1.0000]])

from vllm import LLM

Initialize model
name = "jinaai/jina-embeddings-v5-text-nano-clustering"
model = LLM(
    model=name,
    dtype="float16",
    runner="pooling",
    trustremotecode=True,
    poolerconfig=PoolerConfig(seqpooling_type="LAST", normalize=True)
)

Create text prompts
query = "Overview of climate change impacts on coastal cities"
query_prompt = f"Query: {query}"

document = "The impacts of climate change on coastal cities are significant.."
document_prompt = f"Document: {document}"

Encode all prompts
prompts = [queryprompt, documentprompt]
outputs = model.encode(prompts, pooling_task="embed")

embed_query = outputs[0].outputs.data
embed_document = outputs[1].outputs.data

Since our nano model is based on jinaai/jina-embeddings-v5-text-nano, which is not yet supported by llama.cpp, we provide our own branch of llama.cpp, which implements the necessary changes to support it for now.

To start the OpenAI API compatible HTTP server, run with the respective model version:

llama-server \
  -hf jinaai/jina-embeddings-v5-text-nano-clustering:F16 \
  --embedding \
  --pooling last \
  --batch-size 8192 \
  --ubatch-size 8192 \
  --ctx-size 8192

Client:

curl -X POST "http://127.0.0.1:8080/v1/embeddings" \
  -H "Content-Type: application/json" \
  -d '{
    "input": [
      "Document: A beautiful sunset over the beach",
      "Document: Un beau coucher de soleil sur la plage",
      "Document: 海滩上美丽的日落",
      "Document: 浜辺に沈む美しい夕日",
      "Document: Golden sunlight melts into the horizon, painting waves in warm amber and rose, while the sky whispers goodnight to the quiet, endless sea."
    ]
  }'

Note: For the clustering variant, always add Document: prefix in front of your input as shown above.