_

Hardware and Software

Training Factors: The models were pretrained using the Megatron-DeepSpeed library on the LUMI cluster in Finland.

Carbon Footprint: Pretraining one model took approximately 70k GPU hours of computation on AMD MI250X GPUs (assuming 2 GPUs per one AMD MI250X device), each of which draws 500W.
LUMI is one of the most eco-efficient data centers in the world, and its energy consumption is covered 100% with renewable electricity.

_

Example usage

Let's try to use this model for English-to-Norwegian machine translation using simple zero-shot prompting:

from transformers import AutoTokenizer, AutoModelForCausalLM

First, we will have to import the tokenizer and the language model
tokenizer = AutoTokenizer.from_pretrained("norallm/normistral-7b-warm")
model = AutoModelForCausalLM.from_pretrained("norallm/normistral-7b-warm").cuda().eval()

Now we will define the zero-shot prompt template
prompt = """Engelsk: {0}
Bokmål:"""

A function that will take care of generating the output
@torch.no_grad()
def generate(text):
    text = prompt.format(text)
    inputids = tokenizer(text, returntensors='pt').input_ids.cuda()
    prediction = model.generate(
        input_ids,
        maxnewtokens=64,
        do_sample=False,
        eostokenid=tokenizer('\n').input_ids
    )
    return tokenizer.decode(prediction[0, input_ids.size(1):]).strip()

Now you can simply call the generate function with an English text you want to translate:
generate("I'm super excited about this Norwegian NORA model! Can it translate these sentences?")
> this should output: 'Jeg er super spent på denne norske NORA modellen! Kan den oversette disse setningene?'

Example usage on a GPU with ~16GB VRAM (try for yourself in Google Colab)

pip install bitsandbytes
pip install accelerate

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(
    "norallm/normistral-7b-warm"
)

This setup needs about 8gb VRAM
Setting loadin8bit=False -> 15gb VRAM
Using torch.float32 and loadin8bit=False -> 21gb VRAM
model = AutoModelForCausalLM.from_pretrained(
    "norallm/normistral-7b-warm",
    device_map='auto',
    loadin8bit=True,
    torch_dtype=torch.bfloat16
)

_

Quantization

Provided files

How to run from Python code

You can use GGUF models from Python using the llama-cpp-python for example.

#### How to load this model in Python code, using llama-cpp-python

For full documentation, please see: llama-cpp-python docs.

#### First install the package

Run one of the following commands, according to your system:

# Base llama-ccp-python with no GPU acceleration
pip install llama-cpp-python
With NVidia CUDA acceleration
CMAKEARGS="-DLLAMACUBLAS=on" pip install llama-cpp-python
Or with OpenBLAS acceleration
CMAKEARGS="-DLLAMABLAS=ON -DLLAMABLASVENDOR=OpenBLAS" pip install llama-cpp-python
Or with CLBLast acceleration
CMAKEARGS="-DLLAMACLBLAST=on" pip install llama-cpp-python
Or with AMD ROCm GPU acceleration (Linux only)
CMAKEARGS="-DLLAMAHIPBLAS=on" pip install llama-cpp-python
Or with Metal GPU acceleration for macOS systems only
CMAKEARGS="-DLLAMAMETAL=on" pip install llama-cpp-python

In windows, to set the variables CMAKE_ARGS in PowerShell, follow this format; eg for NVidia CUDA:
$env:CMAKEARGS = "-DLLAMAOPENBLAS=on"
pip install llama-cpp-python

#### Simple llama-cpp-python example code

from llama_cpp import Llama

Directly from huggingface-hub (requires huggingface-hub to be installed)
Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
llm = Llama.from_pretrained(
  repo_id="norallm/normistral-7b-warm",  # HuggingFace repository containing the GGUF files.
  filename="*Q4KM.gguf", # suffix of the filename containing the level of quantization. 
  n_ctx=32768,  # The max sequence length to use - note that longer sequence lengths require much more resources
  n_threads=8,            # The number of CPU threads to use, tailor to your system and the resulting performance
  ngpulayers=35         # The number of layers to offload to GPU, if you have GPU acceleration available
)

Simple inference example
output = llm(
  "Engelsk: Hello everyone! I'm a language model, how are you doing today?\nBokmål:", # Prompt
  max_tokens=512,  # Generate up to 512 tokens
  stop=["</s>"],   # Example stop token
  echo=True,       # Whether to echo the prompt
  temperature=0.3  # Temperature to set, for Q3KM, Q4KM, Q5KM, and Q6_0 it is recommended to set it relatively low.
)

Citation

@inproceedings{samuel-etal-2025-small,
    title = "Small Languages, Big Models: {A} Study of Continual Training on Languages of {Norway}",
    author = "Samuel, David  and
      Mikhailov, Vladislav  and
      Velldal, Erik  and
      {\O}vrelid, Lilja  and
      Charpentier, Lucas Georges Gabriel  and
      Kutuzov, Andrey  and
      Oepen, Stephan",
    editor = "Johansson, Richard  and
      Stymne, Sara",
    booktitle = "Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)",
    month = mar,
    year = "2025",
    address = "Tallinn, Estonia",
    publisher = "University of Tartu Library",
    url = "https://aclanthology.org/2025.nodalida-1.61/",
    pages = "573--608",
    ISBN = "978-9908-53-109-0"
}