tinyopsec/Llama-3-Instruct-Neurona-8b-GGUF

Llama-3-Instruct-Neurona-8b GGUF

GGUF quantizations of Iker/Llama-3-Instruct-Neurona-8b.

Neurona 8B is a Spanish-first instruction-tuned language model based on Meta-Llama-3-8B-Instruct, fine-tuned on a mixture of 24 Spanish and English datasets covering RAG, function calling, code assistance, summarization, translation, and question answering. Trained with Axolotl on 4×A100 80GB GPUs.

---

Quantization Summary

File	Bits	Size	Use Case
model_f16.gguf	16	~16.1 GB	Full precision reference
model_q8_0.gguf	8	~8.5 GB	Best quality, high VRAM
model_q6_k.gguf	6	~6.6 GB	Near-lossless, recommended
model_q5_k_m.gguf	5	~5.7 GB	Balanced quality/size
model_q5_k_s.gguf	5	~5.4 GB	Slightly smaller Q5
model_q4_k_m.gguf	4	~4.9 GB	Good quality, everyday use
model_q4_k_s.gguf	4	~4.6 GB	Smaller Q4
model_q3_k_l.gguf	3	~4.0 GB	Low VRAM, acceptable quality
model_q3_k_m.gguf	3	~3.7 GB	Lower VRAM
model_q3_k_s.gguf	3	~3.4 GB	Minimum quality Q3
model_q2_k.gguf	2	~2.9 GB	Very low VRAM, degraded quality

---

VRAM Requirements

Quantization	VRAM (model only)	Fits on
Q8_0	~9 GB	RTX 3090/4090, A10
Q6_K	~7 GB	RTX 3080, A10
Q5_K_M	~6 GB	RTX 3070/4070
Q4_K_M	~5 GB	RTX 3060/4060
Q3_K_M	~4 GB	RTX 3050, GTX 1660
Q2_K	~3 GB	Low-end GPUs, CPU

---

Usage

llama.cpp

./llama-cli -m model_q4_k_m.gguf -p "Hola, ¿cómo estás?" -n 256

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="model_q4_k_m.gguf", n_ctx=8192)
response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "¿Cuál es la capital de España?"}]
)
print(response["choices"][0]["message"]["content"])

LM Studio

1. Open LM Studio → Search tinyopsec/Llama-3-Instruct-Neurona-8b-GGUF
2. Download preferred quantization
3. Load and chat

Ollama

ollama run hf.co/tinyopsec/Llama-3-Instruct-Neurona-8b-GGUF:Q4_K_M

---

Chat Template

Uses LLaMA 3 chat template with <|begin_of_text|>, <|start_header_id|>, <|end_header_id|>, <|eot_id|> tokens.

---

Original Model

Iker/Llama-3-Instruct-Neurona-8b — base: meta-llama/Meta-Llama-3-8B-Instruct, license: llama3.