LLM
Llama 3.2 1B OpenHermes 2.5 GGUF
GGUF quantized version of Llama 3.2 1B fine-tuned on OpenHermes-2.5 dataset for efficient CPU/GPU inference.
About this model
This model provides a GGUF-quantized version of the Llama 3.2 1B model that was fine-tuned on the OpenHermes-2.5 dataset by Teknium. The base model (artificialguybr/LLAMA-3.2-1B-OpenHermes2.5) was trained using synthetic data generated by GPT-4, following the OpenHermes methodology for instruction following and chat capabilities.
The GGUF format enables efficient inference on CPU and Apple Silicon via llama.cpp, ollama, and compatible runtimes. Multiple quantization levels are typically available (Q4_K_M, Q5_K_M, Q8_0, etc.), trading off model size for quality.
Trigger format: ChatML. The model uses the standard ChatML conversation template with system, user, and assistant roles. Recommended settings: temperature 0.7, top_p 0.9, max tokens 2048. Licensed under Apache 2.0.
Project signals
- 86 Hugging Face downloads
- 3 Hugging Face likes
Topics
transformers · gguf · llama · instruct · finetune · chatml · gpt4 · synthetic data · distillation · dataset:teknium/openhermes-2.5