LLM
Qwen2.5 0.5B OpenHermes2.5 GGUF
GGUF quantization of Qwen2.5-0.5B fine-tuned on OpenHermes 2.5, for efficient local CPU inference.
About this model
This is the GGUF quantized release of artificialguybr's Qwen2.5-0.5B-OpenHermes2.5, a 0.49B parameter instruction fine-tune of Qwen2.5-0.5B trained on the OpenHermes 2.5 dataset — a curated compilation of 1 million primarily synthetically generated instruction and chat samples (Apache-2.0 licensed). The GGUF format makes the model runnable in llama.cpp-based runtimes such as Ollama, LM Studio, and llama-cpp-python, which keeps the already-tiny 0.5B model usable on modest hardware, including CPU-only machines.
The underlying fine-tune was trained with the Axolotl framework using a cosine learning-rate schedule (1e-05), BF16 mixed precision, a 4096-token sequence length, and 3 epochs. It inherits the Qwen2.5 architecture: Transformers with RoPE, SwiGLU, RMSNorm, and GQA attention (14 query heads, 2 KV heads), with a 32,768-token context length. The card recommends updating to Hugging Face transformers 4.37.0 or later; for GGUF usage, follow TheBloke's standard GGUF guides for loading and concatenating multi-part files.
As a small instruct model it is best used for lightweight conversational and text-generation tasks where low memory and fast inference matter more than peak quality — a companion to larger OpenHermes-style models. No dedicated trigger prompt is required: the model follows standard ChatML-style instruction prompting from the OpenHermes 2.5 fine-tune. License is Apache-2.0, matching both the base Qwen2.5 and the OpenHermes 2.5 dataset.
Learn more
Read the in-depth guide about Qwen2.5 0.5B OpenHermes2.5 GGUF
Project signals
- 139 Hugging Face downloads
- 2 Hugging Face likes
Topics
transformers · gguf · qwen · instruct · finetune · chatml · gpt4 · synthetic data · distillation · dataset:teknium/openhermes-2.5