LLM
Gemma2 2B OpenHermes2.5 GGUF
GGUF quantized 2B Gemma 2 model fine-tuned on the OpenHermes-2.5 synthetic GPT-4 dataset in ChatML format for efficient CPU/local inference.
About this model
Gemma2-2B-OpenHermes2.5-gguf is a GGUF-quantized version of the Gemma 2 2B base model fine-tuned on the Teknium OpenHermes-2.5 dataset, a large corpus of synthetic GPT-4-generated conversations. The quantization enables lower-resource local and CPU inference while retaining the chat-optimized weights of the original fine-tune. The model uses ChatML formatting and an instruction-following setup trained via synthetic data and distillation techniques.
This model is distributed under the Apache-2.0 license and supports transformers and GGUF backends. It is endpoints-compatible and tagged for general-purpose chat and instruct use. The dataset, Teknium/OpenHermes-2.5, is a well-known synthetic instruction-following collection derived from GPT-4 outputs, which gives the model strong general conversational capability despite its small 2B parameter size.
To use the GGUF weights, load the model with a compatible GGUF runtime such as llama.cpp, ollama, or text-generation-webui with the GGUF backend. For multi-part GGUF files, follow TheBloke's concatenation guidance. No specific instance prompt or trigger word is required; the model responds to standard ChatML-style instruction prompts. Recommended usage is for lightweight local chat agents, prototyping, and edge deployment where VRAM is limited.
Model details
| Category | LLM |
|---|---|
| Hugging Face downloads | 492 |
| Hugging Face likes | 4 |
| Data last synced | 2026-11-12 |
Tags
transformers · gguf · gemma · instruct · finetune · chatml · gpt4 · synthetic data · distillation · dataset:teknium/openhermes-2.5