Gemma 2 2B OpenHermes2.5: A Lightweight Instruct Finetune

TL;DR:

• Gemma 2 2B OpenHermes2.5 is Google's 2B Gemma 2 base model fine-tuned on the OpenHermes-2.5 dataset, distributed under Apache 2.0.
• It is built for instruction following and chat, and uses the ChatML conversation format (system/user/assistant turns).
• At 2B parameters it runs on modest hardware, including CPU-only inference and free-tier GPU instances.

What the model does

Gemma 2 2B OpenHermes2.5 is a text-generation model created by ArtificialGuyBR. It starts from google/gemma-2-2b and is fine-tuned on teknium/OpenHermes-2.5, a dataset built largely from GPT-4 synthetic data. The result is a small instruction-tuned model aimed at general language tasks: answering questions, following instructions, and holding conversational turns.

Because the OpenHermes-2.5 dataset is formatted in ChatML, the model expects prompts in that structure — <|im_start|> and <|im_end|> tokens separating system, user, and assistant turns. The model card tags (chatml, instruct, gpt4, synthetic data, distillation) reflect this directly.

Training details

According to the model card, the fine-tune used a single NVIDIA A100-SXM4-80GB GPU with the Axolotl training framework on top of Hugging Face Transformers. The base model is google/gemma-2-2b, released under the Gemma license; the finetune itself is distributed under Apache 2.0, which makes it permissive for commercial and research use. Training data and procedure are limited to the summary above — the card does not publish hyperparameters.

How to use it

The model is published as transformers-compatible safetensors weights and is listed as compatible with text-generation-inference and Hugging Face inference endpoints. In practice you can load it with the standard Transformers pipeline API or through TGI, and the size makes it viable on CPU, small GPUs, and free-tier cloud instances. For conversational use, apply the ChatML template rather than sending raw instruction text, since that is the format the fine-tune was trained on.

For other lightweight instruct options in the same space, see our Gemma 2 variants and 2B instruct model roundup; if you are choosing between base and instruct weights, the finetune vs base model guide covers the tradeoffs.

Common errors and fixes

Error | Cause | Fix | Source

|---|---|---|---

Rambling or off-topic replies | Prompt sent as plain text instead of ChatML turns | Wrap input in <|im_start|>system/user/assistant blocks | Model card Slow generation on CPU | 2B model still runs dense attention per token | Use 4-bit/8-bit quantization or a GPU backend | Model card tags (transformers, safetensors) CORS/format errors on inference endpoints | TGI sometimes needs an explicit chat template | Request with a ChatML template or the TGI chat API | Model card (text-generation-inference tag)

FAQ

What is Gemma 2 2B OpenHermes2.5? A fine-tune of Google's Gemma 2 2B on the OpenHermes-2.5 dataset, made by ArtificialGuyBR for instruction following and general language tasks.

What base model is it trained on? google/gemma-2-2b, per the model card's "Finetuned from model" field.

What license does it use? Apache 2.0, stated in the model card. The base Gemma 2 model itself has its own Gemma terms of use.

What prompt format does it expect? ChatML, matching the OpenHermes-2.5 dataset format (system/user/assistant turns with im_start/im_end tokens).

Can it run without a big GPU? At 2B parameters it is small by LLM standards; the card does not publish VRAM requirements, but the size is compatible with CPU inference and modest GPUs.

How was it trained? With Axolotl and Hugging Face Transformers on one NVIDIA A100-SXM4-80GB GPU, per the model card.

Sources

• Hugging Face model card: artificialguybr/Gemma2-2B-OpenHermes2.5 — training data, hardware, license, and usage notes.