LLM
QWEN 2 1.5B Synthia I GGUF
GGUF-quantized instruct variant of QWEN 2 1.5B fine-tuned on the Synthia-v1.5-I dataset for ChatML-format conversational use.
About this model
QWEN-2-1.5B-Synthia-I-gguf is a GGUF-quantized version of the QWEN 2 1.5B Synthia-I model, an instruct-tuned fine-tune of Qwen 2 1.5B. The model was trained on the migtissera/Synthia-v1.5-I dataset, which contains synthetic data generated to emulate GPT-4 responses, and is designed for chat and instruction-following tasks.
The model uses the ChatML prompt format, making it compatible with Hugging Face transformers and any ChatML-aware frontend such as llama.cpp, LM Studio, or text-generation-webui. Quantization to GGUF format enables efficient CPU-based inference with minimal quality degradation, allowing it to run on consumer hardware without a GPU.
As a distilled, smaller-parameter model, it trades some generation quality for speed and low resource requirements. It is tagged as endpoints_compatible, meaning it can be deployed on Hugging Face Inference Endpoints. The Apache 2.0 license permits both commercial and non-commercial use.
Recommended usage: load with a ChatML-aware runtime, pair with a GGUF backend (e.g. llama.cpp), and use moderate temperatures (0.2–0.7) for instruction following. The quantized format supports multi-part file concatenation; consult TheBloke's GGUF README for setup details.
Project signals
- 366 Hugging Face downloads
- 3 Hugging Face likes
Topics
transformers · gguf · instruct · finetune · chatml · gpt4 · synthetic data · distillation · dataset:migtissera/synthia-v1.5-i · endpoints_compatible