LLM · Text Generation
Qwen2-1.5B Synthia II Redmond
Qwen2-1.5B fine-tuned on 20.7k Synthia v1.5-II instruction examples for improved instruction following. Apache-2.0 license.
About this model
This model is a fine-tuned version of Qwen/Qwen2-1.5B trained on the Synthia v1.5-II dataset containing over 20.7k instruction-following examples. The fine-tuning was sponsored by Redmond.ai and performed using Axolotl with Transformers 4.45.0.
The model inherits the base Qwen2-1.5B capabilities in language understanding, structured data processing, multilingual support, and long context handling (4096 tokens), while being specifically enhanced for instruction following and conversational tasks.
Training used 3 epochs with a learning rate of 1e-05, cosine LR scheduler with 100 warmup steps, Adam optimizer, and sample packing enabled. Total train batch size was 40 with gradient accumulation of 8.
License: Apache-2.0. Framework: Transformers, PyTorch 2.3.1, Axolotl 0.4.1. Intended for instruction following, text generation, and conversational AI applications.
Project signals
- 22 Hugging Face downloads
- 2 Hugging Face likes
Topics
transformers · pytorch · qwen2 · text-generation · instruction-tuning · conversational · base_model:qwen/qwen2-1.5b · base_model:finetune:qwen/qwen2-1.5b · text-generation-inference · endpoints_compatible