LLM · Text Generation
Qwen 2.5 0.5B Synthia I
Fine-tuned Qwen 2.5 0.5B LLM trained on 20.7k instruction-following examples from Synthia v1.5-I dataset. Optimized for conversational AI and task completion.
About this model
Qwen 2.5 0.5B Synthia I is a fine-tuned version of the Qwen/Qwen2.5-0.5B base model, enhanced through instruction tuning on the Synthia v1.5-I dataset containing over 20,700 examples. The model delivers improved instruction-following capabilities, text generation, and conversational AI performance while maintaining the base model's multilingual support (29+ languages), 32,768 token context length, and structured data handling.
Key specifications: 0.49B parameters (0.36B non-embedding), 24 transformer layers, Grouped-Query Attention (14 Q heads, 2 KV heads), trained with Adam optimizer at 1e-5 learning rate for 3 epochs using a batch size of 40 (gradient accumulation). Built with Transformers 4.45.0.dev0 and PyTorch 2.3.1.
Use for: Instruction following, conversational applications, text generation tasks, and any scenario requiring reliable instruction adherence from a compact 0.5B parameter model.
Project signals
- 23 Hugging Face downloads
- 2 Hugging Face likes
Topics
transformers · pytorch · qwen2 · text-generation · instruction-tuning · conversational · base_model:qwen/qwen2.5-0.5b · base_model:finetune:qwen/qwen2.5-0.5b · text-generation-inference · endpoints_compatible