LLM · Text Generation
Llama 3.2 1B Synthia II Redmond
Fine-tuned Llama 3.2 1B model trained on Synthia-v1.5-II dataset for improved instruction following in 8 languages.
About this model
Llama 3.2 1B Synthia II Redmond is a fine-tuned version of NousResearch's Llama 3.2 1B, trained on the Synthia-v1.5-II instruction dataset to improve instruction-following capabilities. The base model is a multilingual LLM developed by Meta supporting English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
Training used the Axolotl framework over 3 epochs with a learning rate of 2e-05, batch size of 1, gradient accumulation of 8 steps, a cosine learning rate scheduler with 100 warmup steps, and the Paged AdamW 8-bit optimizer. Framework versions include Transformers 4.46.1, PyTorch 2.3.1+cu121, Datasets 3.0.1, and Tokenizers 0.20.3. The model is distributed as safetensors weights (~1.2B parameters in BF16).
Intended uses are instruction-following tasks, conversational AI applications, and NLP research. The model is released under the Llama 3.2 Community License Agreement, which requires attribution and compliance with Meta's Acceptable Use Policy. GPU compute for training was provided by RedmondAI.
Learn more
Read the in-depth guide about Llama 3.2 1B Synthia II Redmond
Project signals
- 37 Hugging Face downloads
- 1 Hugging Face likes
Topics
transformers · pytorch · safetensors · llama · text-generation · facebook · meta · llama-3 · base_model:nousresearch/llama-3.2-1b · base_model:finetune:nousresearch/llama-3.2-1b