LLM · Text Generation
Llama3 8B Redmond Code290k
Fine-tuned LLaMA 3 8B for code generation and explanations across 10+ programming languages.
About this model
Llama3 8B Redmond Code290k is a fine-tuned variant of NousResearch/Meta-Llama-3-8B, trained on the ajibawa-2023/Code-290k-ShareGPT dataset — a collection of approximately 290,000 conversations in Vicuna/ShareGPT format. The model is optimized for code generation and code explanation across multiple programming languages including Python, Java, JavaScript, Go, C++, Rust, Ruby, SQL, MySQL, R, Julia, and Haskell.
The model was trained using Axolotl version 0.4.0 with a learning rate of 2e-05, cosine learning rate scheduling with 100 warmup steps, 3 epochs (2 per the training hyperparameters), and a context length of 8192 tokens. Training used 8-bit Adam optimizer with gradient checkpointing and flash attention enabled.
Recommended usage: the model works with any standard Hugging Face Transformers pipeline for text generation. It is compatible with text-generation-inference and endpoints. For code generation tasks, prompts should be clear and specific. The model performs best on mainstream programming languages and may degrade on niche languages or highly abstract code.
Limitations noted by the creator include reduced performance on rare programming languages, potential issues with unseen coding styles or conventions, limited handling of extremely complex code, and possible gaps in explaining highly abstract or theoretical concepts.
Learn more
Project signals
- 13 Hugging Face downloads
- 5 Hugging Face likes
Topics
transformers · safetensors · llama · text-generation · conversational · base_model:nousresearch/meta-llama-3-8b · base_model:finetune:nousresearch/meta-llama-3-8b · text-generation-inference · endpoints_compatible