LLM · Text Generation
Llama3 8B SQL Create Context
Fine-tuned Llama 3 8B that generates SQL queries from natural language using CREATE TABLE statements as context to prevent hallucination.
About this model
Llama3 8B SQL Create Context is a text-to-SQL model fine-tuned from NousResearch/Meta-Llama-3-8B on the b-mc2/sql-create-context dataset. It takes a natural language question and a SQL CREATE TABLE statement as input, then outputs a SQL query that answers the question based on the provided schema.
The model was trained on 78,577 examples combining the WikiSQL and Spider datasets, designed to prevent hallucination of column and table names — a common failure mode in text-to-SQL systems. By grounding generation in the actual table schema via CREATE TABLE statements, the model avoids inventing non-existent tables or columns.
Training used Axolotl with cosine learning rate scheduling (lr=2e-5, 100 warmup steps), 3 epochs, paged AdamW 8-bit optimizer, and flash attention on an 8192 sequence length. It achieved a validation loss of 0.0201. The model is intended for applications where the table schema is known and is best suited for straightforward queries — it may struggle with complex joins, subqueries, or ambiguous questions.
Learn more
Project signals
- 13 Hugging Face downloads
- 7 Hugging Face likes
Topics
transformers · pytorch · safetensors · llama · text-generation · conversational · base_model:nousresearch/meta-llama-3-8b · base_model:finetune:nousresearch/meta-llama-3-8b · text-generation-inference · endpoints_compatible