Caleb Fahlgren
		
	commited on
		
		
					Commit 
							
							·
						
						de36965
	
1
								Parent(s):
							
							5acf1f7
								
add readme
Browse files
    	
        README.md
    CHANGED
    
    | @@ -1,3 +1,30 @@ | |
| 1 | 
             
            ---
         | 
| 2 | 
             
            license: apache-2.0
         | 
|  | |
|  | |
|  | |
|  | |
|  | |
| 3 | 
             
            ---
         | 
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | 
|  | |
| 1 | 
             
            ---
         | 
| 2 | 
             
            license: apache-2.0
         | 
| 3 | 
            +
            language:
         | 
| 4 | 
            +
              - en
         | 
| 5 | 
            +
            pipeline_tag: text-generation
         | 
| 6 | 
            +
            tags:
         | 
| 7 | 
            +
              - code
         | 
| 8 | 
             
            ---
         | 
| 9 | 
            +
            # sql-coder-7b compiled for WebGPU - q4f32_1
         | 
| 10 | 
            +
             | 
| 11 | 
            +
            - Original model: [SQLCoder 7B](https://huggingface.co/defog/sqlcoder-7b)
         | 
| 12 | 
            +
            - compiled by: Caleb Fahlgren: [👋 on Twitter](https://twitter.com/calebfahlgren)
         | 
| 13 | 
            +
             | 
| 14 | 
            +
            SQLCoder-7B is a 7B parameter model that outperforms gpt-3.5-turbo for natural language to SQL generation tasks on our sql-eval framework, and significantly outperforms all popular open-source models. When fine-tuned on a given schema, it also outperforms gpt-4
         | 
| 15 | 
            +
             | 
| 16 | 
            +
            SQLCoder-7B is fine-tuned on a base Mistral-7B model.
         | 
| 17 | 
            +
             | 
| 18 | 
            +
             | 
| 19 | 
            +
            ## Results on novel datasets not seen in training
         | 
| 20 | 
            +
            | model              | perc_correct |
         | 
| 21 | 
            +
            | ------------------ | ------------ |
         | 
| 22 | 
            +
            | gpt4-2023-10-04    | 82.0         |
         | 
| 23 | 
            +
            | defog-sqlcoder2    | 74.5         |
         | 
| 24 | 
            +
            | gpt4-2023-08-28    | 74.0         |
         | 
| 25 | 
            +
            | defog-sqlcoder-7b  | 71.0         |
         | 
| 26 | 
            +
            | gpt-3.5-2023-10-04 | 66.0         |
         | 
| 27 | 
            +
            | claude-2           | 64.5         |
         | 
| 28 | 
            +
            | gpt-3.5-2023-08-28 | 61.0         |
         | 
| 29 | 
            +
            | claude_instant_1   | 61.0         |
         | 
| 30 | 
            +
            | text-davinci-003   | 52.5         |
         | 
