Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ We evaluate the code solution generation ability of TLLM on three benchmarks: Wi
|
|
16 |
| TaPEX | 38.5 | β | β | β | 83.9 | 15.0 | / | 45.8 |
|
17 |
| TaPas | 31.5 | β | β | β | 74.2 | 23.1 | / | 42.92 |
|
18 |
| TableLlama | 24.0 | 22.2 | 18.9 | 6.4 | 43.7 | 9.0 | / | 20.7 |
|
19 |
-
| GPT3.5 | 58.5 |<ins>72.1</ins>| 71.2 | 60.8 | 81.7
|
20 |
| GPT4 |**74.1**|**77.1**|**78.4**|**69.5** | 84.0 | 69.5 | 77.8 | **75.8**|
|
21 |
| Llama2-Chat (13B) | 48.8 | 49.6 | 67.7 | 61.5 | β | β | β | 56.9 |
|
22 |
| CodeLlama (13B) | 43.4 | 47.2 | 57.2 | 49.7 | 38.3 | 21.9 | 47.6 | 43.6 |
|
@@ -33,17 +33,17 @@ The prompts we used for generating code solutions and text answers are introduce
|
|
33 |
### Code Solution
|
34 |
The prompt template for the insert, delete, update, query, and plot operations on a single table.
|
35 |
```
|
36 |
-
Below are the first few lines of a CSV file. You need to write a Python program to solve the provided question.
|
37 |
|
38 |
Header and first few lines of CSV file:
|
39 |
{csv_data}
|
40 |
|
41 |
-
Question: {question}
|
42 |
```
|
43 |
|
44 |
The prompt template for the merge operation on two tables.
|
45 |
```
|
46 |
-
Below are the first few lines two CSV file. You need to write a Python program to solve the provided question.
|
47 |
|
48 |
Header and first few lines of CSV file 1:
|
49 |
{csv_data1}
|
@@ -51,7 +51,7 @@ Header and first few lines of CSV file 1:
|
|
51 |
Header and first few lines of CSV file 2:
|
52 |
{csv_data2}
|
53 |
|
54 |
-
Question: {question}
|
55 |
```
|
56 |
|
57 |
The csv_data field is filled with the first few lines of your provided table file. Below is an example:
|
@@ -67,7 +67,7 @@ I,0.33,0.255,0.08,0.205,0.0895,0.0395,0.055,7
|
|
67 |
### Text Answer
|
68 |
The prompt template for direct text answer generation on short tables.
|
69 |
````
|
70 |
-
Offer a thorough and accurate solution that directly addresses the Question outlined in the [Question].
|
71 |
### [Table Text]
|
72 |
{table_descriptions}
|
73 |
|
@@ -79,5 +79,5 @@ Offer a thorough and accurate solution that directly addresses the Question outl
|
|
79 |
### [Question]
|
80 |
{question}
|
81 |
|
82 |
-
### [Solution]
|
83 |
````
|
|
|
16 |
| TaPEX | 38.5 | β | β | β | 83.9 | 15.0 | / | 45.8 |
|
17 |
| TaPas | 31.5 | β | β | β | 74.2 | 23.1 | / | 42.92 |
|
18 |
| TableLlama | 24.0 | 22.2 | 18.9 | 6.4 | 43.7 | 9.0 | / | 20.7 |
|
19 |
+
| GPT3.5 | 58.5 |<ins>72.1</ins>| 71.2 | 60.8 | 81.7 | 67.4 | 77.1 | 69.8 |
|
20 |
| GPT4 |**74.1**|**77.1**|**78.4**|**69.5** | 84.0 | 69.5 | 77.8 | **75.8**|
|
21 |
| Llama2-Chat (13B) | 48.8 | 49.6 | 67.7 | 61.5 | β | β | β | 56.9 |
|
22 |
| CodeLlama (13B) | 43.4 | 47.2 | 57.2 | 49.7 | 38.3 | 21.9 | 47.6 | 43.6 |
|
|
|
33 |
### Code Solution
|
34 |
The prompt template for the insert, delete, update, query, and plot operations on a single table.
|
35 |
```
|
36 |
+
[INST]Below are the first few lines of a CSV file. You need to write a Python program to solve the provided question.
|
37 |
|
38 |
Header and first few lines of CSV file:
|
39 |
{csv_data}
|
40 |
|
41 |
+
Question: {question}[/INST]
|
42 |
```
|
43 |
|
44 |
The prompt template for the merge operation on two tables.
|
45 |
```
|
46 |
+
[INST]Below are the first few lines two CSV file. You need to write a Python program to solve the provided question.
|
47 |
|
48 |
Header and first few lines of CSV file 1:
|
49 |
{csv_data1}
|
|
|
51 |
Header and first few lines of CSV file 2:
|
52 |
{csv_data2}
|
53 |
|
54 |
+
Question: {question}[/INST]
|
55 |
```
|
56 |
|
57 |
The csv_data field is filled with the first few lines of your provided table file. Below is an example:
|
|
|
67 |
### Text Answer
|
68 |
The prompt template for direct text answer generation on short tables.
|
69 |
````
|
70 |
+
[INST]Offer a thorough and accurate solution that directly addresses the Question outlined in the [Question].
|
71 |
### [Table Text]
|
72 |
{table_descriptions}
|
73 |
|
|
|
79 |
### [Question]
|
80 |
{question}
|
81 |
|
82 |
+
### [Solution][INST/]
|
83 |
````
|