Update README.md
Browse files
README.md
CHANGED
|
@@ -69,6 +69,30 @@ pipe(do_closed_qa(test_article, question), max_new_tokens=128, temperature=0)[0]
|
|
| 69 |
|
| 70 |
# Training details
|
| 71 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
This model was trained for 1000 steps (1.2 epochs) with the model being evaluated every 50 steps. We then chose the best model from these evaluations based on validation loss.
|
| 73 |
We used the [qlora](https://github.com/artidoro/qlora) package from artidoro.
|
| 74 |
We trained with the following hyperparameters:
|
|
|
|
| 69 |
|
| 70 |
# Training details
|
| 71 |
|
| 72 |
+
We trained using the following three minimalistic prompt templates for the three tasks in STX:
|
| 73 |
+
|
| 74 |
+
* SNOW
|
| 75 |
+
```
|
| 76 |
+
f"""元の日本語:
|
| 77 |
+
{original_ja}
|
| 78 |
+
|
| 79 |
+
シンプルな日本語:"""
|
| 80 |
+
```
|
| 81 |
+
* TyDiQA
|
| 82 |
+
```
|
| 83 |
+
f"""{passage_text}
|
| 84 |
+
|
| 85 |
+
{question_text}"""
|
| 86 |
+
```
|
| 87 |
+
```
|
| 88 |
+
* XLSum
|
| 89 |
+
```
|
| 90 |
+
f"""記事:
|
| 91 |
+
{original_ja}
|
| 92 |
+
|
| 93 |
+
要約:"""
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
This model was trained for 1000 steps (1.2 epochs) with the model being evaluated every 50 steps. We then chose the best model from these evaluations based on validation loss.
|
| 97 |
We used the [qlora](https://github.com/artidoro/qlora) package from artidoro.
|
| 98 |
We trained with the following hyperparameters:
|