AmelieSchreiber
commited on
Commit
•
27d5c3c
1
Parent(s):
faa1af6
Update README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ license: mit
|
|
7 |
In this model, we wanted to see how the performance metrics were effected by adapting additional weight matrices with QLoRA. This was
|
8 |
shown to be the most important hyperparameter for improvement in performance metrics by far. So, we decided to test the difference between
|
9 |
simply using the query, key, and value weight matrix adapters to using adapters for all possible weight matrices. The comparison for the
|
10 |
-
first epoch can be seen below:
|
11 |
|
12 |
### This model
|
13 |
|
|
|
7 |
In this model, we wanted to see how the performance metrics were effected by adapting additional weight matrices with QLoRA. This was
|
8 |
shown to be the most important hyperparameter for improvement in performance metrics by far. So, we decided to test the difference between
|
9 |
simply using the query, key, and value weight matrix adapters to using adapters for all possible weight matrices. The comparison for the
|
10 |
+
first epoch can be seen below. Note the minor performance improvements for the model using every possible weight matrix:
|
11 |
|
12 |
### This model
|
13 |
|