Update model 2024-08-30

README.md

@@ -50,8 +50,8 @@ print(tokenizer.detokenize(translated[0][0]['tokens']))
 
 | testset                               | BLEU  |
 |---------------------------------------|-------|
-| test dataset (from train/dev/test)   	| 41.4	|
-| Flores200 dataset                     | 35.4	|
+| test dataset (from train/dev/test)   	| 43.1	|
+| Flores200 dataset                     | 37.0	|
 
 ## Additional information
 * https://github.com/Softcatala/nmt-models

config.json

@@ -1,2 +1,9 @@
 {
-}
+  "add_source_bos": false,
+  "add_source_eos": false,
+  "bos_token": "<s>",
+  "decoder_start_token": "<s>",
+  "eos_token": "</s>",
+  "layer_norm_epsilon": null,
+  "unk_token": "<unk>"
+}

model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dff810808ae118b1820379e8294ce2f18aef009302f67eaac130c5d563f295c8
-size 70727761
+oid sha256:ab8e375a485280844f43b66b20332ff2410bae31b124d49a23bb90bbd7620631
+size 70727741

model_description.txt

@@ -1,9 +1,9 @@
 Model description: cat-fra
-Date: 2022-11-14
-TF version 2.10.0, OpenNMT version 2.29.1, CTranslate2 version 2.24.0
+Date: 2024-08-30
+TF version 2.11.0, OpenNMT version 2.32.0, CTranslate2 version 3.22.0
 Test data set
-  BLEU|nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 41.4 67.8/47.5/35.9/27.7 (BP = 0.978 ratio = 0.979 hyp_len = 155890 ref_len = 159288)
-chrF2|nrefs:1|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 65.1
+  BLEU|nrefs:1|bs:1000|seed:12345|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 43.1 (μ = 43.1 ± 0.5) 68.7/48.9/37.2/28.8 (BP = 0.989 ratio = 0.989 hyp_len = 268705 ref_len = 271618)
+chrF2|nrefs:1|bs:1000|seed:12345|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 67.1 (μ = 67.1 ± 0.3)
 Flores data set
-  BLEU|nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 35.4 65.0/42.4/29.8/21.5 (BP = 0.970 ratio = 0.971 hyp_len = 27511 ref_len = 28343)
-chrF2|nrefs:1|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 61.3
+  BLEU|nrefs:1|bs:1000|seed:12345|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 37.0 (μ = 36.9 ± 1.3) 65.5/43.4/31.1/22.9 (BP = 0.981 ratio = 0.981 hyp_len = 27809 ref_len = 28343)
+chrF2|nrefs:1|bs:1000|seed:12345|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 62.4 (μ = 62.3 ± 0.8)

sp_m.model

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:239d3c092565a4b9bb813209fd818d10229a3b506a0f00969ba8819e4139f0b2
-size 1168335
+oid sha256:955624c95ecf99abc8aef8717f12823ac36adc0a8a439a33164f8ebffda0e630
+size 1159044