metadata

library_name: transformers
license: apache-2.0
base_model: answerdotai/ModernBERT-base
tags:
  - generated_from_trainer
model-index:
  - name: ModernBERT-base-iob2-20241223232203
    results: []

ModernBERT-base-iob2-20241223232203

This model is a fine-tuned version of answerdotai/ModernBERT-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.6304
Overall Strict Precision: 0.1472
Overall Strict Recall: 0.1448
Overall Strict F1: 0.1460
Overall Ent Type Precision: 0.3106
Overall Ent Type Recall: 0.3054
Overall Ent Type F1: 0.3080
Overall Partial Precision: 0.2674
Overall Partial Recall: 0.2630
Overall Partial F1: 0.2651
Overall Exact Precision: 0.1567
Overall Exact Recall: 0.1541
Overall Exact F1: 0.1554
Bookingnumber Strict Precision: 0.0
Bookingnumber Strict Recall: 0.0
Bookingnumber Strict F1: 0
Bookingnumber Ent Type Precision: 0.0000
Bookingnumber Ent Type Recall: 0.0078
Bookingnumber Ent Type F1: 0.0000
Bookingnumber Partial Precision: 0.0001
Bookingnumber Partial Recall: 0.0547
Bookingnumber Partial F1: 0.0001
Bookingnumber Exact Precision: 0.0000
Bookingnumber Exact Recall: 0.0195
Bookingnumber Exact F1: 0.0000
Invoicedate Strict Precision: 0.0100
Invoicedate Strict Recall: 0.0868
Invoicedate Strict F1: 0.0180
Invoicedate Ent Type Precision: 0.0302
Invoicedate Ent Type Recall: 0.2614
Invoicedate Ent Type F1: 0.0542
Invoicedate Partial Precision: 0.0266
Invoicedate Partial Recall: 0.2298
Invoicedate Partial F1: 0.0476
Invoicedate Exact Precision: 0.0114
Invoicedate Exact Recall: 0.0987
Invoicedate Exact F1: 0.0205
Companycountry Strict Precision: 0.0000
Companycountry Strict Recall: 0.0134
Companycountry Strict F1: 0.0000
Companycountry Ent Type Precision: 0.0002
Companycountry Ent Type Recall: 0.1769
Companycountry Ent Type F1: 0.0005
Companycountry Partial Precision: 0.0001
Companycountry Partial Recall: 0.0952
Companycountry Partial F1: 0.0003
Companycountry Exact Precision: 0.0000
Companycountry Exact Recall: 0.0134
Companycountry Exact F1: 0.0000
Companyname Strict Precision: 0.0008
Companyname Strict Recall: 0.0065
Companyname Strict F1: 0.0014
Companyname Ent Type Precision: 0.0290
Companyname Ent Type Recall: 0.2446
Companyname Ent Type F1: 0.0518
Companyname Partial Precision: 0.0221
Companyname Partial Recall: 0.1862
Companyname Partial F1: 0.0395
Companyname Exact Precision: 0.0012
Companyname Exact Recall: 0.0102
Companyname Exact F1: 0.0022
Hotelvatnumber Strict Precision: 0.0012
Hotelvatnumber Strict Recall: 0.0279
Hotelvatnumber Strict F1: 0.0022
Hotelvatnumber Ent Type Precision: 0.0083
Hotelvatnumber Ent Type Recall: 0.1981
Hotelvatnumber Ent Type F1: 0.0159
Hotelvatnumber Partial Precision: 0.0060
Hotelvatnumber Partial Recall: 0.1437
Hotelvatnumber Partial F1: 0.0115
Hotelvatnumber Exact Precision: 0.0014
Hotelvatnumber Exact Recall: 0.0342
Hotelvatnumber Exact F1: 0.0027
Hotelfaxnumber Strict Precision: 0.0016
Hotelfaxnumber Strict Recall: 0.0219
Hotelfaxnumber Strict F1: 0.0029
Hotelfaxnumber Ent Type Precision: 0.0124
Hotelfaxnumber Ent Type Recall: 0.1729
Hotelfaxnumber Ent Type F1: 0.0232
Hotelfaxnumber Partial Precision: 0.0084
Hotelfaxnumber Partial Recall: 0.1167
Hotelfaxnumber Partial F1: 0.0157
Hotelfaxnumber Exact Precision: 0.0018
Hotelfaxnumber Exact Recall: 0.0249
Hotelfaxnumber Exact F1: 0.0033
Hotelphonenumber Strict Precision: 0.0083
Hotelphonenumber Strict Recall: 0.0639
Hotelphonenumber Strict F1: 0.0147
Hotelphonenumber Ent Type Precision: 0.0336
Hotelphonenumber Ent Type Recall: 0.2589
Hotelphonenumber Ent Type F1: 0.0595
Hotelphonenumber Partial Precision: 0.0301
Hotelphonenumber Partial Recall: 0.2318
Hotelphonenumber Partial F1: 0.0533
Hotelphonenumber Exact Precision: 0.0110
Hotelphonenumber Exact Recall: 0.0850
Hotelphonenumber Exact F1: 0.0195
Hotelbankaccount Strict Precision: 0.0
Hotelbankaccount Strict Recall: 0
Hotelbankaccount Strict F1: 0
Hotelbankaccount Ent Type Precision: 0.0
Hotelbankaccount Ent Type Recall: 0
Hotelbankaccount Ent Type F1: 0
Hotelbankaccount Partial Precision: 0.0
Hotelbankaccount Partial Recall: 0
Hotelbankaccount Partial F1: 0
Hotelbankaccount Exact Precision: 0.0
Hotelbankaccount Exact Recall: 0
Hotelbankaccount Exact F1: 0
Companyvatnumber Strict Precision: 0.0
Companyvatnumber Strict Recall: 0.0
Companyvatnumber Strict F1: 0
Companyvatnumber Ent Type Precision: 0.0010
Companyvatnumber Ent Type Recall: 0.1043
Companyvatnumber Ent Type F1: 0.0020
Companyvatnumber Partial Precision: 0.0009
Companyvatnumber Partial Recall: 0.0980
Companyvatnumber Partial F1: 0.0018
Companyvatnumber Exact Precision: 0.0001
Companyvatnumber Exact Recall: 0.0150
Companyvatnumber Exact F1: 0.0003
Pagenumber Strict Precision: 0.0032
Pagenumber Strict Recall: 0.2296
Pagenumber Strict F1: 0.0063
Pagenumber Ent Type Precision: 0.0056
Pagenumber Ent Type Recall: 0.4061
Pagenumber Ent Type F1: 0.0111
Pagenumber Partial Precision: 0.0044
Pagenumber Partial Recall: 0.3197
Pagenumber Partial F1: 0.0087
Pagenumber Exact Precision: 0.0032
Pagenumber Exact Recall: 0.2312
Pagenumber Exact F1: 0.0063
Hotelcity Strict Precision: 0.0005
Hotelcity Strict Recall: 0.0274
Hotelcity Strict F1: 0.0009
Hotelcity Ent Type Precision: 0.0022
Hotelcity Ent Type Recall: 0.1272
Hotelcity Ent Type F1: 0.0043
Hotelcity Partial Precision: 0.0020
Hotelcity Partial Recall: 0.1150
Hotelcity Partial F1: 0.0039
Hotelcity Exact Precision: 0.0007
Hotelcity Exact Recall: 0.0399
Hotelcity Exact F1: 0.0013
Roomno Strict Precision: 0.0015
Roomno Strict Recall: 0.6968
Roomno Strict F1: 0.0029
Roomno Ent Type Precision: 0.0015
Roomno Ent Type Recall: 0.6968
Roomno Ent Type F1: 0.0029
Roomno Partial Precision: 0.0015
Roomno Partial Recall: 0.7110
Roomno Partial F1: 0.0030
Roomno Exact Precision: 0.0015
Roomno Exact Recall: 0.7110
Roomno Exact F1: 0.0030
Hoteltaxcode Strict Precision: 0.0000
Hoteltaxcode Strict Recall: 0.0016
Hoteltaxcode Strict F1: 0.0001
Hoteltaxcode Ent Type Precision: 0.0011
Hoteltaxcode Ent Type Recall: 0.0685
Hoteltaxcode Ent Type F1: 0.0022
Hoteltaxcode Partial Precision: 0.0006
Hoteltaxcode Partial Recall: 0.0377
Hoteltaxcode Partial F1: 0.0012
Hoteltaxcode Exact Precision: 0.0000
Hoteltaxcode Exact Recall: 0.0023
Hoteltaxcode Exact F1: 0.0001
Documenttype Strict Precision: 0.0055
Documenttype Strict Recall: 0.1654
Documenttype Strict F1: 0.0107
Documenttype Ent Type Precision: 0.0142
Documenttype Ent Type Recall: 0.4232
Documenttype Ent Type F1: 0.0274
Documenttype Partial Precision: 0.0102
Documenttype Partial Recall: 0.3054
Documenttype Partial F1: 0.0198
Documenttype Exact Precision: 0.0056
Documenttype Exact Recall: 0.1670
Documenttype Exact F1: 0.0108
Companyzipcode Strict Precision: 0.0072
Companyzipcode Strict Recall: 0.1910
Companyzipcode Strict F1: 0.0139
Companyzipcode Ent Type Precision: 0.0086
Companyzipcode Ent Type Recall: 0.2263
Companyzipcode Ent Type F1: 0.0165
Companyzipcode Partial Precision: 0.0091
Companyzipcode Partial Recall: 0.2400
Companyzipcode Partial F1: 0.0175
Companyzipcode Exact Precision: 0.0082
Companyzipcode Exact Recall: 0.2166
Companyzipcode Exact F1: 0.0158
Currencycode Strict Precision: 0.1345
Currencycode Strict Recall: 0.5663
Currencycode Strict F1: 0.2174
Currencycode Ent Type Precision: 0.1347
Currencycode Ent Type Recall: 0.5671
Currencycode Ent Type F1: 0.2177
Currencycode Partial Precision: 0.1350
Currencycode Partial Recall: 0.5684
Currencycode Partial F1: 0.2182
Currencycode Exact Precision: 0.1346
Currencycode Exact Recall: 0.5666
Currencycode Exact F1: 0.2175
Companycity Strict Precision: 0.0047
Companycity Strict Recall: 0.1145
Companycity Strict F1: 0.0090
Companycity Ent Type Precision: 0.0122
Companycity Ent Type Recall: 0.2989
Companycity Ent Type F1: 0.0234
Companycity Partial Precision: 0.0112
Companycity Partial Recall: 0.2741
Companycity Partial F1: 0.0215
Companycity Exact Precision: 0.0052
Companycity Exact Recall: 0.1285
Companycity Exact F1: 0.0101
Cardnumber Strict Precision: 0.0
Cardnumber Strict Recall: 0.0
Cardnumber Strict F1: 0
Cardnumber Ent Type Precision: 0.0
Cardnumber Ent Type Recall: 0.0
Cardnumber Ent Type F1: 0
Cardnumber Partial Precision: 0.0
Cardnumber Partial Recall: 0.0
Cardnumber Partial F1: 0
Cardnumber Exact Precision: 0.0
Cardnumber Exact Recall: 0.0
Cardnumber Exact F1: 0
Grossamount Strict Precision: 0.0
Grossamount Strict Recall: 0.0
Grossamount Strict F1: 0
Grossamount Ent Type Precision: 0.0000
Grossamount Ent Type Recall: 0.0043
Grossamount Ent Type F1: 0.0000
Grossamount Partial Precision: 0.0000
Grossamount Partial Recall: 0.0021
Grossamount Partial F1: 0.0000
Grossamount Exact Precision: 0.0
Grossamount Exact Recall: 0.0
Grossamount Exact F1: 0
Hoteladdress Strict Precision: 0.0005
Hoteladdress Strict Recall: 0.0057
Hoteladdress Strict F1: 0.0009
Hoteladdress Ent Type Precision: 0.0150
Hoteladdress Ent Type Recall: 0.1715
Hoteladdress Ent Type F1: 0.0276
Hoteladdress Partial Precision: 0.0115
Hoteladdress Partial Recall: 0.1312
Hoteladdress Partial F1: 0.0211
Hoteladdress Exact Precision: 0.0012
Hoteladdress Exact Recall: 0.0135
Hoteladdress Exact F1: 0.0022
Hotelzipcode Strict Precision: 0.0015
Hotelzipcode Strict Recall: 0.0615
Hotelzipcode Strict F1: 0.0029
Hotelzipcode Ent Type Precision: 0.0021
Hotelzipcode Ent Type Recall: 0.0893
Hotelzipcode Ent Type F1: 0.0042
Hotelzipcode Partial Precision: 0.0029
Hotelzipcode Partial Recall: 0.1211
Hotelzipcode Partial F1: 0.0056
Hotelzipcode Exact Precision: 0.0022
Hotelzipcode Exact Recall: 0.0940
Hotelzipcode Exact F1: 0.0044
Cardtype Strict Precision: 0.0
Cardtype Strict Recall: 0.0
Cardtype Strict F1: 0
Cardtype Ent Type Precision: 0.0
Cardtype Ent Type Recall: 0.0
Cardtype Ent Type F1: 0
Cardtype Partial Precision: 0.0
Cardtype Partial Recall: 0.0
Cardtype Partial F1: 0
Cardtype Exact Precision: 0.0
Cardtype Exact Recall: 0.0
Cardtype Exact F1: 0
Reservationnumber Strict Precision: 0.0
Reservationnumber Strict Recall: 0.0
Reservationnumber Strict F1: 0
Reservationnumber Ent Type Precision: 0.0
Reservationnumber Ent Type Recall: 0.0
Reservationnumber Ent Type F1: 0
Reservationnumber Partial Precision: 0.0
Reservationnumber Partial Recall: 0.0
Reservationnumber Partial F1: 0
Reservationnumber Exact Precision: 0.0
Reservationnumber Exact Recall: 0.0
Reservationnumber Exact F1: 0
Hotelemail Strict Precision: 0.0058
Hotelemail Strict Recall: 0.0545
Hotelemail Strict F1: 0.0104
Hotelemail Ent Type Precision: 0.0375
Hotelemail Ent Type Recall: 0.3545
Hotelemail Ent Type F1: 0.0678
Hotelemail Partial Precision: 0.0218
Hotelemail Partial Recall: 0.2062
Hotelemail Partial F1: 0.0394
Hotelemail Exact Precision: 0.0058
Hotelemail Exact Recall: 0.0546
Hotelemail Exact F1: 0.0104
Companyaddress Strict Precision: 0.0020
Companyaddress Strict Recall: 0.0161
Companyaddress Strict F1: 0.0035
Companyaddress Ent Type Precision: 0.0276
Companyaddress Ent Type Recall: 0.2248
Companyaddress Ent Type F1: 0.0491
Companyaddress Partial Precision: 0.0245
Companyaddress Partial Recall: 0.2002
Companyaddress Partial F1: 0.0437
Companyaddress Exact Precision: 0.0032
Companyaddress Exact Recall: 0.0259
Companyaddress Exact F1: 0.0057
Hotelcountry Strict Precision: 0.0
Hotelcountry Strict Recall: 0
Hotelcountry Strict F1: 0
Hotelcountry Ent Type Precision: 0.0
Hotelcountry Ent Type Recall: 0
Hotelcountry Ent Type F1: 0
Hotelcountry Partial Precision: 0.0
Hotelcountry Partial Recall: 0
Hotelcountry Partial F1: 0
Hotelcountry Exact Precision: 0.0
Hotelcountry Exact Recall: 0
Hotelcountry Exact F1: 0
Checkin Strict Precision: 0.0127
Checkin Strict Recall: 0.1420
Checkin Strict F1: 0.0234
Checkin Ent Type Precision: 0.0376
Checkin Ent Type Recall: 0.4189
Checkin Ent Type F1: 0.0689
Checkin Partial Precision: 0.0296
Checkin Partial Recall: 0.3299
Checkin Partial F1: 0.0543
Checkin Exact Precision: 0.0140
Checkin Exact Recall: 0.1560
Checkin Exact F1: 0.0257
Netamount Strict Precision: 0.0
Netamount Strict Recall: 0.0
Netamount Strict F1: 0
Netamount Ent Type Precision: 0.0000
Netamount Ent Type Recall: 0.0311
Netamount Ent Type F1: 0.0000
Netamount Partial Precision: 0.0000
Netamount Partial Recall: 0.0186
Netamount Partial F1: 0.0000
Netamount Exact Precision: 0.0
Netamount Exact Recall: 0.0
Netamount Exact F1: 0
Taxamount Strict Precision: 0.0
Taxamount Strict Recall: 0.0
Taxamount Strict F1: 0
Taxamount Ent Type Precision: 0.0
Taxamount Ent Type Recall: 0.0
Taxamount Ent Type F1: 0
Taxamount Partial Precision: 0.0
Taxamount Partial Recall: 0.0
Taxamount Partial F1: 0
Taxamount Exact Precision: 0.0
Taxamount Exact Recall: 0.0
Taxamount Exact F1: 0
Hotelname Strict Precision: 0.0001
Hotelname Strict Recall: 0.0018
Hotelname Strict F1: 0.0002
Hotelname Ent Type Precision: 0.0071
Hotelname Ent Type Recall: 0.1243
Hotelname Ent Type F1: 0.0134
Hotelname Partial Precision: 0.0046
Hotelname Partial Recall: 0.0796
Hotelname Partial F1: 0.0086
Hotelname Exact Precision: 0.0002
Hotelname Exact Recall: 0.0039
Hotelname Exact F1: 0.0004
Invoicenumber Strict Precision: 0.0040
Invoicenumber Strict Recall: 0.0856
Invoicenumber Strict F1: 0.0076
Invoicenumber Ent Type Precision: 0.0092
Invoicenumber Ent Type Recall: 0.1976
Invoicenumber Ent Type F1: 0.0176
Invoicenumber Partial Precision: 0.0104
Invoicenumber Partial Recall: 0.2222
Invoicenumber Partial F1: 0.0198
Invoicenumber Exact Precision: 0.0067
Invoicenumber Exact Recall: 0.1428
Invoicenumber Exact F1: 0.0127
Checkout Strict Precision: 0.0074
Checkout Strict Recall: 0.0766
Checkout Strict F1: 0.0135
Checkout Ent Type Precision: 0.0303
Checkout Ent Type Recall: 0.3126
Checkout Ent Type F1: 0.0552
Checkout Partial Precision: 0.0222
Checkout Partial Recall: 0.2288
Checkout Partial F1: 0.0404
Checkout Exact Precision: 0.0078
Checkout Exact Recall: 0.0802
Checkout Exact F1: 0.0142
Travellername Strict Precision: 0.0003
Travellername Strict Recall: 0.0073
Travellername Strict F1: 0.0005
Travellername Ent Type Precision: 0.0056
Travellername Ent Type Recall: 0.1581
Travellername Ent Type F1: 0.0109
Travellername Partial Precision: 0.0040
Travellername Partial Recall: 0.1124
Travellername Partial F1: 0.0077
Travellername Exact Precision: 0.0009
Travellername Exact Recall: 0.0245
Travellername Exact F1: 0.0017
Externalreservationnumber Strict Precision: 0.0
Externalreservationnumber Strict Recall: 0
Externalreservationnumber Strict F1: 0
Externalreservationnumber Ent Type Precision: 0.0
Externalreservationnumber Ent Type Recall: 0
Externalreservationnumber Ent Type F1: 0
Externalreservationnumber Partial Precision: 0.0
Externalreservationnumber Partial Recall: 0
Externalreservationnumber Partial F1: 0
Externalreservationnumber Exact Precision: 0.0
Externalreservationnumber Exact Recall: 0
Externalreservationnumber Exact F1: 0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 16
total_train_batch_size: 128
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.5
num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Overall Strict Precision	Overall Strict Recall	Overall Strict F1	Overall Ent Type Precision	Overall Ent Type Recall	Overall Ent Type F1	Overall Partial Precision	Overall Partial Recall	Overall Partial F1	Overall Exact Precision	Overall Exact Recall	Overall Exact F1	Bookingnumber Strict Recall	Bookingnumber Ent Type Precision	Bookingnumber Ent Type Recall	Bookingnumber Ent Type F1	Bookingnumber Partial Precision	Bookingnumber Partial Recall	Bookingnumber Partial F1	Bookingnumber Exact Precision	Bookingnumber Exact Recall	Bookingnumber Exact F1	Invoicedate Strict Precision	Invoicedate Strict Recall	Invoicedate Strict F1	Invoicedate Ent Type Precision	Invoicedate Ent Type Recall	Invoicedate Ent Type F1	Invoicedate Partial Precision	Invoicedate Partial Recall	Invoicedate Partial F1	Invoicedate Exact Precision	Invoicedate Exact Recall	Invoicedate Exact F1	Grossamount Strict Precision	Grossamount Strict Recall	Grossamount Strict F1	Grossamount Ent Type Precision	Grossamount Ent Type Recall	Grossamount Ent Type F1	Grossamount Partial Precision	Grossamount Partial Recall	Grossamount Partial F1	Grossamount Exact Precision	Grossamount Exact Recall	Grossamount Exact F1	Companycountry Strict Precision	Companycountry Strict Recall	Companycountry Strict F1	Companycountry Ent Type Precision	Companycountry Ent Type Recall	Companycountry Ent Type F1	Companycountry Partial Precision	Companycountry Partial Recall	Companycountry Partial F1	Companycountry Exact Precision	Companycountry Exact Recall	Companycountry Exact F1	Hoteladdress Strict Precision	Hoteladdress Strict Recall	Hoteladdress Strict F1	Hoteladdress Ent Type Precision	Hoteladdress Ent Type Recall	Hoteladdress Ent Type F1	Hoteladdress Partial Precision	Hoteladdress Partial Recall	Hoteladdress Partial F1	Hoteladdress Exact Precision	Hoteladdress Exact Recall	Hoteladdress Exact F1	Hotelfaxnumber Strict Precision	Hotelfaxnumber Strict Recall	Hotelfaxnumber Strict F1	Hotelfaxnumber Ent Type Precision	Hotelfaxnumber Ent Type Recall	Hotelfaxnumber Ent Type F1	Hotelfaxnumber Partial Precision	Hotelfaxnumber Partial Recall	Hotelfaxnumber Partial F1	Hotelfaxnumber Exact Precision	Hotelfaxnumber Exact Recall	Hotelfaxnumber Exact F1	Hotelvatnumber Strict Precision	Hotelvatnumber Strict Recall	Hotelvatnumber Strict F1	Hotelvatnumber Ent Type Precision	Hotelvatnumber Ent Type Recall	Hotelvatnumber Ent Type F1	Hotelvatnumber Partial Precision	Hotelvatnumber Partial Recall	Hotelvatnumber Partial F1	Hotelvatnumber Exact Precision	Hotelvatnumber Exact Recall	Hotelvatnumber Exact F1	Hotelphonenumber Strict Precision	Hotelphonenumber Strict Recall	Hotelphonenumber Strict F1	Hotelphonenumber Ent Type Precision	Hotelphonenumber Ent Type Recall	Hotelphonenumber Ent Type F1	Hotelphonenumber Partial Precision	Hotelphonenumber Partial Recall	Hotelphonenumber Partial F1	Hotelphonenumber Exact Precision	Hotelphonenumber Exact Recall	Hotelphonenumber Exact F1	Hotelzipcode Ent Type Precision	Hotelzipcode Ent Type Recall	Hotelzipcode Ent Type F1	Hotelzipcode Partial Precision	Hotelzipcode Partial Recall	Hotelzipcode Partial F1	Hotelzipcode Exact Precision	Hotelzipcode Exact Recall	Hotelzipcode Exact F1	Companyname Strict Precision	Companyname Strict Recall	Companyname Strict F1	Companyname Ent Type Precision	Companyname Ent Type Recall	Companyname Ent Type F1	Companyname Partial Precision	Companyname Partial Recall	Companyname Partial F1	Companyname Exact Precision	Companyname Exact Recall	Companyname Exact F1	Cardtype Strict Precision	Cardtype Strict Recall	Cardtype Strict F1	Cardtype Ent Type Precision	Cardtype Ent Type Recall	Cardtype Ent Type F1	Cardtype Partial Precision	Cardtype Partial Recall	Cardtype Partial F1	Cardtype Exact Precision	Cardtype Exact Recall	Cardtype Exact F1	Reservationnumber Strict Precision	Reservationnumber Strict Recall	Reservationnumber Strict F1	Reservationnumber Ent Type Precision	Reservationnumber Ent Type Recall	Reservationnumber Ent Type F1	Reservationnumber Partial Precision	Reservationnumber Partial Recall	Reservationnumber Partial F1	Reservationnumber Exact Precision	Reservationnumber Exact Recall	Reservationnumber Exact F1	Hotelbankaccount Strict Precision	Hotelbankaccount Strict Recall	Hotelbankaccount Strict F1	Hotelbankaccount Ent Type Precision	Hotelbankaccount Ent Type Recall	Hotelbankaccount Ent Type F1	Hotelbankaccount Partial Precision	Hotelbankaccount Partial Recall	Hotelbankaccount Partial F1	Hotelbankaccount Exact Precision	Hotelbankaccount Exact Recall	Hotelbankaccount Exact F1	Hotelemail Strict Precision	Hotelemail Strict Recall	Hotelemail Strict F1	Hotelemail Ent Type Precision	Hotelemail Ent Type Recall	Hotelemail Ent Type F1	Hotelemail Partial Precision	Hotelemail Partial Recall	Hotelemail Partial F1	Hotelemail Exact Precision	Hotelemail Exact Recall	Hotelemail Exact F1	Companyaddress Strict Precision	Companyaddress Strict Recall	Companyaddress Strict F1	Companyaddress Ent Type Precision	Companyaddress Ent Type Recall	Companyaddress Ent Type F1	Companyaddress Partial Precision	Companyaddress Partial Recall	Companyaddress Partial F1	Companyaddress Exact Precision	Companyaddress Exact Recall	Companyaddress Exact F1	Hotelcountry Strict Precision	Hotelcountry Strict Recall	Hotelcountry Strict F1	Hotelcountry Ent Type Precision	Hotelcountry Ent Type Recall	Hotelcountry Ent Type F1	Hotelcountry Partial Precision	Hotelcountry Partial Recall	Hotelcountry Partial F1	Hotelcountry Exact Precision	Hotelcountry Exact Recall	Hotelcountry Exact F1	Checkin Strict Precision	Checkin Strict Recall	Checkin Strict F1	Checkin Ent Type Precision	Checkin Ent Type Recall	Checkin Ent Type F1	Checkin Partial Precision	Checkin Partial Recall	Checkin Partial F1	Checkin Exact Precision	Checkin Exact Recall	Checkin Exact F1	Companyvatnumber Strict Recall	Companyvatnumber Ent Type Recall	Companyvatnumber Partial Precision	Companyvatnumber Partial Recall	Companyvatnumber Partial F1	Companyvatnumber Exact Precision	Companyvatnumber Exact Recall	Companyvatnumber Exact F1	Netamount Ent Type Precision	Netamount Ent Type Recall	Netamount Ent Type F1	Netamount Partial Precision	Netamount Partial Recall	Netamount Partial F1	Netamount Exact Precision	Netamount Exact Recall	Netamount Exact F1	Pagenumber Strict Precision	Pagenumber Strict Recall	Pagenumber Strict F1	Pagenumber Ent Type Precision	Pagenumber Ent Type Recall	Pagenumber Ent Type F1	Pagenumber Partial Precision	Pagenumber Partial Recall	Pagenumber Partial F1	Pagenumber Exact Precision	Pagenumber Exact Recall	Pagenumber Exact F1	Taxamount Strict Precision	Taxamount Strict Recall	Taxamount Strict F1	Taxamount Ent Type Precision	Taxamount Ent Type Recall	Taxamount Ent Type F1	Taxamount Partial Precision	Taxamount Partial Recall	Taxamount Partial F1	Taxamount Exact Precision	Taxamount Exact Recall	Taxamount Exact F1	Hotelcity Partial Precision	Hotelcity Partial Recall	Hotelcity Partial F1	Hotelcity Exact Precision	Hotelcity Exact Recall	Hotelcity Exact F1	Hotelname Ent Type Precision	Hotelname Ent Type Recall	Hotelname Ent Type F1	Hotelname Partial Precision	Hotelname Partial Recall	Hotelname Partial F1	Hotelname Exact Precision	Hotelname Exact Recall	Hotelname Exact F1	Roomno Strict Precision	Roomno Strict Recall	Roomno Strict F1	Roomno Ent Type Precision	Roomno Ent Type Recall	Roomno Ent Type F1	Roomno Partial Precision	Roomno Partial Recall	Roomno Partial F1	Roomno Exact Precision	Roomno Exact Recall	Roomno Exact F1	Hoteltaxcode Strict Precision	Hoteltaxcode Strict Recall	Hoteltaxcode Strict F1	Hoteltaxcode Ent Type Precision	Hoteltaxcode Ent Type Recall	Hoteltaxcode Ent Type F1	Hoteltaxcode Partial Precision	Hoteltaxcode Partial Recall	Hoteltaxcode Partial F1	Hoteltaxcode Exact Precision	Hoteltaxcode Exact Recall	Hoteltaxcode Exact F1	Documenttype Strict Recall	Documenttype Ent Type Recall	Documenttype Partial Precision	Documenttype Partial Recall	Documenttype Partial F1	Documenttype Exact Precision	Documenttype Exact Recall	Documenttype Exact F1	Invoicenumber Strict Precision	Invoicenumber Strict Recall	Invoicenumber Strict F1	Invoicenumber Ent Type Precision	Invoicenumber Ent Type Recall	Invoicenumber Ent Type F1	Invoicenumber Partial Precision	Invoicenumber Partial Recall	Invoicenumber Partial F1	Invoicenumber Exact Precision	Invoicenumber Exact Recall	Invoicenumber Exact F1	Companyzipcode Strict Recall	Companyzipcode Ent Type Precision	Companyzipcode Ent Type Recall	Companyzipcode Ent Type F1	Companyzipcode Partial Precision	Companyzipcode Partial Recall	Companyzipcode Partial F1	Companyzipcode Exact Precision	Companyzipcode Exact Recall	Companyzipcode Exact F1	Checkout Ent Type Precision	Checkout Ent Type Recall	Checkout Ent Type F1	Checkout Partial Precision	Checkout Partial Recall	Checkout Partial F1	Checkout Exact Precision	Checkout Exact Recall	Checkout Exact F1	Currencycode Strict Precision	Currencycode Strict Recall	Currencycode Strict F1	Currencycode Ent Type Precision	Currencycode Ent Type Recall	Currencycode Ent Type F1	Currencycode Partial Precision	Currencycode Partial Recall	Currencycode Partial F1	Currencycode Exact Precision	Currencycode Exact Recall	Currencycode Exact F1	Travellername Strict Precision	Travellername Strict Recall	Travellername Strict F1	Travellername Ent Type Precision	Travellername Ent Type Recall	Travellername Ent Type F1	Travellername Partial Precision	Travellername Partial Recall	Travellername Partial F1	Travellername Exact Precision	Travellername Exact Recall	Travellername Exact F1	Companycity Strict Precision	Companycity Strict Recall	Companycity Strict F1	Companycity Ent Type Precision	Companycity Ent Type Recall	Companycity Ent Type F1	Companycity Partial Precision	Companycity Partial Recall	Companycity Partial F1	Companycity Exact Precision	Companycity Exact Recall	Companycity Exact F1	Cardnumber Strict Precision	Cardnumber Strict Recall	Cardnumber Strict F1	Cardnumber Ent Type Precision	Cardnumber Ent Type Recall	Cardnumber Ent Type F1	Cardnumber Partial Precision	Cardnumber Partial Recall	Cardnumber Partial F1	Cardnumber Exact Precision	Cardnumber Exact Recall	Cardnumber Exact F1	Externalreservationnumber Strict Recall	Externalreservationnumber Ent Type Recall	Externalreservationnumber Partial Recall	Externalreservationnumber Exact Recall
67.6732	1.0	574	1.6453	0.0000	0.0001	0.0000	0.0000	0.0001	0.0000	0.0005	0.0016	0.0008	0.0004	0.0012	0.0006	0.0	0.0	0.0	0	0.0000	0.0004	0.0000	0.0	0.0	0	0.0	0.0	0	0.0000	0.0001	0.0000	0.0000	0.0003	0.0000	0.0000	0.0002	0.0000	0.0	0.0	0	0.0	0.0	0	0.0000	0.0018	0.0001	0.0000	0.0004	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0035	0.0000	0.0000	0.0035	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0001	0.0051	0.0001	0.0000	0.0014	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0008	0.0000	0.0000	0.0008	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0021	0.0000	0.0000	0.0021	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0023	0.0000	0.0000	0.0023	0.0000	0.0	0.0	0.0000	0.0017	0.0001	0.0000	0.0017	0.0001	0.0	0.0	0	0.0001	0.0455	0.0001	0.0001	0.0455	0.0001	0.0	0.0	0	0.0	0.0	0	0.0000	0.0011	0.0000	0.0000	0.0011	0.0000	0.0	0.0	0	0.0	0.0	0	0.0000	0.0047	0.0000	0.0000	0.0047	0.0000	0.0000	0.0055	0.0000	0.0000	0.0055	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0010	0.0000	0.0000	0.0010	0.0000	0.0000	0.0012	0.0000	0.0000	0.0012	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0.0000	0.0064	0.0000	0.0000	0.0064	0.0000	0.0	0.0	0	0.0	0.0	0	0.0000	0.0024	0.0000	0.0000	0.0020	0.0000	0.0	0.0	0.0	0	0.0000	0.0023	0.0000	0.0000	0.0023	0.0000	0.0	0.0	0	0.0000	0.0024	0.0001	0.0000	0.0006	0.0000	0.0	0.0	0	0.0	0.0	0	0.0001	0.0095	0.0001	0.0001	0.0093	0.0001	0.0	0.0	0	0.0	0.0	0	0.0000	0.0120	0.0001	0.0000	0.0120	0.0001	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0.0	0.0
21.9914	2.0	1148	1.0782	0.0004	0.0120	0.0009	0.0009	0.0241	0.0017	0.0019	0.0520	0.0037	0.0008	0.0211	0.0015	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0194	0.0000	0.0000	0.0146	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0003	0.0000	0.0003	0.0170	0.0005	0.0012	0.0801	0.0024	0.0003	0.0202	0.0006	0.0000	0.0003	0.0000	0.0002	0.0211	0.0003	0.0002	0.0257	0.0004	0.0000	0.0031	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0.0	0	0.0000	0.0008	0.0000	0.0000	0.0014	0.0000	0.0000	0.0003	0.0000	0.0000	0.0014	0.0000	0.0000	0.0057	0.0000	0.0000	0.0127	0.0000	0.0000	0.0014	0.0000	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0	0	0.0	0	0	0.0	0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.1	0.0000	0.0000	0.1	0.0000	0.0000	0.1	0.0000	0.0000	0.1	0.0000	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0000	0.0161	0.0000	0.0	0.0	0	0.0000	0.0114	0.0000	0.0000	0.0185	0.0001	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0	0	0.0	0	0	0.0	0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0000	0.0056	0.0000	0.0000	0.0028	0.0000	0.0	0.0	0	0.0004	0.6561	0.0009	0.0004	0.6561	0.0009	0.0004	0.6561	0.0009	0.0004	0.6561	0.0009	0.0	0.0	0	0.0	0.0	0	0.0000	0.2632	0.0000	0.0000	0.2632	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0	0	0	0
16.5132	3.0	1722	0.8859	0.0630	0.1089	0.0799	0.1137	0.1966	0.1441	0.1114	0.1925	0.1411	0.0661	0.1143	0.0838	0.0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0007	0.0124	0.0013	0.0070	0.1248	0.0133	0.0062	0.1109	0.0118	0.0010	0.0182	0.0019	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0004	0.0000	0.0014	0.0456	0.0027	0.0015	0.0491	0.0029	0.0003	0.0082	0.0005	0.0	0.0	0	0.0002	0.0245	0.0004	0.0005	0.0588	0.0010	0.0001	0.0124	0.0002	0.0001	0.0007	0.0001	0.0087	0.1036	0.0160	0.0105	0.1250	0.0193	0.0011	0.0130	0.0020	0.0012	0.0118	0.0022	0.0161	0.1568	0.0291	0.0155	0.1517	0.0282	0.0021	0.0209	0.0039	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0244	0.0000	0.0000	0.0244	0.0000	0.0000	0.0366	0.0000	0.0000	0.0244	0.0000	0.0	0.0	0	0.0	0.0	0	0.0000	0.0085	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0227	0.0000	0.0000	0.0227	0.0000	0.0	0.0	0	0.0	0.0	0	0.0000	0.0224	0.0000	0.0000	0.0149	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0675	0.5819	0.1210	0.0675	0.5819	0.1210	0.0676	0.5821	0.1211	0.0675	0.5821	0.1211	0.0	0.0	0	0.0000	0.0046	0.0000	0.0000	0.0414	0.0001	0.0000	0.0115	0.0000	0.0	0.0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0002	0.0000	0.0005	0.0321	0.0010	0.0008	0.0523	0.0017	0.0001	0.0075	0.0002	0.0	0.0	0	0.0	0.0	0	0.0000	0.0516	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0004	0.0001	0.0110	0.1547	0.0206	0.0058	0.0815	0.0108	0.0002	0.0029	0.0004	0.0001	0.0014	0.0001	0.0041	0.0806	0.0078	0.0051	0.1006	0.0097	0.0005	0.0105	0.0010	0	0	0.0	0	0	0.0	0	0	0.0001	0.0025	0.0003	0.0071	0.1248	0.0135	0.0066	0.1151	0.0124	0.0002	0.0030	0.0003	0.0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0003	0.0000	0.0012	0.0572	0.0024	0.0011	0.0497	0.0021	0.0000	0.0018	0.0001	0.0	0.0	0	0.0000	0.0078	0.0000	0.0001	0.0245	0.0001	0.0000	0.0142	0.0001	0.0000	0.0005	0.0000	0.0040	0.0825	0.0076	0.0047	0.0981	0.0090	0.0001	0.0014	0.0001	0.0	0.0	0	0.0000	0.0141	0.0000	0.0000	0.0386	0.0001	0.0000	0.0141	0.0000	0	0	0	0
13.7712	4.0	2296	0.7088	0.1125	0.1187	0.1155	0.2452	0.2586	0.2517	0.2183	0.2302	0.2241	0.1205	0.1271	0.1237	0.0	0.0000	0.0096	0.0000	0.0000	0.0144	0.0000	0.0	0.0	0	0.0060	0.0543	0.0108	0.0241	0.2178	0.0435	0.0219	0.1979	0.0395	0.0074	0.0670	0.0134	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0003	0.0028	0.0006	0.0228	0.1981	0.0408	0.0196	0.1709	0.0352	0.0007	0.0064	0.0013	0.0002	0.0052	0.0003	0.0042	0.1276	0.0081	0.0034	0.1028	0.0065	0.0005	0.0138	0.0009	0.0007	0.0098	0.0013	0.0080	0.1111	0.0149	0.0064	0.0894	0.0120	0.0012	0.0160	0.0021	0.0051	0.0389	0.0090	0.0277	0.2125	0.0490	0.0260	0.2000	0.0461	0.0078	0.0598	0.0138	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0002	0.0528	0.0004	0.0002	0.0649	0.0005	0.0001	0.0148	0.0001	0.0006	0.0933	0.0012	0.0020	0.3140	0.0039	0.0013	0.2055	0.0026	0.0006	0.0949	0.0012	0.0002	0.0214	0.0003	0.0006	0.0856	0.0012	0.0007	0.0991	0.0014	0.0002	0.0295	0.0004	0.0001	0.3364	0.0002	0.0001	0.3364	0.0002	0.0001	0.3458	0.0003	0.0001	0.3458	0.0003	0.0	0.0	0	0.0002	0.0229	0.0005	0.0002	0.0144	0.0003	0.0000	0.0023	0.0000	0.0004	0.0209	0.0008	0.0074	0.3635	0.0145	0.0041	0.1985	0.0079	0.0004	0.0212	0.0009	0.0013	0.0920	0.0026	0.0016	0.1121	0.0032	0.0019	0.1331	0.0038	0.0017	0.1178	0.0033	0.1201	0.5661	0.1982	0.1202	0.5664	0.1983	0.1204	0.5676	0.1987	0.1202	0.5663	0.1983	0.0003	0.0197	0.0006	0.0016	0.1063	0.0032	0.0019	0.1265	0.0038	0.0005	0.0299	0.0009	0.0	0.0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0002	0.0028	0.0003	0.0070	0.1098	0.0131	0.0070	0.1101	0.0131	0.0010	0.0162	0.0019	0.0003	0.0259	0.0006	0.0005	0.0395	0.0009	0.0010	0.0880	0.0021	0.0007	0.0594	0.0014	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0025	0.0227	0.0044	0.0308	0.2849	0.0557	0.0169	0.1562	0.0305	0.0025	0.0233	0.0045	0.0007	0.0058	0.0012	0.0171	0.1521	0.0307	0.0195	0.1739	0.0351	0.0021	0.0188	0.0038	0	0	0.0	0	0	0.0	0	0	0.0051	0.0583	0.0094	0.0288	0.3289	0.0529	0.0207	0.2372	0.0381	0.0057	0.0654	0.0105	0.0	0.0	0.0	0	0.0000	0.0102	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0010	0.0001	0.0046	0.1001	0.0087	0.0031	0.0670	0.0059	0.0001	0.0030	0.0003	0.0016	0.0476	0.0031	0.0040	0.1199	0.0078	0.0053	0.1572	0.0102	0.0031	0.0929	0.0060	0.0015	0.0163	0.0027	0.0197	0.2179	0.0362	0.0136	0.1504	0.0249	0.0016	0.0173	0.0029	0.0001	0.0033	0.0001	0.0021	0.0995	0.0042	0.0017	0.0798	0.0033	0.0003	0.0146	0.0006	0	0	0	0
11.2268	5.0	2870	0.6421	0.1414	0.1394	0.1404	0.3013	0.2970	0.2991	0.2607	0.2570	0.2589	0.1510	0.1489	0.1499	0.0	0.0000	0.0043	0.0000	0.0000	0.0470	0.0001	0.0000	0.0085	0.0000	0.0097	0.0820	0.0174	0.0301	0.2544	0.0539	0.0271	0.2293	0.0485	0.0114	0.0966	0.0205	0.0000	0.0132	0.0000	0.0001	0.1498	0.0003	0.0001	0.0815	0.0001	0.0000	0.0132	0.0000	0.0006	0.0054	0.0012	0.0285	0.2366	0.0509	0.0221	0.1835	0.0394	0.0010	0.0087	0.0019	0.0009	0.0231	0.0018	0.0075	0.1853	0.0144	0.0056	0.1379	0.0107	0.0012	0.0304	0.0024	0.0015	0.0198	0.0027	0.0118	0.1605	0.0220	0.0082	0.1109	0.0152	0.0017	0.0229	0.0031	0.0073	0.0564	0.0130	0.0324	0.2494	0.0573	0.0290	0.2233	0.0513	0.0101	0.0777	0.0179	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0009	0.1016	0.0017	0.0008	0.0956	0.0016	0.0001	0.0136	0.0002	0.0028	0.2170	0.0055	0.0053	0.4090	0.0104	0.0040	0.3147	0.0080	0.0028	0.2184	0.0055	0.0003	0.0189	0.0006	0.0019	0.1200	0.0037	0.0017	0.1098	0.0034	0.0005	0.0303	0.0009	0.0013	0.6574	0.0026	0.0013	0.6574	0.0026	0.0013	0.6723	0.0027	0.0013	0.6723	0.0027	0.0000	0.0016	0.0001	0.0010	0.0652	0.0020	0.0006	0.0363	0.0011	0.0000	0.0028	0.0001	0.0045	0.1442	0.0088	0.0131	0.4191	0.0254	0.0091	0.2923	0.0177	0.0046	0.1456	0.0088	0.0059	0.1689	0.0114	0.0070	0.2017	0.0136	0.0075	0.2160	0.0145	0.0068	0.1941	0.0131	0.1333	0.5685	0.2160	0.1335	0.5692	0.2162	0.1338	0.5706	0.2167	0.1334	0.5688	0.2161	0.0039	0.1072	0.0076	0.0103	0.2812	0.0198	0.0096	0.2625	0.0185	0.0044	0.1200	0.0085	0.0	0.0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0098	0.0000	0.0000	0.0049	0.0000	0.0	0.0	0	0.0004	0.0046	0.0007	0.0135	0.1597	0.0250	0.0107	0.1267	0.0198	0.0011	0.0132	0.0021	0.0012	0.0519	0.0023	0.0017	0.0756	0.0033	0.0025	0.1102	0.0049	0.0019	0.0845	0.0037	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0051	0.0477	0.0092	0.0364	0.3405	0.0657	0.0209	0.1958	0.0378	0.0051	0.0478	0.0092	0.0017	0.0134	0.0029	0.0260	0.2113	0.0464	0.0240	0.1952	0.0428	0.0029	0.0236	0.0052	0	0	0.0	0	0	0.0	0	0	0.0116	0.1310	0.0214	0.0361	0.4056	0.0662	0.0281	0.3165	0.0517	0.0129	0.1453	0.0237	0.0	0.0000	0.0373	0.0000	0.0000	0.0261	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0001	0.0016	0.0002	0.0069	0.1213	0.0130	0.0044	0.0779	0.0084	0.0002	0.0036	0.0004	0.0036	0.0797	0.0070	0.0085	0.1864	0.0163	0.0098	0.2142	0.0187	0.0062	0.1362	0.0119	0.0059	0.0623	0.0107	0.0283	0.3011	0.0517	0.0201	0.2137	0.0367	0.0061	0.0654	0.0112	0.0002	0.0059	0.0004	0.0051	0.1501	0.0098	0.0036	0.1065	0.0070	0.0007	0.0219	0.0014	0	0	0	0
10.0487	6.0	3444	0.6319	0.1459	0.1445	0.1452	0.3078	0.3049	0.3063	0.2651	0.2626	0.2639	0.1552	0.1537	0.1545	0.0	0.0000	0.0043	0.0000	0.0000	0.0413	0.0001	0.0000	0.0087	0.0000	0.0100	0.0865	0.0179	0.0300	0.2607	0.0539	0.0265	0.2301	0.0476	0.0114	0.0988	0.0204	0.0000	0.0090	0.0000	0.0002	0.1497	0.0004	0.0001	0.0793	0.0002	0.0000	0.0090	0.0000	0.0007	0.0063	0.0013	0.0287	0.2442	0.0514	0.0219	0.1865	0.0393	0.0012	0.0099	0.0021	0.0011	0.0271	0.0021	0.0080	0.1980	0.0155	0.0058	0.1438	0.0112	0.0014	0.0335	0.0026	0.0015	0.0217	0.0029	0.0121	0.1707	0.0227	0.0082	0.1156	0.0154	0.0018	0.0249	0.0033	0.0082	0.0636	0.0146	0.0334	0.2579	0.0591	0.0300	0.2318	0.0531	0.0110	0.0850	0.0195	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0010	0.1028	0.0019	0.0009	0.0984	0.0018	0.0001	0.0154	0.0003	0.0031	0.2321	0.0062	0.0055	0.4080	0.0109	0.0044	0.3220	0.0086	0.0032	0.2338	0.0062	0.0004	0.0266	0.0009	0.0020	0.1231	0.0040	0.0019	0.1114	0.0036	0.0006	0.0374	0.0012	0.0014	0.6958	0.0027	0.0014	0.6958	0.0027	0.0014	0.7091	0.0028	0.0014	0.7091	0.0028	0.0000	0.0016	0.0001	0.0011	0.0670	0.0021	0.0006	0.0373	0.0012	0.0000	0.0028	0.0001	0.0053	0.1613	0.0102	0.0138	0.4223	0.0268	0.0099	0.3036	0.0192	0.0053	0.1629	0.0103	0.0070	0.1876	0.0134	0.0082	0.2222	0.0159	0.0087	0.2356	0.0169	0.0079	0.2128	0.0152	0.1335	0.5672	0.2161	0.1336	0.5678	0.2163	0.1340	0.5692	0.2169	0.1335	0.5674	0.2162	0.0045	0.1123	0.0086	0.0119	0.2981	0.0228	0.0108	0.2726	0.0209	0.0050	0.1260	0.0096	0.0	0.0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0005	0.0058	0.0009	0.0147	0.1693	0.0270	0.0113	0.1307	0.0208	0.0012	0.0136	0.0022	0.0014	0.0605	0.0028	0.0021	0.0877	0.0040	0.0028	0.1196	0.0055	0.0022	0.0920	0.0042	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0058	0.0546	0.0104	0.0372	0.3524	0.0673	0.0217	0.2053	0.0392	0.0058	0.0547	0.0104	0.0019	0.0158	0.0034	0.0271	0.2251	0.0484	0.0241	0.2003	0.0431	0.0031	0.0256	0.0055	0	0	0.0	0	0	0.0	0	0	0.0125	0.1407	0.0229	0.0371	0.4194	0.0683	0.0291	0.3287	0.0535	0.0137	0.1548	0.0252	0.0	0.0000	0.0280	0.0000	0.0000	0.0175	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0001	0.0018	0.0002	0.0072	0.1244	0.0136	0.0046	0.0798	0.0087	0.0002	0.0037	0.0004	0.0038	0.0840	0.0073	0.0089	0.1947	0.0170	0.0100	0.2197	0.0192	0.0064	0.1414	0.0123	0.0073	0.0754	0.0132	0.0300	0.3118	0.0547	0.0219	0.2279	0.0400	0.0076	0.0790	0.0139	0.0002	0.0074	0.0005	0.0052	0.1562	0.0101	0.0037	0.1092	0.0071	0.0008	0.0226	0.0015	0	0	0	0
9.7277	7.0	4018	0.6304	0.1467	0.1444	0.1455	0.3102	0.3053	0.3078	0.2669	0.2627	0.2648	0.1562	0.1538	0.1550	0.0	0.0000	0.0038	0.0000	0.0000	0.0418	0.0001	0.0000	0.0114	0.0000	0.0100	0.0861	0.0178	0.0302	0.2616	0.0542	0.0265	0.2295	0.0476	0.0114	0.0982	0.0204	0.0000	0.0108	0.0000	0.0002	0.1667	0.0005	0.0001	0.0887	0.0002	0.0000	0.0108	0.0000	0.0008	0.0069	0.0015	0.0289	0.2439	0.0517	0.0221	0.1861	0.0395	0.0012	0.0104	0.0022	0.0012	0.0278	0.0022	0.0083	0.1972	0.0159	0.0060	0.1441	0.0116	0.0015	0.0346	0.0028	0.0016	0.0224	0.0030	0.0125	0.1736	0.0232	0.0085	0.1179	0.0158	0.0018	0.0255	0.0034	0.0082	0.0631	0.0145	0.0335	0.2581	0.0593	0.0300	0.2306	0.0530	0.0109	0.0841	0.0193	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0010	0.1046	0.0020	0.0009	0.0948	0.0018	0.0001	0.0147	0.0003	0.0032	0.2287	0.0062	0.0056	0.4078	0.0111	0.0044	0.3202	0.0087	0.0032	0.2303	0.0063	0.0004	0.0256	0.0008	0.0022	0.1308	0.0043	0.0020	0.1161	0.0038	0.0006	0.0374	0.0012	0.0015	0.6884	0.0029	0.0015	0.6884	0.0029	0.0015	0.7025	0.0030	0.0015	0.7025	0.0030	0.0000	0.0014	0.0000	0.0011	0.0689	0.0022	0.0006	0.0383	0.0012	0.0000	0.0026	0.0001	0.0055	0.1651	0.0106	0.0142	0.4253	0.0274	0.0102	0.3061	0.0197	0.0055	0.1668	0.0107	0.0071	0.1883	0.0137	0.0084	0.2232	0.0162	0.0090	0.2377	0.0173	0.0081	0.2139	0.0155	0.1342	0.5667	0.2171	0.1344	0.5675	0.2174	0.1347	0.5689	0.2179	0.1343	0.5670	0.2172	0.0047	0.1157	0.0090	0.0121	0.2993	0.0232	0.0110	0.2738	0.0212	0.0052	0.1298	0.0101	0.0	0.0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0084	0.0000	0.0000	0.0042	0.0000	0.0	0.0	0	0.0004	0.0050	0.0008	0.0149	0.1700	0.0274	0.0114	0.1304	0.0210	0.0011	0.0130	0.0021	0.0015	0.0608	0.0029	0.0021	0.0876	0.0041	0.0029	0.1198	0.0056	0.0023	0.0933	0.0044	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0000	0.1	0.0000	0.0000	0.1	0.0000	0.0057	0.0542	0.0104	0.0374	0.3537	0.0676	0.0217	0.2055	0.0393	0.0057	0.0543	0.0104	0.0019	0.0159	0.0035	0.0275	0.2254	0.0490	0.0244	0.2000	0.0435	0.0031	0.0256	0.0056	0	0	0.0	0	0	0.0	0	0	0.0126	0.1413	0.0232	0.0376	0.4213	0.0691	0.0296	0.3315	0.0543	0.0139	0.1560	0.0256	0.0	0.0000	0.0311	0.0000	0.0000	0.0186	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0001	0.0015	0.0002	0.0071	0.1236	0.0135	0.0046	0.0792	0.0086	0.0002	0.0034	0.0004	0.0040	0.0857	0.0076	0.0093	0.1993	0.0177	0.0104	0.2240	0.0199	0.0066	0.1433	0.0127	0.0074	0.0769	0.0136	0.0303	0.3141	0.0553	0.0222	0.2299	0.0405	0.0078	0.0809	0.0143	0.0003	0.0073	0.0005	0.0056	0.1567	0.0108	0.0039	0.1102	0.0076	0.0008	0.0235	0.0016	0	0	0	0
9.7238	8.0	4592	0.6304	0.1472	0.1448	0.1460	0.3106	0.3054	0.3080	0.2674	0.2630	0.2651	0.1567	0.1541	0.1554	0.0	0.0000	0.0078	0.0000	0.0001	0.0547	0.0001	0.0000	0.0195	0.0000	0.0100	0.0868	0.0180	0.0302	0.2614	0.0542	0.0266	0.2298	0.0476	0.0114	0.0987	0.0205	0.0000	0.0134	0.0000	0.0002	0.1769	0.0005	0.0001	0.0952	0.0003	0.0000	0.0134	0.0000	0.0008	0.0065	0.0014	0.0290	0.2446	0.0518	0.0221	0.1862	0.0395	0.0012	0.0102	0.0022	0.0012	0.0279	0.0022	0.0083	0.1981	0.0159	0.0060	0.1437	0.0115	0.0014	0.0342	0.0027	0.0016	0.0219	0.0029	0.0124	0.1729	0.0232	0.0084	0.1167	0.0157	0.0018	0.0249	0.0033	0.0083	0.0639	0.0147	0.0336	0.2589	0.0595	0.0301	0.2318	0.0533	0.0110	0.0850	0.0195	0.0	0	0	0.0	0	0	0.0	0	0	0.0	0	0	0.0010	0.1043	0.0020	0.0009	0.0980	0.0018	0.0001	0.0150	0.0003	0.0032	0.2296	0.0063	0.0056	0.4061	0.0111	0.0044	0.3197	0.0087	0.0032	0.2312	0.0063	0.0005	0.0274	0.0009	0.0022	0.1272	0.0043	0.0020	0.1150	0.0039	0.0007	0.0399	0.0013	0.0015	0.6968	0.0029	0.0015	0.6968	0.0029	0.0015	0.7110	0.0030	0.0015	0.7110	0.0030	0.0000	0.0016	0.0001	0.0011	0.0685	0.0022	0.0006	0.0377	0.0012	0.0000	0.0023	0.0001	0.0055	0.1654	0.0107	0.0142	0.4232	0.0274	0.0102	0.3054	0.0198	0.0056	0.1670	0.0108	0.0072	0.1910	0.0139	0.0086	0.2263	0.0165	0.0091	0.2400	0.0175	0.0082	0.2166	0.0158	0.1345	0.5663	0.2174	0.1347	0.5671	0.2177	0.1350	0.5684	0.2182	0.1346	0.5666	0.2175	0.0047	0.1145	0.0090	0.0122	0.2989	0.0234	0.0112	0.2741	0.0215	0.0052	0.1285	0.0101	0.0	0.0	0.0	0.0	0	0.0	0.0	0	0.0000	0.0043	0.0000	0.0000	0.0021	0.0000	0.0	0.0	0	0.0005	0.0057	0.0009	0.0150	0.1715	0.0276	0.0115	0.1312	0.0211	0.0012	0.0135	0.0022	0.0015	0.0615	0.0029	0.0021	0.0893	0.0042	0.0029	0.1211	0.0056	0.0022	0.0940	0.0044	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0058	0.0545	0.0104	0.0375	0.3545	0.0678	0.0218	0.2062	0.0394	0.0058	0.0546	0.0104	0.0020	0.0161	0.0035	0.0276	0.2248	0.0491	0.0245	0.2002	0.0437	0.0032	0.0259	0.0057	0	0	0.0	0	0	0.0	0	0	0.0127	0.1420	0.0234	0.0376	0.4189	0.0689	0.0296	0.3299	0.0543	0.0140	0.1560	0.0257	0.0	0.0000	0.0311	0.0000	0.0000	0.0186	0.0000	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0	0.0	0	0.0001	0.0018	0.0002	0.0071	0.1243	0.0134	0.0046	0.0796	0.0086	0.0002	0.0039	0.0004	0.0040	0.0856	0.0076	0.0092	0.1976	0.0176	0.0104	0.2222	0.0198	0.0067	0.1428	0.0127	0.0074	0.0766	0.0135	0.0303	0.3126	0.0552	0.0222	0.2288	0.0404	0.0078	0.0802	0.0142	0.0003	0.0073	0.0005	0.0056	0.1581	0.0109	0.0040	0.1124	0.0077	0.0009	0.0245	0.0017	0	0	0	0

Framework versions

Transformers 4.48.0.dev0
Pytorch 2.3.1
Datasets 3.2.0
Tokenizers 0.21.0