

TITLE
Run_ner.py falsely aligns prediction list

COMMENTS
9

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Environment info
- `transformers` version: 3.5.1
- Platform: Linux-5.9.1-kd-cluster-x86_64-with-glibc2.10
- Python version: 3.8.0
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik 
@stefan-it 

## Information

I am using the bert-base-german-cased model, which I trained on an NER task to predict entity labels.
When I run prediction on the test dataset, BERT tells me that the maximum sequence length (300) is exhausted and so no predictions will be made on a number of items.
However, this seems to be a wrong error report. The problem appears to be that in the align_predictions function, the preds_list variable gets a wrong dimension - the predictions are made correctly, but the index is shifted, so that they point to the wrong words. 
An example of what I mean: 
```
Amtsgericht Ort
Leipzig Ort
Abteilung O
für O
Strafsachen O
```
becomes
```
Amtsgericht O
Leipzig O
Abteilung O
für Ort
Strafsachen Ort
```
in the preds_list.
The write_predictions_to_file function then gets tangled up by this shifted index and (I think falsely) declares a sequence length error.

Strangely, another test document works just fine, without any difference between them that I could find. No sequence length error and no false indices there.

The problem arises when using:
* [] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

I'm afraid it isn't so easy to reproduce because it requires the trained model and the data.

The align_predictions function:
```
    def align_predictions(predictions: np.ndarray, label_ids: np.ndarray) -> Tuple[List[int], List[int]]:
        preds = np.argmax(predictions, axis=2)

        batch_size, seq_len = preds.shape

        out_label_list = [[] for _ in range(batch_size)]
        preds_list = [[] for _ in range(batch_size)]

        for i in range(batch_size):
            for j in range(seq_len):
                if label_ids[i, j] != nn.CrossEntropyLoss().ignore_index:
                    out_label_list[i].append(label_map[label_ids[i][j]])
                    preds_list[i].append(label_map[preds[i][j]])

        return preds_list, out_label_list
```

Console output:
```
01/21/2021 11:07:01 - INFO - filelock -   Lock 140422091422400 released on /home/IAIS/tschmude/bert_remote/examples/token-classification/Data_processing_scripts/CrossVal_Files/Rotation/Test_file_swap/cached_test_BertTokenizer_340.lock
/home/IAIS/tschmude/anaconda3/envs/bert_env_remote/lib/python3.8/site-packages/torch/nn/parallel/_functions.py:64: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '
  0%|                                                     | 0/1 [00:00<?, ?it/s]/home/IAIS/tschmude/anaconda3/envs/bert_env_remote/lib/python3.8/site-packages/seqeval/metrics/v1.py:57: UndefinedMetricWarning: Recall and F-score are ill-defined and being set to 0.0 in labels with no true samples. Use `zero_division` parameter to control this behavior.
  _warn_prf(average, modifier, msg_start, len(result))
01/21/2021 11:07:30 - INFO - run_ner -   
                                    precision    recall  f1-score   support
                             Datum       1.00      1.00      1.00         1
                    Gestaendnis_ja       1.00      1.00      1.00         1
                               Ort       0.50      1.00      0.67         1
                    Schadensbetrag       1.00      1.00      1.00         1
Strafe_Gesamtfreiheitsstrafe_Dauer       1.00      1.00      1.00         1
                 Strafe_Tatbestand       0.00      0.00      0.00         0
             Taeter_Drogenbezug_ja       1.00      1.00      1.00         1
                         micro avg       0.55      1.00      0.71         6
                         macro avg       0.79      0.86      0.81         6
                      weighted avg       0.92      1.00      0.94         6
/tmp/pycharm_project_44/src/transformers/trainer.py:1174: FutureWarning: This method is deprecated, use `Trainer.is_world_process_zero()` instead.
  warnings.warn("This method is deprecated, use `Trainer.is_world_process_zero()` instead.", FutureWarning)
01/21/2021 11:08:52 - WARNING - tasks -   Maximum sequence length exceeded: No prediction for 'Angeklagte'.
01/21/2021 11:08:52 - WARNING - tasks -   Maximum sequence length exceeded: No prediction for 'trägt'.
01/21/2021 11:08:52 - WARNING - tasks -   Maximum sequence length exceeded: No prediction for 'die'.
01/21/2021 11:08:52 - WARNING - tasks -   Maximum sequence length exceeded: No prediction for 'Kosten'.
01/21/2021 11:08:52 - WARNING - tasks -   Maximum sequence length exceeded: No prediction for 'des'.
01/21/2021 11:08:52 - WARNING - tasks -   Maximum sequence length exceeded: No prediction for 'Verfahrens'.
(and a lot more of these)
```
Config that the model was trained on:
```
Training Arguments:
	Data directory: ...ng_scripts/CrossVal_Files/Rotation/Train_file_swap
	Model: ...bert-base-german-cased
	Epochs: 8
	Seq length: 300
	Learning rate: 5e-05
	Batch size: 16
	Seed: 105
	Do Train: True
	Do Eval: True
	Do Test: True
```

## Expected behavior

That the predictions for the documents are correctly listed and written to file without a sequence length error or shifted indices.

Thank you for your help!