

TITLE
Dynamic padding did not work as expected for custom audio dataset

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Describe the bug

Hello everyone,  following-up of [this post](https://discuss.huggingface.co/t/dynamic-padding-not-working-for-audio-custom-dataset/14532) and [official blog post](https://huggingface.co/blog/fine-tune-wav2vec2-english) on fine-tuning Wav2Vec model. Turns out it did not pad correctly w.r.t. input features.

## Steps to reproduce the bug
```python

def map_speech_to_array(batch):
    """
    map the wav file to audio signals

    :param batch: the loaded dataset, with audio file location as "column"
    :type batch: datasets.dataset_dict.DatasetDict
    """
    speech_array, sampling_rate = sf.read(batch["audio_loc"])
    batch["speech"] = speech_array
    batch["sampling_rate"] = sampling_rate
    batch["audio_loc"] = batch["audio_loc"]
    batch["text"] = batch["text"]
    return batch


def prepare_dataset(batch):
    """
    data preprocess with Wav2Vec customized processor

    :param batch: the loaded dataset
    :type batch: datasets.dataset_dict.DatasetDict
    :param processor: the customized 
    :type processor: transformers.models.wav2vec2.processing_wav2vec2.Wav2Vec2Processor
    """
    batch["input_values"] = processor(batch["speech"], sampling_rate=batch["sampling_rate"]).input_values[0]
    with processor.as_target_processor():
        labels = processor(batch["text"]).input_ids
        batch["labels"] = labels
    return batch
loaded_dt = load_dataset(
    'csv',
    data_files={"train": "../manifest/train.csv",
                "test": "../manifest/test.csv"})

loaded_dt = loaded_dt.map(
    map_speech_to_array)
loaded_dt = loaded_dt.map(prepare_dataset)
train_dt = loaded_dt["train"]
```

I tried the following code for padding investigation 

```python
tokenizer = Wav2Vec2CTCTokenizer(
    "../vocab/vocab.json", unk_token="[UNK]", pad_token="[PAD]", word_delimiter_token="|",
    padding=True, truncation=True)
feature_extractor = Wav2Vec2FeatureExtractor(
    feature_size=1, sampling_rate=SAMPLE_RATE, padding_value=0.0,
    do_normalize=True, return_attention_mask=False,
    padding=True)
processor = Wav2Vec2Processor(feature_extractor=feature_extractor, tokenizer=tokenizer)
train_dt =train_dt.map(map_speech_to_array)
train_dt = train_dt.map(prepare_dataset)
input_features = []
label_features = []
for i, item in enumerate(train_dt):
  input_features.append({"input_values":train_dt[i]["input_values"]})
  label_features.append({"input_ids":train_dt[i]["labels"]})
print(len(label_features[0]["input_ids"]))
batch = processor.pad(
  input_features,
  padding=True,
  return_tensors="pt")
print(batch)
with processor.as_target_processor():
  labels_batch = processor.pad(
      label_features,
      padding=True,
      return_tensors="pt")
```

## Expected results

The code above is line-by-line breakdown of `DataCollatorCTCWithPadding` provided in the official blog. It should start to fine-tune Wav2Vec model.

## Actual results

```
 labels_batch = processor.pad(
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/transformers/models/wav2vec2/processing_wav2vec2.py", line 150, in pad
    return self.current_processor.pad(*args, **kwargs)
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2776, in pad
    padding_strategy, _, max_length, _ = self._get_padding_truncation_strategies(
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2312, in _get_padding_truncation_strategies
    if padding_strategy != PaddingStrategy.DO_NOT_PAD and (not self.pad_token or self.pad_token_id < 0):
TypeError: '<' not supported between instances of 'NoneType' and 'int'
```

I got a very similar error when I used `Trainer`

```python
model = Wav2Vec2ForCTC.from_pretrained(
    "facebook/wav2vec2-base", 
    ctc_loss_reduction="mean", 
    pad_token_id=processor.tokenizer.pad_token_id)
model.to(DEVICE)
model.freeze_feature_extractor()
data_collator = DataCollatorCTCWithPadding(
    processor=processor, padding=True)
training_args = TrainingArguments(
    output_dir="../fine-tuned/wav2vec",
    group_by_length=True,
    per_device_train_batch_size=32,
    evaluation_strategy="steps",
    num_train_epochs=30,
    fp16=True,
    gradient_checkpointing=True, 
    save_steps=500,
    eval_steps=500,
    logging_steps=500,
    learning_rate=1e-4,
    weight_decay=0.005,
    warmup_steps=1000,
    save_total_limit=2,
    push_to_hub=False)
trainer = Trainer(
    model=model,
    data_collator=data_collator,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=loaded_dt["train"],
    eval_dataset=loaded_dt["test"],
    tokenizer=processor.feature_extractor)
trainer.train()
```

Which returns:

```
trainer.train()
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/transformers/trainer.py", line 1306, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 561, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "wav2vec_test.py", line 73, in __call__
    labels_batch = self.processor.pad(
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/transformers/models/wav2vec2/processing_wav2vec2.py", line 150, in pad
    return self.current_processor.pad(*args, **kwargs)
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2776, in pad
    padding_strategy, _, max_length, _ = self._get_padding_truncation_strategies(
  File "/home/lixx3013/anaconda3/envs/toolkit/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2312, in _get_padding_truncation_strategies
    if padding_strategy != PaddingStrategy.DO_NOT_PAD and (not self.pad_token or self.pad_token_id < 0):
TypeError: '<' not supported between instances of 'NoneType' and 'int'
  0%|                   
```

## Environment info
- `datasets` version: 1.18.3
- Platform: Linux-4.15.0-166-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyArrow version: 6.0.1


Any suggestions? Thanks in advance.