text: 
  
  TITLE
  Using PaddingStrategy and TruncationStrategy throws an UnboundLocalError in tokenizers
  
  COMMENTS
  2
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
  <!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
       Don't forget to fill out the missing fields in that output! -->
       
  - `transformers` version: 3.3.1
  - Platform: Darwin-19.6.0-x86_64-i386-64bit
  - Python version: 3.7.7
  - PyTorch version (GPU?): 1.6.0 (False)
  - Tensorflow version (GPU?): not installed (NA)
  - Using GPU in script?: <fill in>
  - Using distributed or parallel set-up in script?: <fill in>
  
  ### Who can help
  @mfuntowicz I am almost sure I am using it right. But after looking at the code I found that there are two variables that are being accessed before assignment. Thanks in advance.
  
  ## Information
  
  This is exactly what I am doing:
  
  I am trying to load a tokenizer using `AutoTokenizer` and encode a single string. I am using a pretrained tokenizer `distilbert-base-uncased`
  
  ## To reproduce
  
  Steps to reproduce the behavior:
  
  This is exactly what I am running:
  
  ```
  from transformers import AutoTokenizer
  from transformers.tokenization_utils_base import PaddingStrategy
  
  tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased", use_fast=True)
  encoded_sentence = tokenizer.encode_plus(
          "some input text",
          return_attention_mask=True,
          padding=PaddingStrategy.MAX_LENGTH,
          add_special_tokens=True,
          max_length=20,
          return_token_type_ids=True
      )
  print(encoded_sentence)
  ```
  
  This throws the following error:
  
  ```
  Traceback (most recent call last):
    File "......", line 22, in <module>
      return_token_type_ids=True
    File ".../.venv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2029, in encode_plus
      **kwargs,
    File "..../.venv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1837, in _get_padding_truncation_strategies
      if padding_strategy != PaddingStrategy.DO_NOT_PAD and (not self.pad_token or self.pad_token_id < 0):
  UnboundLocalError: local variable 'padding_strategy' referenced before assignment
  ```
  I have padded some of the paths to hide my local directories.
  
  The same thing happens for truncation too. E.g.,
  
  ```
  from transformers import AutoTokenizer
  from transformers.tokenization_utils_base import TruncationStrategy
  tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased", use_fast=True)
  encoded_sentence = tokenizer.encode_plus(
          "some input text",
          return_attention_mask=True,
          add_special_tokens=True,
          max_length=20,
          return_token_type_ids=True,
          truncation=TruncationStrategy.LONGEST_FIRST
      )
  print(encoded_sentence)
  ```
  
  This raises the following error:
  
  ```
  Traceback (most recent call last):
    File ".../scratch.py", line 22, in <module>
      truncation=TruncationStrategy.LONGEST_FIRST
    File ".../.venv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2029, in encode_plus
      **kwargs,
    File ".../.venv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1846, in _get_padding_truncation_strategies
      truncation_strategy != TruncationStrategy.DO_NOT_TRUNCATE
  UnboundLocalError: local variable 'truncation_strategy' referenced before assignment
  ```
  
  ## Expected behavior
  
  This should not throw an exception like this. I looked at the code as well. I kind of know what is the actual issue. 
  