text: 
  
  TITLE
  Transfomers Pipline: Batching does not work for Sentence-Pair Text Classification 
  
  COMMENTS
  4
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ### System Info
  
  ```shell
  - `transformers` version: 4.6.1
  - Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
  - Python version: 3.7.13
  - PyTorch version (GPU?): 1.11.0+cu113 (True)
  - Tensorflow version (GPU?): 2.8.0 (True)
  - Using GPU in script?: yes / no (Both)
  - Using distributed or parallel set-up in script?: No
  ```
  
  
  ### Who can help?
  
  @Narsil 
  
  ### Information
  
  - [ ] The official example scripts
  - [X] My own modified scripts
  
  ### Tasks
  
  - [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
  - [X] My own task or dataset (give details below)
  
  ### Reproduction
  
  Run: 
  ```python
  from transformers import pipeline
  model = pipeline(
     task="text-classification", 
     model="roberta-large-mnli", # does also happen for our own fine-tunes roberta models
     device=0 # does also happen on CPU
  )  
  n_samples = 10000
  sample = ['The earth is not flat.', 'Physicists will find it shocking, but there are plenty of people around the world who genuinely believe the Earth is flat...']
  data = [sample]* n_samples
  
  model([sample]) # works
  model(data, batch_size=1) # results in the follwing error
  ```
  Output
  ```
  Some weights of the model checkpoint at roberta-large-mnli were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.weight', 'roberta.pooler.dense.bias']
  - This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  - This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  
  ---------------------------------------------------------------------------
  
  RuntimeError                              Traceback (most recent call last)
  
  [<ipython-input-2-62b2d5a9a338>](https://localhost:8080/#) in <module>()
        6 
        7 model([sample])
  ----> 8 model(data, batch_size=1)
  
  14 frames
  
  [/usr/local/lib/python3.7/dist-packages/transformers/models/roberta/modeling_roberta.py](https://localhost:8080/#) in forward(self, hidden_states)
      347     def forward(self, hidden_states):
      348         hidden_states = self.dense(hidden_states)
  --> 349         hidden_states = self.intermediate_act_fn(hidden_states)
      350         return hidden_states
      351 
  
  RuntimeError: CUDA out of memory. Tried to allocate 5.34 GiB (GPU 0; 14.76 GiB total capacity; 9.35 GiB already allocated; 4.00 GiB free; 9.37 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
  ```
  
  ### Expected behavior
  
  ```shell
  The pipeline should run even with a very large list of inputs if the batch size is low enough.
  ```
  