text: 
  
  TITLE
  a multitude of deprecations for pytorch-1.7+
  
  COMMENTS
  2
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  This is not urgent. There is a ton of deprecation warnings across many modules with pytorch-1.7+ and a few with python-3.8:
  (I hard-wrapped the lines to avoid the need to scroll, but it makes somewhat harder to see the warnings):
  ```
  src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
  src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
  src/transformers/modeling_deberta.py:18
  src/transformers/modeling_deberta.py:18:
  DeprecationWarning: Using or importing the ABCs from 'collections' instead of
  from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
  working from collections import Sequence
  
  tests/test_logging.py::HfArgumentParserTest::test_integration
    tests/test_logging.py:40:
    DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    logger.warn(msg)
  
  tests/test_logging.py::HfArgumentParserTest::test_integration
    tests/test_logging.py:48:
    DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    logger.warn(msg)
  
  tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_gpt2.py:164:
  TracerWarning: Converting a tensor to a Python float might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! w = w / (float(v.size(-1)) ** 0.5)
  
  tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_gpt2.py:169:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! mask = self.bias[:, :, ns - nd : ns, :ns]
  
  tests/test_modeling_auto.py::AutoModelTest::test_from_identifier_from_model_type
  tests/test_modeling_auto.py::AutoModelTest::test_from_pretrained_identifier
  src/transformers/modeling_auto.py:821:
  FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed
  in a future version. Please use `AutoModelForCausalLM` for causal language
  models, `AutoModelForMaskedLM` for masked language models and
  `AutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(
  
  tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_no_configs
  tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:432:
  UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape.
  This may consume a large amount of memory. warnings.warn(
  
  tests/test_modeling_albert.py::AlbertModelTest::test_torchscript
  tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_attentions
  tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_albert.py:229:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :seq_length]
  
  tests/test_modeling_albert.py: 3 warnings tests/test_modeling_bert.py: 3
  warnings tests/test_modeling_bert_generation.py: 3 warnings
  tests/test_modeling_distilbert.py: 2 warnings tests/test_modeling_dpr.py: 3
  warnings tests/test_modeling_flaubert.py: 3 warnings
  tests/test_modeling_electra.py: 3 warnings tests/test_modeling_layoutlm.py: 3
  warnings tests/test_modeling_roberta.py: 3 warnings tests/test_modeling_xlm.py:
  3 warnings tests/test_modeling_xlnet.py: 3 warnings
  src/transformers/modeling_utils.py:1670:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! input_tensor.shape == tensor_shape for input_tensor
  in input_tensors
  
  tests/test_modeling_bert_generation.py: 32 warnings
    src/transformers/modeling_bert_generation.py:417:
    DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    logger.warn("If you want to use `BertGenerationDecoder` as a standalone, add
    `is_decoder=True.`")
  
  tests/test_modeling_bert.py::BertModelTest::test_torchscript
  tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_attentions
  tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_dpr.py::DPRModelTest::test_torchscript
  tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_attentions
  tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bert.py:191:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :seq_length]
  
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bart.py:175:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! if decoder_padding_mask is not None and
  decoder_padding_mask.shape[1] > 1:
  
  tests/test_modeling_bart.py: 3 warnings tests/test_modeling_flaubert.py: 3
  warnings tests/test_modeling_fsmt.py: 3 warnings tests/test_modeling_roberta.py:
  3 warnings tests/test_modeling_xlm.py: 3 warnings
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/functional.py:1836:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert padding_idx < weight.size(0), 'Padding_idx
  must be within num_embeddings'
  
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bart.py:720:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert key_padding_mask is None or
  key_padding_mask.shape == (bsz, src_len)
  
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bart.py:722:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
  tgt_len, src_len)
  
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bart.py:740:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
  tgt_len, self.head_dim)
  
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bart.py:287:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! if torch.isinf(x).any() or torch.isnan(x).any():
  
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
  tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bart.py:1190:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! if len(torch.unique(eos_mask.sum(1))) > 1:
  
  tests/test_modeling_common.py::UtilsFunctionsTest::test_top_k_top_p_filtering
    tests/test_modeling_common.py:1196:
    UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
    one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
    internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
    non_inf_idx = (output != -float("inf")).nonzero().to(device=torch_device)
  
  tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript
  tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_attentions
  tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_hidden_state
  src/transformers/modeling_bert_generation.py:156:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :seq_length]
  
  tests/test_modeling_flaubert.py: 14 warnings tests/test_modeling_xlm.py: 14
  warnings
  src/transformers/modeling_xlm.py:1220:
  FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
  choice models. Please use the attention mask instead. warnings.warn(
  
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_flaubert_lm_head
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_model_outputs_equivalence
  tests/test_modeling_xlm.py::XLMModelTest::test_model_outputs_equivalence
  tests/test_modeling_xlm.py::XLMModelTest::test_xlm_lm_head
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/_reduction.py:14:
  UserWarning: reduction='elementwise_mean' is deprecated, please use
  reduction='mean' instead. warnings.warn("reduction='elementwise_mean' is
  deprecated, please use reduction='mean' instead.")
  
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_flaubert.py:188:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.size(0) == bs
  
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_flaubert.py:189:
  TracerWarning: Converting a tensor to a Python number might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.max().item() <= slen
  
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_flaubert.py:189:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.max().item() <= slen
  
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_xlm.py:95:
  TracerWarning: Converting a tensor to a Python number might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.max().item() <= slen
  
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_xlm.py:95:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.max().item() <= slen
  
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
  tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_xlm.py:106:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert mask.size() == (bs, slen)
  
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_fsmt.py:1224:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! if max_pos > self.weight.size(0):
  
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_fsmt.py:763:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert embed_dim == self.embed_dim
  
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_fsmt.py:764:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert list(query.size()) == [tgt_len, bsz,
  embed_dim]
  
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_fsmt.py:805:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
  tgt_len, src_len)
  
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_fsmt.py:814:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert key_padding_mask is None or
  key_padding_mask.size()[:2] == (
  
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
  tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_fsmt.py:833:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
  tgt_len, self.head_dim)
  
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_att_mask_past
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past_large_inputs
  src/transformers/modeling_gpt2.py:530:
  FutureWarning: The `past` argument is deprecated and will be removed in a future
  version, use `past_key_values` instead. warnings.warn(
  
  tests/test_modeling_electra.py::ElectraModelTest::test_torchscript
  tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_attentions
  tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_electra.py:180:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :seq_length]
  
  tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript
  tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_layoutlm.py:87:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :seq_length]
  
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/tensor.py:547:
  TracerWarning: torch.tensor results are registered as constants in the trace.
  You can safely ignore this warning if you use this function to create tensors
  out of constant variables that would be the same every time you call this
  function. In any other case, this might cause the trace to be incorrect. return
  torch.tensor(other, dtype=dtype, device=self.device) ** self
  
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_funnel.py:314:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! num_remove = shift * len(pooled_pos)
  
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_funnel.py:638:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! pooling_flag = pooling_flag and block_index > 0
  
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_funnel.py:481:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! shift = 2 if q_head.shape[1] != context_len else 1
  
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_funnel.py:431:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! positional_attn = positional_attn[..., :context_len]
  
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
  tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_funnel.py:678:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! output = output[:, : target_len - 1]
  
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
  tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_gpt2.py:1058:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! pooled_logits = logits[range(batch_size),
  sequence_lengths]
  
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_openai.py:467:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[None, :
  input_shape[-1]]
  
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_openai.py:180:
  TracerWarning: Converting a tensor to a Python float might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! w = w / math.sqrt(v.size(-1))
  
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_openai.py:183:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! b = self.bias[:, :, : w.size(-2), : w.size(-1)]
  
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
  tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_openai.py:823:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! pooled_logits = logits[range(batch_size),
  sequence_lengths]
  
  tests/test_modeling_rag.py: 12 warnings tests/test_retrieval_rag.py: 1 warning
  src/transformers/tokenization_utils_base.py:613:
  UserWarning: To copy construct from a tensor, it is recommended to use
  sourceTensor.clone().detach() or
  sourceTensor.clone().detach().requires_grad_(True), rather than
  torch.tensor(sourceTensor). tensor = as_tensor(value)
  
  tests/test_modeling_reformer.py: 58 warnings tests/test_modeling_transfo_xl.py:
  18 warnings
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/modules/container.py:434:
  UserWarning: Setting attributes on ParameterList is not supported.
  warnings.warn("Setting attributes on ParameterList is not supported.")
  
  tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
  tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
  tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_mobilebert.py:192:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :seq_length]
  
  tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
  tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
  tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_mobilebert.py:534:
  TracerWarning: torch.tensor results are registered as constants in the trace.
  You can safely ignore this warning if you use this function to create tensors
  out of constant variables that would be the same every time you call this
  function. In any other case, this might cause the trace to be incorrect.
  torch.tensor(1000),
  
  tests/test_modeling_reformer.py::ReformerLSHAttnModelTest::test_reformer_cached_inference
    src/transformers/modeling_reformer.py:899:
    UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
    one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
    internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
    relevant_bucket_idx = (bucket_idx == (bucket_idx.shape[-1] - 1)).nonzero()
  
  tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
  tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_attentions
  tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_utils.py:244:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! if causal_mask.shape[1] < attention_mask.shape[1]:
  
  tests/test_modeling_t5.py: 95 warnings
    /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/utils.py:760:
    DeprecationWarning: an integer is required (got type
    torch._C._onnx.TensorProtoDataType). Implicit conversion to integers using
    __int__ is deprecated, and may be removed in a future version of Python.
    return getattr(node, kind + "_")(name, value)
  
  tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
  tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
  tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:1638:
  DeprecationWarning: an integer is required (got type float). Implicit conversion
  to integers using __int__ is deprecated, and may be removed in a future version
  of Python. value_t=torch.tensor([fill_value],
  dtype=sym_help.scalar_type_to_pytorch_type[dtype]))
  
  tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_identifier_from_model_type
  tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_pretrained_identifier
  src/transformers/modeling_tf_auto.py:697:
  FutureWarning: The class `TFAutoModelWithLMHead` is deprecated and will be
  removed in a future version. Please use `TFAutoModelForCausalLM` for causal
  language models, `TFAutoModelForMaskedLM` for masked language models and
  `TFAutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(
  
  tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_attentions
  tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_squeezebert.py:78:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :seq_length]
  
  tests/test_modeling_tf_flaubert.py: 9 warnings tests/test_modeling_tf_xlm.py: 9
  warnings
  src/transformers/modeling_tf_xlm.py:994:
  FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
  choice models. Please use the attention mask instead. warnings.warn(
  
  tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_graph_mode
  tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_graph_mode
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:493:
  FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
  choice models. Please use the attention mask instead. return
  py_builtins.overload_of(f)(*args)
  
  tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_compile_tf_model
  tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_config
  tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_keras_save_load
  tests/test_modeling_xlnet.py::XLNetModelTest::test_config
  tests/test_modeling_xlnet.py::XLNetModelTest::test_correct_missing_keys
  tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_save_load
  tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_train_pipeline_custom_model
  tests/test_modeling_xlnet.py::XLNetModelTest::test_save_load
  src/transformers/configuration_xlnet.py:205:
  FutureWarning: This config doesn't use attention memories, a core feature of
  XLNet. Consider setting `mem_len` to a non-zero value, for example `xlnet =
  XLNetLMHeadModel.from_pretrained('xlnet-base-cased'', mem_len=1024)`, for
  accurate training performance as well as an order of magnitude faster inference.
  Starting from version 3.5.0, the default parameter will be 1024, following the
  implementation in https://arxiv.org/abs/1906.08237 warnings.warn(
  
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_xlm.py:531:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.size(0) == bs
  
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_xlm.py:532:
  TracerWarning: Converting a tensor to a Python number might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.max().item() <= slen
  
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_xlm.py:532:
  TracerWarning: Converting a tensor to a Python boolean might cause the trace to
  be incorrect. We can't record the data flow of Python values, so this value will
  be treated as a constant in the future. This means that the trace might not
  generalize to other inputs! assert lengths.max().item() <= slen
  
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
  tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
  src/transformers/modeling_xlm.py:546:
  TracerWarning: Converting a tensor to a Python index might cause the trace to be
  incorrect. We can't record the data flow of Python values, so this value will be
  treated as a constant in the future. This means that the trace might not
  generalize to other inputs! position_ids = self.position_ids[:, :slen]
  
  tests/test_optimization.py::OptimizationTest::test_adafactor
    src/transformers/optimization.py:512:
    UserWarning: This overload of add_ is deprecated: add_(Number alpha, Tensor
    other) Consider using one of the following signatures instead: add_(Tensor
    other, *, Number alpha) (Triggered internally at
    /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
    exp_avg_sq.mul_(beta2t).add_(1.0 - beta2t, update)
  
  tests/test_optimization.py::ScheduleInitTest::test_schedulers
    /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:247:
    UserWarning: To get the last learning rate computed by the scheduler, please
    use `get_last_lr()`. warnings.warn("To get the last learning rate computed by
    the scheduler, "
  
  tests/test_optimization.py::ScheduleInitTest::test_schedulers
    /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:131:
    UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`.
    In PyTorch 1.1.0 and later, you should call them in the opposite order:
    `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will
    result in PyTorch skipping the first value of the learning rate schedule. See
    more details at
    https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
    warnings.warn("Detected call of `lr_scheduler.step()` before
    `optimizer.step()`. "
  
  tests/test_optimization.py::ScheduleInitTest::test_schedulers
    /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:216:
    UserWarning: Please also save or load the state of the optimizer when saving
    or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)
  
  tests/test_optimization.py::ScheduleInitTest::test_schedulers
    /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:234:
    UserWarning: Please also save or load the state of the optimizer when saving
    or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)
  
  tests/test_tokenization_auto.py::AutoTokenizerTest::test_tokenizer_identifier_with_correct_config
  tests/test_tokenization_mbart.py::MBartEnroIntegrationTest::test_batch_fairseq_parity
  tests/test_tokenization_t5.py::T5TokenizationTest::test_empty_target_text
  tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
  tests/test_tokenization_t5.py::T5TokenizationTest::test_max_target_length
  tests/test_tokenization_t5.py::T5TokenizationTest::test_outputs_not_longer_than_maxlen
  tests/test_tokenization_t5.py::T5TokenizationTest::test_prepare_seq2seq_batch
  src/transformers/tokenization_utils_base.py:1421:
  FutureWarning: The `max_len` attribute has been deprecated and will be removed
  in a future version, use `model_max_length` instead. warnings.warn(
  
  tests/test_tokenization_albert.py: 2 warnings tests/test_tokenization_bart.py: 2
  warnings tests/test_tokenization_bert.py: 2 warnings
  tests/test_tokenization_bert_generation.py: 1 warning
  tests/test_tokenization_bertweet.py: 1 warning
  tests/test_tokenization_blenderbot.py: 1 warning
  tests/test_tokenization_ctrl.py: 1 warning tests/test_tokenization_camembert.py:
  2 warnings tests/test_tokenization_distilbert.py: 4 warnings
  tests/test_tokenization_dpr.py: 8 warnings tests/test_tokenization_fsmt.py: 1
  warning tests/test_tokenization_funnel.py: 2 warnings
  tests/test_tokenization_herbert.py: 2 warnings tests/test_tokenization_gpt2.py:
  1 warning tests/test_tokenization_layoutlm.py: 2 warnings
  tests/test_tokenization_marian.py: 1 warning tests/test_tokenization_lxmert.py:
  2 warnings tests/test_tokenization_mbart.py: 2 warnings
  tests/test_tokenization_pegasus.py: 2 warnings
  tests/test_tokenization_openai.py: 1 warning tests/test_tokenization_phobert.py:
  1 warning tests/test_tokenization_deberta.py: 1 warning
  tests/test_tokenization_prophetnet.py: 1 warning
  tests/test_tokenization_reformer.py: 1 warning
  tests/test_tokenization_squeezebert.py: 4 warnings
  tests/test_tokenization_t5.py: 2 warnings tests/test_tokenization_roberta.py: 2
  warnings tests/test_tokenization_transfo_xl.py: 1 warning
  tests/test_tokenization_xlm.py: 1 warning
  tests/test_tokenization_xlm_prophetnet.py: 1 warning
  tests/test_tokenization_xlnet.py: 2 warnings
  tests/test_tokenization_xlm_roberta.py: 2 warnings
  src/transformers/tokenization_utils_base.py:2025:
  FutureWarning: The `pad_to_max_length` argument is deprecated and will be
  removed in a future version, use `padding=True` or `padding='longest'` to pad to
  the longest sequence in the batch, or use `padding='max_length'` to pad to a max
  length. In this case, you can give a specific length with `max_length` (e.g.
  `max_length=45`) or leave max_length to None to pad to the maximal input size of
  the model (e.g. 512 for Bert). warnings.warn(
  
  tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
  tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_treatment
  src/transformers/tokenization_t5.py:183:
  UserWarning: This sequence already has </s>. In future versions this behavior
  may lead to duplicated eos tokens being added. warnings.warn(
  
  tests/test_trainer.py: 44 warnings
    /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/parallel/_functions.py:64:
    UserWarning: Was asked to gather along dimension 0, but all input tensors were
    scalars; will instead unsqueeze and return a vector. warnings.warn('Was asked
    to gather along dimension 0, but all '
  
  tests/test_trainer.py::TrainerIntegrationTest::test_can_resume_training
  tests/test_trainer_callback.py::TrainerCallbackTest::test_event_flow
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/cuda/nccl.py:48:
  DeprecationWarning: Using or importing the ABCs from 'collections' instead of
  from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
  working if not isinstance(inputs, collections.Container) or isinstance(inputs,
  torch.Tensor):
  
  -- Docs: https://docs.pytest.org/en/stable/warnings.html
  ``` 
  
  @LysandreJik 