text: 
  
  TITLE
  failure to use conda-forge apex  with torch1.6 and --amp_backend='apex' + --fp16_opt_level O1 
  
  COMMENTS
  2
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  ## Environment info
  <!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
       Don't forget to fill out the missing fields in that output! -->
       
  - `transformers` version: 3.2.0
  - Platform: Linux-4.4.0-1111-aws-x86_64-with-debian-stretch-sid
  - Python version: 3.6.8
  - PyTorch version (GPU?): 1.6.0 (True)
  - Tensorflow version (GPU?): 2.3.1 (True)
  - Using GPU in script?: YES
  - Using distributed or parallel set-up in script?: NO
  
  ### Who can help
  <!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
   If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
   Please tag fewer than 3 people.
   
   -->
  
   @stas00 
  
  ## Information
  
  Model I am using: BART
  
  The problem arises when using:
  
  torch 1.6 + conda-forge apex w/ --fp16 --amp_backend='apex' + --fp16_opt_level O1 to run finetune.py
  
  ```
    File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 429, in fit
      self.accelerator_backend.setup(model)
    File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 53, in setup
      model = self.trainer.precision_connector.connect(model)
    File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/trainer/connectors/precision_connector.py", line 78, in connect
      model, optimizers = self.backend.connect(model, self.trainer.optimizers)
    File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/plugins/apex.py", line 38, in connect
      self.trainer.reinit_scheduler_properties(optimizers, self.trainer.lr_schedulers)
    File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/pytorch_lightning/trainer/optimizers.py", line 143, in reinit_scheduler_properties
      scheduler.__class__.__mro__[idx].__init__(scheduler, optimizer)
    File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/torch/optim/lr_scheduler.py", line 74, in __init__
      self.optimizer.step = with_counter(self.optimizer.step)
    File "/home/ubuntu/anaconda3/envs/nightly/lib/python3.6/site-packages/torch/optim/lr_scheduler.py", line 56, in with_counter
      instance_ref = weakref.ref(method.__self__)
  AttributeError: 'function' object has no attribute '__self__'
  ```
  
  
  The tasks I am working on is:
  * [ ] Finetune BART on XSUM 
  
  ## To reproduce
  
  Steps to reproduce the behavior:
  It is the same issue mentioned in https://github.com/huggingface/transformers/issues/8403#issuecomment-724787083
  If you control find "\_\_self\_\_"
  
  
  <!-- If you have code snippets, error messages, stack traces please provide them here as well.
       Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
       Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
  
  ## Expected behavior
  
  <!-- A clear and concise description of what you would expect to happen. -->
  