text: 
  
  TITLE
  [s2s] test_finetune_trainer_slow fails when run in group
  
  COMMENTS
  3
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  On dual gpu when running `test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow` alone - all is good.
  
  when running it with all the other tests in that file it fails:
  
  ```
  RUN_SLOW=1 pytest -sv test_finetune_trainer.py
  [...]
  self = <seq2seq.test_finetune_trainer.TestFinetuneTrainer testMethod=test_finetune_trainer_slow>
  
      @slow
      def test_finetune_trainer_slow(self):
          # There is a missing call to __init__process_group somewhere
          output_dir = self.run_trainer(
              eval_steps=2, max_len="128", model_name=MARIAN_MODEL, num_train_epochs=10, distributed=False
          )
  
          # Check metrics
          logs = TrainerState.load_from_json(os.path.join(output_dir, "trainer_state.json")).log_history
          eval_metrics = [log for log in logs if "eval_loss" in log.keys()]
          first_step_stats = eval_metrics[0]
          last_step_stats = eval_metrics[-1]
  
  >       assert first_step_stats["eval_bleu"] < last_step_stats["eval_bleu"]  # model learned nothing
  E       AssertionError: assert 0.0 < 0.0
  
  test_finetune_trainer.py:130: AssertionError
  ----------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------
  WARNING  seq2seq.finetune_trainer:finetune_trainer.py:160 Process rank: -1, device: cuda:0, n_gpu: 2, distributed training: False, 16-bits training: False
  
  ==================================================================== short test summary info ====================================================================
  FAILED test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow - AssertionError: assert 0.0 < 0.0
  ================================================ 1 failed, 7 passed, 1 skipped, 17 warnings in 102.82s (0:01:42) ================================================
  ```
  
  For some reason it fails to learn anything when some other tests run before it. 
  
  tested with pytorch-nightly + py38.
  