

TITLE
[s2s] metrics.json is wrong on multigpu

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
overwritten by last rank to save it.

@nateraw is there a way to check if my module `is_rank_zero` or some such?