

TITLE
DeBerta unable to load from cache

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
### System Info

```shell
Transformers=4.18.0
Pytorch=1.3.0a0+ee77ccb
Ipython=6.2.1
Python=3.6.9
the transformers-cli is unfortunately not accurate--I am on a networked file system with a base conda environment containing torch on other large libraries installed on every machine and transformers installed in a virtualenv in the filesystem.
```


### Who can help?

@LysandreJik 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
In [1]: import transformers
In [2]: from transformers import AutoConfig, AutoModel
In [3]: config = AutoConfig.from_pretrained('microsoft/deberta-base', cache_dir='path-to-uploaded-transformers-cache')
In [6]: model = AutoModel.from_pretrained('microsoft/deberta-base', config=config, cache_dir='path-to-uploaded-transformers-cache')
ValueError                                Traceback (most recent call last)
/state/partition1/llgrid/pkg/anaconda/anaconda3-2020a/lib/python3.6/tarfile.py in nti(s)
    188             s = nts(s, "ascii", "strict")
--> 189             n = int(s.strip() or "0", 8)
    190         except ValueError:

ValueError: invalid literal for int() with base 8: 'v2\nq\x02((X'
```

### Expected behavior
I have a shared filesystem where I would like to use the Deberta model
The filesystem does not allow file lock because of how the network is setup, so I download all models locally and upload them. 
I have downloaded the Deberta model to a cache dir. Output looks like this:

```shell
me@node:~$ ls transformers_cache/
05056f257c8d2b63ad16fd26f847c9ab9ee34e33cdfad926e132be824b237869.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b
05056f257c8d2b63ad16fd26f847c9ab9ee34e33cdfad926e132be824b237869.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b.json
05056f257c8d2b63ad16fd26f847c9ab9ee34e33cdfad926e132be824b237869.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b.lock
c2bc27a1c7529c177696ff76b1e74cba8667be14e202359f20f9114e407f43e2.a39abb1c6179fb264c2db685f9a056b7cb8d4bc48d729888d292a2280debf8e2
c2bc27a1c7529c177696ff76b1e74cba8667be14e202359f20f9114e407f43e2.a39abb1c6179fb264c2db685f9a056b7cb8d4bc48d729888d292a2280debf8e2.json
c2bc27a1c7529c177696ff76b1e74cba8667be14e202359f20f9114e407f43e2.a39abb1c6179fb264c2db685f9a056b7cb8d4bc48d729888d292a2280debf8e2.lock
ce0ac094af27cf80bbf403595a6d47f1fc632981bf1d4c5bf69968568cbea410.e8ad27cc324bb0dc448d4d95f63e48f72688fb318a4c4c3f623485621b0b515c
ce0ac094af27cf80bbf403595a6d47f1fc632981bf1d4c5bf69968568cbea410.e8ad27cc324bb0dc448d4d95f63e48f72688fb318a4c4c3f623485621b0b515c.json
ce0ac094af27cf80bbf403595a6d47f1fc632981bf1d4c5bf69968568cbea410.e8ad27cc324bb0dc448d4d95f63e48f72688fb318a4c4c3f623485621b0b515c.lock
dataset-metadata.json
dde0725208c11536042f6f416c538792d44a2d57d1ae399bbd1bc5867e02c465.0a3ec262cb3d4f634c72ce55f2766bb88771e6499b2512830e2e63bf19dbf97a
dde0725208c11536042f6f416c538792d44a2d57d1ae399bbd1bc5867e02c465.0a3ec262cb3d4f634c72ce55f2766bb88771e6499b2512830e2e63bf19dbf97a.json
dde0725208c11536042f6f416c538792d44a2d57d1ae399bbd1bc5867e02c465.0a3ec262cb3d4f634c72ce55f2766bb88771e6499b2512830e2e63bf19dbf97a.lock
e313266bff73867debdfa78c78a9a4966d5e78281ac4ed7048c178b16a37eba7.fb501413b9cef9cef6babdc543bb4153cbec58d52bce077647efba3e3f14ccf3
e313266bff73867debdfa78c78a9a4966d5e78281ac4ed7048c178b16a37eba7.fb501413b9cef9cef6babdc543bb4153cbec58d52bce077647efba3e3f14ccf3.json
e313266bff73867debdfa78c78a9a4966d5e78281ac4ed7048c178b16a37eba7.fb501413b9cef9cef6babdc543bb4153cbec58d52bce077647efba3e3f14ccf3.lock

```
However when I try to load from the cache, I get the header error listed above.
This error is not reproduced on my local machine (MacOS)
```
