

TITLE
Error: isTensor() INTERNAL ASSERT FAILED from traced RoBERTa model on iOS using LibTorch

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
I've exported RoBERTa from a traced model for running on iOS using LibTorch and I'm getting this error when running prediction in the app: `isTensor() INTERNAL ASSERT FAILED at /Users/distiller/project/aten/src/ATen/core/ivalue_inl.h:86, please report a bug to PyTorch. Expected Tensor but got Tuple (toTensor at /Users/distiller/project/aten/src/ATen/core/ivalue_inl.h:86)`. I'm using the BertTokenizer because I have a small, fixed vocabulary (not natural language), and found It easier to use this vocabulary with the Bert tokenizer (happy to be corrected on this). I can train and test the model without issue in Python.
     
My conversion code is as follows (it's very possible I've done something wrong here!):

```
tokenizer = BertTokenizer('./data/vocab.txt')

config = RobertaConfig(
    vocab_size=858,
    max_position_embeddings=258,
    num_attention_heads=6,
    num_hidden_layers=4,
    type_vocab_size=1,
    torchscript=True
)

model = RobertaForMaskedLM(config=config).from_pretrained('./trained_RoBERTa')
model.cpu()
model.eval()

example_input = torch.LongTensor(1, 256).random_(0, 857).cpu()
traced_model = torch.jit.trace(model, example_input)
traced_model.save('./exports/trained_RoBERTa.pt')
```

Transformers version: 3.2.0
Ubuntu 18.04
Python 3.7.2
PyTorch 1.5
Cuda 10.2

I should mention that if there's a relatively painless path to using CoreML instead of LibTorch I'd love to hear about it.