

TITLE
Fine-tuning LMwithNSP

COMMENTS
5

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
when i fine tuning bert using BertForPreTraining i got error here --> outputs = model(input_ids=input_ids, attention_mask=input_mask, token_type_ids=segment_ids,
                            labels=lm_label_ids, next_sentence_label=is_next)
and in this line also loss.backward()
i got this error RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle) and this error occur when shape of prediction not equal labels but i checked shape like this len(prediction_logits) == len(lm_label_ids) and same shape
what is the problem ?
