

TITLE
Calculating Confidence score for Question Answering Models

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
For QA task (extractive QA) the pipeline provides/returns 4 values. 
1. a probability score
2.  start index
3. end index
4. extracted answer
(https://huggingface.co/transformers/main_classes/pipelines.html#transformers.QuestionAnsweringPipeline)

But If I am using the model class directly   (not using pipeline) like the code below then i am unable to find the probability score :

`  
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForQuestionAnswering.from_pretrained(MODEL_NAME,return_dict=True)
_encoding = tokenizer.encode_plus(question, docText, return_tensors="pt",max_length=4096)
input_ids = encoding["input_ids"]
attention_mask = encoding["attention_mask"]

start_scores, end_scores = model(input_ids, attention_mask=attention_mask).values()
all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist())
answer_tokens = all_tokens[torch.argmax(start_scores) :torch.argmax(end_scores)+1]
answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))_
`
This answer object has only 3 values. extracted answer, start score and end score.I cant find any way or function to calculate the probability score. I want it because I want to sort multiple answers according to their probability/confidence score. 

This is a duplicate of Issue #5768 but that issue has been marked close without any answer.