

TITLE
Use Marian-MT to evaluate translated outputs by printing out per-word log-probility

COMMENTS
10

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# ❓ Questions & Help

I am going to evaluate the Marian's opus-mt to evaluate the translated output y given the input x. 
I want to get the model's log-probability of y given x, i.e., P(y|x). 
I didn't find any usage example in the document [here](https://huggingface.co/transformers/model_doc/marian.html#multilingual-models).
@sshleifer do you know an example in this case? Thanks!

For example:
Source input (x):  I would like to run this experiment.
Translated output (y) in Chinese: 我     希望   跑        这个    实验   。
The word-level log probability:     -0.3  -0.4   -0.23  -0.43  -0.23 -0.8

BTW, fairseq supports this function in their [comment-line tools](https://fairseq.readthedocs.io/en/latest/command_line_tools.html). 

