title = "

🐑 Plausibility Evaluation of Context Reliance (PECoRe) 🐑

" subtitle = "

An Interpretability Framework to Detect and Attribute Context Reliance in Language Models

" description = """ Given a query and a context passed as inputs to a LM, PECoRe will identify which tokens in the generated response were dependant on context, and match them with context tokens contributing to their prediction. For more information, check out our ICLR 2024 paper. """ how_it_works = r"""

⚙️ How Does It Work?
PECoRe uses a contrastive approach to attribute context reliance in language models. It compares the model's predictions when the context is present and when it is absent, and attributes the difference in predictions to the context tokens.
""" how_to_use = r"""

🔧 How to Use PECoRe
""" citation = r"""

📚 Citing PECoRe

To refer to the PECoRe framework for context usage detection, cite:


@inproceedings{sarti-etal-2023-quantifying,
    title = "Quantifying the Plausibility of Context Reliance in Neural Machine Translation",
    author = "Sarti, Gabriele and
        Chrupa{\l}a, Grzegorz and
        Nissim, Malvina and
        Bisazza, Arianna",
    booktitle = "The Twelfth International Conference on Learning Representations (ICLR 2024)",
    month = may,
    year = "2024",
    address = "Vienna, Austria",
    publisher = "OpenReview",
    url = "https://openreview.net/forum?id=XTHfNGI3zT"
}
If you use the Inseq implementation of PECoRe (inseq attribute-context), please also cite: 

@inproceedings{sarti-etal-2023-inseq,
    title = "Inseq: An Interpretability Toolkit for Sequence Generation Models",
    author = "Sarti, Gabriele  and
    Feldhus, Nils  and
    Sickert, Ludwig  and
    van der Wal, Oskar and
    Nissim, Malvina and
    Bisazza, Arianna",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-demo.40",
    pages = "421--435",
}
""" examples = [ [ "When was Banff National Park established?", "Banff National Park is Canada's oldest national park, established in 1885 as Rocky Mountains Park. Located in Alberta's Rocky Mountains, 110-180 kilometres (68-112 mi) west of Calgary, Banff encompasses 6,641 square kilometres (2,564 sq mi) of mountainous terrain.", ] ]