arxiv:2406.19215

SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation

Published on Jun 27

· Submitted by

TranSirius on Jun 28

Upvote

Authors:

Zijun Yao ,

Liangming Pan ,

Shulin Cao ,

Juanzi Li

Abstract

This paper introduces Self-aware Knowledge Retrieval (SeaKR), a novel adaptive RAG model that extracts self-aware uncertainty of LLMs from their internal states. SeaKR activates retrieval when the LLMs present high self-aware uncertainty for generation. To effectively integrate retrieved knowledge snippets, SeaKR re-ranks them based on LLM's self-aware uncertainty to preserve the snippet that reduces their uncertainty to the utmost. To facilitate solving complex tasks that require multiple retrievals, SeaKR utilizes their self-aware uncertainty to choose among different reasoning strategies. Our experiments on both complex and simple Question Answering datasets show that SeaKR outperforms existing adaptive RAG methods. We release our code at https://github.com/THU-KEG/SeaKR.

View arXiv page View PDF Add to collection

Community

TranSirius

Paper author Paper submitter Jun 28

This paper propose to utilize the estimated uncertainty of LLMs from their internal state to guide retrieval-augmented generation---an advanced mode of RAG, which is referred to as adaptive RAG.

In particular, we consider two key research questions in adaptive RAG: (1) when to retrieve and (2) how to effectively integrate retrieved knowledge. We design SeaKR to address these two RQs.

For RQ1, SeaKR initiates retrieval only when the LLM is aware of that it is uncertain. For. RQ2, SeaKR designs two strategies: a) Self-aware re-ranking for multiple recalled knowledge snippets; and b) Self-aware reasoning to select the optimal solution among different reasoning strategies.