FACTS is a great paper from @GoogleDeepMind on measuring the factuality of LLM outputs. You can now download their prompt templates from @huggingface to improve LLM-based fact-checking yourself!
π The paper introduces the FACTS Grounding benchmark for evaluating the factuality of LLM outputs.
π€ Fact-checking is automated by an ensemble of LLM judges that verify if a response is fully grounded in a factual reference document.
π§ͺ The authors tested different prompt templates on held-out data to ensure their generalization.
π It's highly educational to read these templates to learn how frontier labs design prompts and understand their limitations.
πΎ You can now download and reuse these prompt templates via the prompt-templates library!
π The library simplifies sharing prompt templates on the HF hub or locally via standardized YAML files. Letβs make LLM work more transparent and reproducible by sharing more templates like this!