Version 1
: Received: 25 September 2024 / Approved: 25 September 2024 / Online: 26 September 2024 (09:08:12 CEST)
How to cite:
Lv, B.; Feng, A.; Xie, C. Decoding by Factual Prompts and Hallucination Prompts Improves Factuality in Large Language Models. Preprints2024, 2024092037. https://doi.org/10.20944/preprints202409.2037.v1
Lv, B.; Feng, A.; Xie, C. Decoding by Factual Prompts and Hallucination Prompts Improves Factuality in Large Language Models. Preprints 2024, 2024092037. https://doi.org/10.20944/preprints202409.2037.v1
Lv, B.; Feng, A.; Xie, C. Decoding by Factual Prompts and Hallucination Prompts Improves Factuality in Large Language Models. Preprints2024, 2024092037. https://doi.org/10.20944/preprints202409.2037.v1
APA Style
Lv, B., Feng, A., & Xie, C. (2024). Decoding by Factual Prompts and Hallucination Prompts Improves Factuality in Large Language Models. Preprints. https://doi.org/10.20944/preprints202409.2037.v1
Chicago/Turabian Style
Lv, B., Ao Feng and Chenlong Xie. 2024 "Decoding by Factual Prompts and Hallucination Prompts Improves Factuality in Large Language Models" Preprints. https://doi.org/10.20944/preprints202409.2037.v1
Abstract
Although large language models demonstrate impressive capabilities, they sometimes generate irrelevant or nonsensical text, or produce outputs that deviate from the provided source input—an occurrence commonly referred to as hallucination. To mitigate this issue, we introduce a novel decoding method that incorporates both factual and hallucination prompts(DFHP), utilizing a contrastive output distribution to highlight the disparity in output probabilities between model predictions influenced by factual prompts and those affected by hallucination prompts. Experiments on both multiple-choice and text generation tasks show that our approach significantly enhances the factual accuracy of large language models without requiring additional training.
Keywords
large language models; hallucinations; prompt; contrastive decoding
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
