How to get your LLMs not to hallucinate
Day 172 / 366
I am a big skeptic of LLMs and Generative AI, but I do believe that they have some good use cases. For instance, it's great for summarization, translation, or rephrasing. Places where there is space for creativity, and the accuracy is not that big of a deal.
LLMs are horrible at being accurate, and the reason for that is they hallucinate. This means that they will confidently give you a wrong answer instead of admitting that they don’t have the answer. And so you can never 100% rely on the output you get from these models.
But if you still want to use them, here are some ways you can stop them from hallucinating
Using RAG
RAG means Retrieval-Augmented Generation. A big reason for hallucination is that LLMs are trained on the information from the web. And the info that you want your LLM to work with might not be that well represented in that training dataset. With RAG, you can provide additional info that the LLM can use to answer your question, thus reducing the chance of hallucination.
Writing better prompts
You can get really creative when writing prompts to stop your model from hallucinating. I have worked on projects where the prompts have included stuff like this
“.. Please make sure you answer accurately. My job depends on your output. If you make up a fake answer then I will surely get fired.”
Maybe AI does have a heart I guess?
Using a second LLM to validate
You could train a second lightweight LLM to double-check the responses of your main LLM. You could do a similar thing to validate the inputs going into your LLM if they are coming from your end users. This will help you prevent prompt injections, where malicious users will try to jailbreak your LLM and get it to do things it’s not supposed to.
At the end of the day, none of this will give you 100% accuracy. You need to remember that LLMs just like other models work off of probabilities. So they will be wrong every now and then. But you can put systems in place to make sure that happens as little as possible.