Learning from Mistakes via Interactive Study Assistant for Large Language Models

Large language models (LLMs) have shown promising capabilities to refine their generation based on feedback. However, LLM refinement based on feedback is not always robust and may produce incorrect answers. In this paper, we propose Large LAnguage Model (SALAM) to learn and correct from their mistakes. Our method introduces a study assistant agent to analyze mistakes and generate improvement guidelines from the main LLM. During inference, it identifies common misunderstandings based on the mistake collections and provides guidelines for LLMs to help them avoid similar mistakes. We further finetune the study assistant using imitation learning with successful feedback interaction. Our experiments on two challenging frameworks (BBH and BBQ) demonstrate that SALAM outperforms baselines by a margin of up to 10.7 in accuracy.