A counterfactual query is typically of the form 'For situation X, why was the outcome Y and not Z?'. A counterfactual explanation (or response to such a query) is of the form "If X was X*, then the outcome would have been Z rather than Y." In this work, we develop a technique to produce counterfactual visual explanations. Given a 'query' image $I$ for which a vision system predicts class $c$, a counterfactual visual explanation identifies how $I$ could change such that the system would output a different specified class $c'$. To do this, we select a 'distractor' image $I'$ that the system predicts as class $c'$ and identify spatial regions in $I$ and $I'$ such that replacing the identified region in $I$ with the identified region in $I'$ would push the system towards classifying $I$ as $c'$. We apply our approach to multiple image classification datasets generating qualitative results showcasing the interpretability and discriminativeness of our counterfactual explanations. To explore the effectiveness of our explanations in teaching humans, we present machine teaching experiments for the task of fine-grained bird classification. We find that users trained to distinguish bird species fare better when given access to counterfactual explanations in addition to training examples.
翻译:反事实查询通常以“ X ” 的形式进行, 为什么结果为 Y 而不是 Z? ” 。 反事实解释( 或对这种查询的答复) 的形式是 : “ 如果X 是 X *, 那么结果将是 Z 而不是 Y ” 。 在这项工作中, 我们开发了一种方法来产生反事实直观解释。 一种“ 要求” 图像的“ 美元 ”, 其视觉系统预测是 $ 的等级是 $, 一个反事实直观解释, 如何改变 $, 使系统产生不同的特定类别 $? 为了做到这一点, 我们选择了一个“ 吸引” 图像( 或对这种查询的答复) $ 。 我们选择了一个“ 吸引” 图像, 表示系统预测为 $ $ $, 并用 $ 和 $ 美元 来确定 空间区域 ” 。 在这个“ ” 方法下, 我们开发了一个系统, 将 将 $ 改为 $ $ c 的 。 我们的方法,, 将多重图像 分类 数据, 显示 质量 显示我们 反事实解释 解释 和 的 解释性 解释 。