Fuzzy Prediction Sets: Conformal Prediction with E-values

We make three contributions to conformal prediction. First, we propose fuzzy conformal prediction sets that offer a degree of exclusion, generalizing beyond the binary inclusion/exclusion offered by classical prediction sets. We connect fuzzy prediction sets to e-values to show this degree of exclusion is equivalent to an exclusion at different confidence levels, capturing precisely what e-values bring to conformal prediction. We show that a fuzzy prediction set is a predictive distribution with an arguably more appropriate error guarantee. Second, we derive optimal conformal prediction sets by interpreting the minimization of the expected measure of a prediction set as an optimal testing problem against a particular alternative. We use this to characterize exactly in what sense traditional conformal prediction is optimal, and show how this may generally be used to construct optimal (fuzzy) prediction sets. Third, we generalize the inheritance of guarantees by subsequent minimax decisions from prediction sets to fuzzy prediction sets. All results generalize beyond the conformal setting to prediction sets for arbitrary models. In particular, we find that constructing a (fuzzy) prediction set for a model is equivalent to constructing a test (e-value) for that model as a hypothesis.