A decision tree recursively splits a feature space $\mathbb{R}^{d}$ and then assigns class labels based on the resulting partition. Decision trees have been part of the basic machine-learning toolkit for decades. A large body of work treats heuristic algorithms to compute a decision tree from training data, usually aiming to minimize in particular the size of the resulting tree. In contrast, little is known about the complexity of the underlying computational problem of computing a minimum-size tree for the given training data. We study this problem with respect to the number $d$ of dimensions of the feature space. We show that it can be solved in $O(n^{2d + 1}d)$ time, but under reasonable complexity-theoretic assumptions it is not possible to achieve $f(d) \cdot n^{o(d / \log d)}$ running time, where $n$ is the number of training examples. The problem is solvable in $(dR)^{O(dR)} \cdot n^{1+o(1)}$ time, if there are exactly two classes and $R$ is an upper bound on the number of tree leaves labeled with the first~class.
翻译:决定树依次分割一个特性空间 $\ mathbb{R ⁇ d} $, 然后根据由此产生的分区分配类标签。 决定树几十年来一直是基本机器学习工具包的一部分。 大量的工作处理从培训数据中计算决定树, 通常是为了尽可能缩小结果树的大小。 相反, 对计算给定培训数据最小尺寸树的基本计算问题的复杂性知之甚少。 我们研究关于特性空间大小的美元数问题。 我们显示, 这个问题可以用$( n ⁇ 2d+1} d) 时间解决, 但是在合理的复杂理论假设下, 无法实现$(d)\cdot n ⁇ o(d/\log d) 运行时间, 其中美元是培训实例的数量。 这个问题可以用$( dR) {O}\ cdot n%1+o(1) 时间来解决 。 如果有两类固定的等级, 则无法实现 $(d)\cdon ⁇ (d) (d/\log d) 运行时间。 $( leg) leas the leas the leas firalf) leas a fir leshlester.