The gauge function, closely related to the atomic norm, measures the complexity of a statistical model, and has found broad applications in machine learning and statistical signal processing. In a high-dimensional learning problem, the gauge function attempts to safeguard against overfitting by promoting a sparse (concise) representation within the learning alphabet. In this work, within the context of linear inverse problems, we pinpoint the source of its success, but also argue that the applicability of the gauge function is inherently limited by its convexity, and showcase several learning problems where the classical gauge function theory fails. We then introduce a new notion of statistical complexity, gauge$_p$ function, which overcomes the limitations of the gauge function. The gauge$_p$ function is a simple generalization of the gauge function that can tightly control the sparsity of a statistical model within the learning alphabet and, perhaps surprisingly, draws further inspiration from the Burer-Monteiro factorization in computational mathematics. We also propose a new learning machine, with the building block of gauge$_p$ function, and arm this machine with a number of statistical guarantees. The potential of the proposed gauge$_p$ function theory is then studied for two stylized applications. Finally, we discuss the computational aspects and, in particular, suggest a tractable numerical algorithm for implementing the new learning machine.
翻译:与原子规范密切相关的测量值函数测量一个统计模型的复杂性,并在机器学习和统计信号处理中发现广泛的应用。在一个高维学习问题中,测量值函数试图通过在学习字母中促进稀少(精密)的表达方式来防止过度适应。在这项工作中,在线性反问题的背景下,我们找出其成功的来源,但也争辩说该测量函数的适用性受到其共性本身的限制,并展示了古典测量函数理论失败的一些学习问题。然后我们引入了一个新的统计复杂性概念,即测量$_p$的功能,它克服了测量功能的局限性。衡量$_p$的功能是测量函数的简单化,它能够严格控制学习字母中统计模型的广度,并且也许令人惊讶地从计算数学中的布勒-蒙泰罗系数化中进一步获得灵感。我们还提议了一台新的学习机器,用测量$_p$的模块功能,用一些统计保证的功能。拟议的测量值_p$的功能是测量器的简单化功能,然后为两个数学的计算过程进行了研究。