We derive a well-defined renormalized version of mutual information that allows to estimate the dependence between continuous random variables in the important case when one is deterministically dependent on the other. This is the situation relevant for feature extraction, where the goal is to produce a low-dimensional effective description of a high-dimensional system. Our approach enables the discovery of collective variables in physical systems, thus adding to the toolbox of artificial scientific discovery, while also aiding the analysis of information flow in artificial neural networks.
翻译:我们获得一个定义明确的、经过重新整顿的相互信息版本,从而可以估计在重要情况下,当一个人决定性地依赖另一个人时,连续随机变量之间的依赖性。这就是与地物提取有关的情况,目的是对高维系统进行低维有效描述。我们的方法使得能够在物理系统中发现集体变量,从而增加人工科学发现的工具箱,同时帮助分析人工神经网络的信息流动。