VIP内容
尽管生成式对抗网络(GAN)的历史并不长，但它已被广泛地研究和用于各种任务，包括其最初的目的，即合成样品的生成。然而，将GAN用于具有不同神经网络结构的不同数据类型，由于其在训练方面的局限性，使得模型很容易出现混乱。这种臭名昭著的GAN训练是众所周知的，并已在许多研究中提出。因此，为了使GAN的训练更加稳定，近年来提出了许多正则化方法。本文综述了近年来引入的正则化方法，其中大部分是近三年来发表的。具体地说，我们关注的是那些可以被普遍使用的方法，而不管神经网络体系结构如何。根据其运算原理将其分为若干组，并分析了各方法之间的差异。此外，为了提供使用这些方法的实际知识，我们调研了在最先进的GANs中经常使用的流行方法。此外，我们还讨论了现有方法的局限性，并提出了未来的研究方向。
Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
热门内容
Video captioning is a challenging task that requires a deep understanding of visual scenes. State-of-the-art methods generate captions using either scene-level or object-level information but without explicitly modeling object interactions. Thus, they often fail to make visually grounded predictions, and are sensitive to spurious correlations. In this paper, we propose a novel spatio-temporal graph model for video captioning that exploits object interactions in space and time. Our model builds interpretable links and is able to provide explicit visual grounding. To avoid unstable performance caused by the variable number of objects, we further propose an object-aware knowledge distillation mechanism, in which local object information is used to regularize global scene features. We demonstrate the efficacy of our approach through extensive experiments on two benchmarks, showing our approach yields competitive performance with interpretable predictions.
最新内容
最新论文
Nearly all statistical inference methods were developed for the regime where the number $N$ of data samples is much larger than the data dimension $p$. Inference protocols such as maximum likelihood (ML) or maximum a posteriori probability (MAP) are unreliable if $p=O(N)$, due to overfitting. This limitation has for many disciplines with increasingly high-dimensional data become a serious bottleneck. We recently showed that in Cox regression for time-to-event data the overfitting errors are not just noise but take mostly the form of a bias, and how with the replica method from statistical physics once can model and predict this bias and the noise statistics. Here we extend our approach to arbitrary generalized linear regression models (GLM), with possibly correlated covariates. We analyse overfitting in ML/MAP inference without having to specify data types or regression models, relying only on the GLM form, and derive generic order parameter equations for the case of $L2$ priors. Second, we derive the probabilistic relationship between true and inferred regression coefficients in GLMs, and show that, for the relevant hyperparameter scaling and correlated covariates, the $L2$ regularization causes a predictable direction change of the coefficient vector. Our results, illustrated by application to linear, logistic, and Cox regression, enable one to correct ML and MAP inferences in GLMs systematically for overfitting bias, and thus extend their applicability into the hitherto forbidden regime $p=O(N)$.
