Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data. However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision. While previous work has focused on exploiting statistical independence to disentangle latent factors, we argue that such requirement is too restrictive and propose instead a non-statistical framework that relies on counterfactual manipulations to uncover a modular structure of the network composed of disentangled groups of internal variables. Experiments with a variety of generative models trained on complex image datasets show the obtained modules can be used to design targeted interventions. This opens the way to applications such as computationally efficient style transfer and the automated assessment of robustness to contextual changes in pattern recognition systems.
The "curse of dimensionality" is a well-known problem in pattern recognition. A widely used approach to tackling the problem is a group of subspace methods, where the original features are projected onto a new space. The lower dimensional subspace is then used to approximate the original features for classification. However, most subspace methods were not originally developed for classification. We believe that direct adoption of these subspace methods for pattern classification should not be considered best practice. In this paper, we present a new information theory based algorithm for selecting subspaces, which can always result in superior performance over conventional methods. This paper makes the following main contributions: i) it improves a common practice widely used by practitioners in the field of pattern recognition, ii) it develops an information theory based technique for systematically selecting the subspaces that are discriminative and therefore are suitable for pattern recognition/classification purposes, iii) it presents extensive experimental results on a variety of computer vision and pattern recognition tasks to illustrate that the subspaces selected based on maximum mutual information criterion will always enhance performance regardless of the classification techniques used.
One of the major challenges in multivariate analysis is the estimation of population covariance matrix from sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or asymptotic results from Random Matrix Theory (RMT). Shrinkage techniques help in pulling extreme correlation values towards certain target values whereas tools from RMT help in removing noisy eigenvalues of SCM. Both of these techniques use different approaches to achieve a similar goal which is to remove noisy correlations and add structure to SCM to overcome the bias-variance trade-off. In this paper, we first critically evaluate the pros and cons of these two techniques and then propose an improved estimator which exploits the advantages of both by taking an optimally weighted convex combination of covariance matrices estimated by an improved shrinkage transformation and a RMT based filter. It is a generalized estimator which can adapt to changing sampling noise conditions in various datasets by performing hyperparameter optimization. We show the effectiveness of this estimator on the problem of designing a financial portfolio with minimum risk. We have chosen this problem because the complex properties of stock market data provide extreme conditions to test the robustness of a covariance estimator. Using data from four of the world's largest stock exchanges, we show that our proposed estimator outperforms existing estimators in minimizing the out-of-sample risk of the portfolio and hence predicts population statistics more precisely. Since covariance analysis is a crucial statistical tool, this estimator can be used in a wide range of machine learning, signal processing and high dimensional pattern recognition applications.
With the development of research on memristor, memristive neural networks (MNNs) have become a hot research topic recently. Because memristor can mimic the spike timing-dependent plasticity (STDP), the research on STDP based MNNs is rapidly increasing. However, although state-of-the-art works on STDP based MNNs have many applications such as pattern recognition, STDP mechanism brings relatively complex hardware framework and low processing speed, which block MNNs' hardware realization. A non-STDP based unsupervised MNN is constructed in this paper. Through the comparison with STDP method on the basis of two common structures including feedforward and crossbar, non-STDP based MNNs not only remain the same advantages as STDP based MNNs including high accuracy and convergence speed in pattern recognition, but also better hardware performance as few hardware resources and higher processing speed. By virtue of the combination of memristive character and simple mechanism, non-STDP based MNNs have better hardware compatibility, which may give a new viewpoint for memristive neural networks' engineering applications.