Gaussian process state-space model (GPSSM) is a probabilistic dynamical system that represents unknown transition and/or measurement models as Gaussian process (GP). The majority of the approaches to learning GP-SSM are focused on handling given time series data. However, in most dynamical systems, data required for model learning arrives sequentially and accumulates over time. Storing all the data requires large amounts of memory, and using it for model learning can be computationally infeasible. To overcome these challenges, we propose an online inference method, onlineGPSSM, for learning the GP-SSM by incorporating stochastic variational inference (VI) and online VI. The proposed method can mitigate the computation time issue without catastrophic forgetting and supports adaptation to changes in a system and/or a real environments. Furthermore, we propose an application of onlineGPSSM to the reinforcement learning (RL) of partially observable dynamical systems by combining onlineGPSSM with Bayesian filtering and trajectory optimization algorithms. Numerical examples are presented to demonstrate the applicability of the proposed method.
翻译:高斯进程状态-空间模型(GPSSM)是一个概率性动态系统,代表了与Gaussian进程(GP)一样的未知过渡和/或测量模型。学习GP-SSM的大多数方法侧重于处理给定的时间序列数据。然而,在大多数动态系统中,模型学习所需的数据按顺序运抵并随时间积累。存储所有数据需要大量内存,而将其用于模型学习则无法进行计算。为了克服这些挑战,我们建议采用在线推论方法(在线GPSSSM),即在线GPSMSSM,通过纳入随机变异推(VI)和在线六来学习GP-SSM。拟议方法可以减轻计算时间问题,而不会灾难性地忘记,支持适应系统和/或真实环境中的变化。此外,我们提议将在线GPSSSSSM与Bayesian过滤和轨迹优化算法相结合,将部分可观测动态系统的强化学习(RL)。