快速快速探索随机树,用于斯托卡优化控制 (Forward-Backward Rapidly-Exploring Random Trees for Stochastic Optimal Control)

We propose a numerical method for the computation of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. By the use of the Girsanov change of probability measures, it is demonstrated how a rapidly-exploring random tree (RRT) method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of the constructed RRT. Moreover, a local entropy-weighted least squares Monte Carlo (LSMC) method is developed to concentrate function approximation accuracy in regions most likely to be visited by optimally controlled trajectories. The results of the proposed methodology are demonstrated on linear and nonlinear stochastic optimal control problems with non-quadratic running costs, which reveal significant convergence improvements over previous FBSDE-based numerical solution methods.

翻译：我们提议了一个数字方法,用于计算Feynman-Kac中出现的前向后向随机差异方程式(FBSDE),该方程式代表了在随机最佳控制问题中的价值功能。通过使用Girsanov改变概率测量方法,可以证明如何在前向融合通道中使用快速勘探随机树(RRT)方法,只要在后向融合通道中受控的漂移条件得到适当补偿。随后,通过在构造的RRT边缘解决一系列倒向的功能近似问题,提出了该值的数值近似值。此外,还开发了一种本地的蒙泰卡洛最小正方形(LSMC)方法,在最有可能通过最佳控制轨迹访问的区域集中功能近似精度。拟议方法的结果在线性和非线性直线性随机最佳控制问题和非线性最佳控制成本上得到证明,这表明与以前基于FBSDE的数字解决方案相比,趋同率有了显著的趋同性改进。