LSTM是一种时间递归神经网络(RNN)[1],论文首次发表于1997年。由于独特的设计结构,LSTM适合于处理和预测时间序列中间隔和延迟非常长的重要事件。

VIP内容

为了克服递归网络(RNN)学习长期依赖的困难,长短期记忆(LSTM)网络于1997年被提出并后续在应用方面取得了重大进展。大量论文证实了LSTM的实用性并试图分析其性质。而“RNN和LSTM是否具有长期记忆?”这个问题依然缺少答案。本论文从统计学的角度回答了这一问题,证明了RNN和LSTM在做时间序列的预测时不具备统计意义上的长期记忆。统计学已有的对于长期记忆的定义并不适用于神经网络,于是我们提出了一个对于神经网络适用的新定义,并利用新定义再次分析了RNN和LSTM的理论性质。为了验证我们的理论,我们对RNN和LSTM进行了最小程度的修改,将他们转换为长期记忆神经网络,并且在具备长期记忆性质的数据集上验证了它们的优越性。

成为VIP会员查看完整内容
0
29

最新内容

This paper is dedicated to team VAA's approach submitted to the Fashion-IQ challenge in CVPR 2020. Given a pair of the image and the text, we present a novel multimodal composition method, RTIC, that can effectively combine the text and the image modalities into a semantic space. We extract the image and the text features that are encoded by the CNNs and the sequential models (e.g., LSTM or GRU), respectively. To emphasize the meaning of the residual of the feature between the target and candidate, the RTIC is composed of N-blocks with channel-wise attention modules. Then, we add the encoded residual to the feature of the candidate image to obtain a synthesized feature. We also explored an ensemble strategy with variants of models and achieved a significant boost in performance comparing to the best single model. Finally, our approach achieved 2nd place in the Fashion-IQ 2020 Challenge with a test score of 48.02 on the leaderboard.

0
0
下载
预览

最新论文

This paper is dedicated to team VAA's approach submitted to the Fashion-IQ challenge in CVPR 2020. Given a pair of the image and the text, we present a novel multimodal composition method, RTIC, that can effectively combine the text and the image modalities into a semantic space. We extract the image and the text features that are encoded by the CNNs and the sequential models (e.g., LSTM or GRU), respectively. To emphasize the meaning of the residual of the feature between the target and candidate, the RTIC is composed of N-blocks with channel-wise attention modules. Then, we add the encoded residual to the feature of the candidate image to obtain a synthesized feature. We also explored an ensemble strategy with variants of models and achieved a significant boost in performance comparing to the best single model. Finally, our approach achieved 2nd place in the Fashion-IQ 2020 Challenge with a test score of 48.02 on the leaderboard.

0
0
下载
预览
Top