In this work, we analyze the capabilities and practical limitations of neural networks (NNs) for sequence-based signal processing which can be seen as an omnipresent property in almost any modern communication systems. In particular, we train multiple state-of-the-art recurrent neural network (RNN) structures to learn how to decode convolutional codes allowing a clear benchmarking with the corresponding maximum likelihood (ML) Viterbi decoder. We examine the decoding performance for various kinds of NN architectures, beginning with classical types like feedforward layers and gated recurrent unit (GRU)-layers, up to more recently introduced architectures such as temporal convolutional networks (TCNs) and differentiable neural computers (DNCs) with external memory. As a key limitation, it turns out that the training complexity increases exponentially with the length of the encoding memory $\nu$ and, thus, practically limits the achievable bit error rate (BER) performance. To overcome this limitation, we introduce a new training-method by gradually increasing the number of ones within the training sequences, i.e., we constrain the amount of possible training sequences in the beginning until first convergence. By consecutively adding more and more possible sequences to the training set, we finally achieve training success in cases that did not converge before via naive training. Further, we show that our network can learn to jointly detect and decode a quadrature phase shift keying (QPSK) modulated code with sub-optimal (anti-Gray) labeling in one-shot at a performance that would require iterations between demapper and decoder in classic detection schemes.
翻译:在这项工作中,我们分析神经网络(NNs)对基于序列的信号处理的能力和实际限制,这种网络在几乎所有现代通信系统中都被视为无处不在的特性。特别是,我们培训多种最先进的经常神经网络(RNN)结构,学习如何用相应的最大可能性(ML)维特比解码器解码共变代码。我们检查各种NNS结构的解码性能,首先是传统类型,如向上层和封闭的经常性单位(GRU),直到最近推出的架构,如时演变网络(TCNs)和不同的神经计算机(DNCS),具有外部记忆。作为一个关键限制,我们发现培训的复杂性随着编码内存时间长度(ML) $\nu$,从而实际上限制了可实现的比差差率(BER)的性能。为了克服这一限制,我们引入了一个新的培训模式,在培训序列内逐渐增加一个数字,也就是说,我们最终在培训阶段里要通过连续的排序中显示一个可能实现的递校程,在培训阶段里,我们最终要显示一个可能实现的递校程的递校程的递校程。