Transformer是谷歌发表的论文《Attention Is All You Need》提出一种完全基于Attention的翻译架构
知识荟萃
论文列表
原文：
- 《Attention is all you need》：https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf
相关论文
- 《Reformer: The Efficient Transformer》：https://arxiv.org/abs/2001.04451
开源代码
- Kyubyong/transformer (TF)
- huggingface/transformers (PyTorch)
VIP内容
旅行商问题(TSP)是最受欢迎和研究最多的组合问题，始于1951年的冯·诺依曼。它推动了几种优化技术的发现，如切割平面、分支定界、局部搜索、拉格朗日松弛和模拟退火。
在过去的五年里，我们看到了一些有前途的技术的出现，在这些技术中(图)神经网络能够学习新的组合算法。主要的问题是，深度学习能否从数据中学习更好的启发式，即取代人类工程的启发式?这很有吸引力，因为开发有效解决NP难题的算法可能需要多年的研究，而许多行业问题本质上是组合在一起的。
在这项工作中，我们提出将最近成功开发的用于自然语言处理的Transformer架构应用于组合TSP。训练是通过强化学习完成的，因此没有TSP训练解决方案，解码使用波束搜索。我们报告了与最近学习的启发式相比性能的改进，TSP50的最佳差距为0.004%，TSP100为0.50%。
http://www.ipam.ucla.edu/abstract/?tid=16703&pcode=DLC2021
成为VIP会员查看完整内容
【Slides】The Transformer Network for the Traveling Salesman Problem
热门内容
最新内容
最新论文
An important aspect of AI design and ethics is to create systems that reflect aggregate preferences of the society. To this end, the techniques of social choice theory are often utilized. We propose a new social choice function motivated by the PageRank algorithm. The function ranks voting options based on the Condorcet graph of pairwise comparisons. To this end, we transform the Condorcet graph into a Markov chain whose stationary distribution provides the scores of the options. We show how the values in the stationary distribution can be interpreted as quantified aggregate support for the voting options, to which the community of voters converges through an imaginary sequence of negotiating steps. Because of that, we suggest the name "convergence voting" for the new voting scheme, and "negotiated community support" for the resulting stationary allocation of scores. Our social choice function can be viewed as a consensus voting method, sitting somewhere between Copeland and Borda. On the one hand, it does not necessarily choose the Condorcet winner, as strong support from a part of the society can outweigh mediocre uniform support. On the other hand, the influence of unpopular candidates on the outcome is smaller than in the primary technique of consensus voting, i.e., the Borda count. We achieve that without having to introduce an ad hoc weighting that some other methods do.
