With the era of big data, an explosive amount of information is now available. This enormous increase of Big Data in both academia and industry requires large-scale data processing systems. A large body of research is behind optimizing Spark's performance to make it state of the art, a fast and general data processing system. Many science and engineering fields have advanced with Big Data analytics, such as Biology, finance, and transportation. Intelligent transportation systems (ITS) gain popularity and direct benefit from the richness of information. The objective is to improve the safety and management of transportation networks by reducing congestion and incidents. The first step toward the goal is better understanding, modeling, and detecting congestion across a network efficiently and effectively. In this study, we introduce an efficient congestion detection model. The underlying network consists of 3017 segments in I-35, I-80, I-29, and I-380 freeways with an overall length of 1570 miles and averaged (0.4-0.6) miles per segment. The result of congestion detection shows the proposed method is 90% accurate while has reduced computation time by 99.88%.
翻译:随着大数据时代的到来,现在可以获得大量爆炸性的信息。大数据在学术界和工业界的巨大增长需要大规模的数据处理系统。大量研究在优化斯帕克的性能以使之成为最新、快速和一般的数据处理系统背后。许多科学和工程领域都随着大数据分析,如生物学、金融和运输的发展而取得进步。智能运输系统(ITS)受到欢迎,并直接受益于信息丰富的内容。目标是通过减少拥堵和事故来改善运输网络的安全和管理。目标的第一步是更好地了解、建模和发现网络的拥挤状况。在这个研究中,我们采用了高效的堵塞探测模型。基本网络由I-35、I-80、I-29和I-380等317个部分组成,总长度为1 570英里,平均每段(0.4-0.6)英里)。堵塞探测结果显示,拟议的方法是90%的准确度,而计算时间减少了99.88%。