Mutation analysis assesses a test suite's adequacy by measuring its ability to detect small artificial faults, systematically seeded into the tested program. Mutation analysis is considered one of the strongest test-adequacy criteria. Mutation testing builds on top of mutation analysis and is a testing technique that uses mutants as test goals to create or improve a test suite. Mutation testing has long been considered intractable because the sheer number of mutants that can be created represents an insurmountable problem -- both in terms of human and computational effort. This has hindered the adoption of mutation testing as an industry standard. For example, Google has a codebase of two billion lines of code and more than 500,000,000 tests are executed on a daily basis. The traditional approach to mutation testing does not scale to such an environment. To address these challenges, this paper presents a scalable approach to mutation testing based on the following main ideas: (1) Mutation testing is done incrementally, mutating only changed code during code review, rather than the entire code base; (2) Mutants are filtered, removing mutants that are likely to be irrelevant to developers, and limiting the number of mutants per line and per code review process; (3) Mutants are selected based on the historical performance of mutation operators, further eliminating irrelevant mutants and improving mutant quality. Evaluation in a code-review-based setting with more than 24,000 developers on more than 1,000 projects shows that the proposed approach produces orders of magnitude fewer mutants and that context-based mutant filtering and selection improve mutant quality and actionability. Overall, the proposed approach represents a mutation testing framework that seamlessly integrates into the software development workflow and is applicable up to large-scale industrial settings.
相关内容
Cyber-physical systems and the Internet of things (IoT) is becoming an integral part of the digital society. The use of IoT services improves human life in many ways. Protection against cyber threats is an important aspect of the functioning of IoT devices. Malicious activities lead to confidential data leakages and incorrect performance of devices are becoming critical. Therefore, development of effective solutions that can protect both IoT devices data and data exchange networks turns in to a real challenge. This study provides a critical analysis of the feasibility of using blockchain technology to protect constrained IoT devices data, justifies the choice of Practical Byzantine Fault Tolerance (PBFT) consensus algorithm for implementation on such devices, and simulates the main distributed ledger scenarios using PBFT. The simulation results demonstrate the efficiency of the blockchain technology for constrained devices and make it possible to evaluate the applicability limits of the chosen consensus algorithm.
Structured Latent Attribute Models (SLAMs) are a family of discrete latent variable models widely used in education, psychology, and epidemiology to model multivariate categorical data. A SLAM assumes that multiple discrete latent attributes explain the dependence of observed variables in a highly structured fashion. Usually, the maximum marginal likelihood estimation approach is adopted for SLAMs, treating the latent attributes as random effects. The increasing scope of modern assessment data involves large numbers of observed variables and high-dimensional latent attributes. This poses challenges to classical estimation methods and requires new methodology and understanding of latent variable modeling. Motivated by this, we consider the joint maximum likelihood estimation (MLE) approach to SLAMs, treating latent attributes as fixed unknown parameters. We investigate estimability, consistency, and computation in the regime where sample size, number of variables, and number of latent attributes all can diverge. We establish the statistical consistency of the joint MLE and propose efficient algorithms that scale well to large-scale data for several popular SLAMs. Simulation studies demonstrate the superior empirical performance of the proposed methods. An application to real data from an international educational assessment gives interpretable findings of cognitive diagnosis.
We present a shared-memory algorithm to compute high-quality solutions to the balanced $k$-way hypergraph partitioning problem. This problem asks for a partition of the vertex set into $k$ disjoint blocks of bounded size that minimizes the connectivity metric (i.e., the sum of the number of different blocks connected by each hyperedge). High solution quality is achieved by parallelizing the core technique of the currently best sequential partitioner KaHyPar: the most extreme $n$-level version of the widely used multilevel paradigm, where only a single vertex is contracted on each level. This approach is made fast and scalable through intrusive algorithms and data structures that allow precise control of parallelism through atomic operations and fine-grained locking. We perform extensive experiments on more than 500 real-world hypergraphs with up to $140$ million vertices and two billion pins (sum of hyperedge sizes). We find that our algorithm computes solutions that are on par with a comparable configuration of KaHyPar while being an order of magnitude faster on average. Moreover, we show that recent non-multilevel algorithms specifically designed to partition large instances have considerable quality penalties and no clear advantage in running time.
We propose Remote Attestation with TOCTOU Avoidance (RATA): a provably secure approach to address the RA TOCTOU problem. With RATA, even malware that erases itself before execution of the next RA, can not hide its ephemeral presence. RATA targets hybrid RA architectures (implemented as Hardware/Software co-designs), which are aimed at low-end embedded devices. We present two alternative techniques - RATAa and RATAb - suitable for devices with and without real-time clocks, respectively. Each is shown to be secure and accompanied by a publicly available and formally verified implementation. Our evaluation demonstrates low hardware overhead of both techniques. Compared with current RA architectures - that offer no TOCTOU protection - RATA incurs no extra runtime overhead. In fact, RATA substantially reduces computational costs of RA execution.
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms of time to solution over previous systems. We achieve this by (i) designing the ZionEX platform with dedicated scale-out network, provisioned with high bandwidth, optimal topology and efficient transport (ii) implementing an optimized PyTorch-based training stack supporting both model and data parallelism (iii) developing sharding algorithms capable of hierarchical partitioning of the embedding tables along row, column dimensions and load balancing them across multiple workers; (iv) adding high-performance core operators while retaining flexibility to support optimizers with fully deterministic updates (v) leveraging reduced precision communications, multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we develop and briefly comment on distributed data ingestion and other supporting services that are required for the robust and efficient end-to-end training in production environments.
The software development industry has been evolving with new development standards and service delivery models. Agile methodologies have reached their completion with DevOps, thereby increasing the quality of the software and creating greater speed in delivery. However, a gap regarding the formalization of its adoption and implementation doubts became relevant. My hypothesis is that, by systematizing the introduction of DevOps into the software development process and defining the function of the members of the DevOps team members, may well make it quicker to implement this process, thus reducing conflicts between the teams. As part of the investigation of this hypothesis, the result of the research will be applied in practical development environments i.e. in a Technology Agency of the State of the Brazilian Government and also at the Brazilian Company Neurotech in order to evaluate its effectiveness from metrics appropriate for DevOps environments.
Alternative recommender systems are critical for ecommerce companies. They guide customers to explore a massive product catalog and assist customers to find the right products among an overwhelming number of options. However, it is a non-trivial task to recommend alternative products that fit customer needs. In this paper, we use both textual product information (e.g. product titles and descriptions) and customer behavior data to recommend alternative products. Our results show that the coverage of alternative products is significantly improved in offline evaluations as well as recall and precision. The final A/B test shows that our algorithm increases the conversion rate by 12 percent in a statistically significant way. In order to better capture the semantic meaning of product information, we build a Siamese Network with Bidirectional LSTM to learn product embeddings. In order to learn a similarity space that better matches the preference of real customers, we use co-compared data from historical customer behavior as labels to train the network. In addition, we use NMSLIB to accelerate the computationally expensive kNN computation for millions of products so that the alternative recommendation is able to scale across the entire catalog of a major ecommerce site.
Train machine learning models on sensitive user data has raised increasing privacy concerns in many areas. Federated learning is a popular approach for privacy protection that collects the local gradient information instead of real data. One way to achieve a strict privacy guarantee is to apply local differential privacy into federated learning. However, previous works do not give a practical solution due to three issues. First, the noisy data is close to its original value with high probability, increasing the risk of information exposure. Second, a large variance is introduced to the estimated average, causing poor accuracy. Last, the privacy budget explodes due to the high dimensionality of weights in deep learning models. In this paper, we proposed a novel design of local differential privacy mechanism for federated learning to address the abovementioned issues. It is capable of making the data more distinct from its original value and introducing lower variance. Moreover, the proposed mechanism bypasses the curse of dimensionality by splitting and shuffling model updates. A series of empirical evaluations on three commonly used datasets, MNIST, Fashion-MNIST and CIFAR-10, demonstrate that our solution can not only achieve superior deep learning performance but also provide a strong privacy guarantee at the same time.
As a new classification platform, deep learning has recently received increasing attention from researchers and has been successfully applied to many domains. In some domains, like bioinformatics and robotics, it is very difficult to construct a large-scale well-annotated dataset due to the expense of data acquisition and costly annotation, which limits its development. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed (i.i.d.) with the test data, which motivates us to use transfer learning to solve the problem of insufficient training data. This survey focuses on reviewing the current researches of transfer learning by using deep neural network and its applications. We defined deep transfer learning, category and review the recent research works based on the techniques used in deep transfer learning.
Machine Learning is a widely-used method for prediction generation. These predictions are more accurate when the model is trained on a larger dataset. On the other hand, the data is usually divided amongst different entities. For privacy reasons, the training can be done locally and then the model can be safely aggregated amongst the participants. However, if there are only two participants in \textit{Collaborative Learning}, the safe aggregation loses its power since the output of the training already contains much information about the participants. To resolve this issue, they must employ privacy-preserving mechanisms, which inevitably affect the accuracy of the model. In this paper, we model the training process as a two-player game where each player aims to achieve a higher accuracy while preserving its privacy. We introduce the notion of \textit{Price of Privacy}, a novel approach to measure the effect of privacy protection on the accuracy of the model. We develop a theoretical model for different player types, and we either find or prove the existence of a Nash Equilibrium with some assumptions. Moreover, we confirm these assumptions via a Recommendation Systems use case: for a specific learning algorithm, we apply three privacy-preserving mechanisms on two real-world datasets. Finally, as a complementary work for the designed game, we interpolate the relationship between privacy and accuracy for this use case and present three other methods to approximate it in a real-world scenario.
相关论文
Yaroslav Meshcheryakov,Anna Melman,Oleg Evsutin,Vladimir Morozov,Yevgeni Koucheryavy
0+阅读 · 4月20日
Yuqi Gu,Gongjun Xu
0+阅读 · 4月17日
Lars Gottesbüren,Tobias Heuer,Peter Sanders,Sebastian Schlag
0+阅读 · 4月16日
Ivan De Oliveira Nunes,Sashidhar Jakkamsetti,Norrathep Rattanavipanon,Gene Tsudik
0+阅读 · 4月15日
Dheevatsa Mudigere,Yuchen Hao,Jianyu Huang,Andrew Tulloch,Srinivas Sridharan,Xing Liu,Mustafa Ozdal,Jade Nie,Jongsoo Park,Liang Luo,Jie Amy Yang,Leon Gao,Dmytro Ivchenko,Aarti Basant,Yuxi Hu,Jiyan Yang,Ehsan K. Ardestani,Xiaodong Wang,Rakesh Komuravelli,Ching-Hsiang Chu,Serhat Yilmaz,Huayu Li,Jiyuan Qian,Zhuobo Feng,Yinbin Ma,Junjie Yang,Ellie Wen,Hong Li,Lin Yang,Chonglin Sun,Whitney Zhao,Dimitry Melts,Krishna Dhulipala,KR Kishore,Tyler Graf,Assaf Eisenman,Kiran Kumar Matam,Adi Gangidi,Guoqiang Jerry Chen,Manoj Krishnan,Avinash Nayak,Krishnakumar Nair,Bharath Muthiah,Mahmoud khorashadi,Pallab Bhattacharya,Petr Lapukhov,Maxim Naumov,Lin Qiao,Mikhail Smelyanskiy,Bill Jia,Vijay Rao
0+阅读 · 4月15日
Luciano de Aguiar Monteiro
0+阅读 · 4月15日
Mingming Guo,Nian Yan,Xiquan Cui,San He Wu,Unaiza Ahsan,Rebecca West,Khalifeh Al Jadda
0+阅读 · 4月15日
Lichao Sun,Jianwei Qian,Xun Chen,Philip S. Yu
3+阅读 · 2020年7月31日
Chuanqi Tan,Fuchun Sun,Tao Kong,Wenchang Zhang,Chao Yang,Chunfang Liu
10+阅读 · 2018年8月6日
Balazs Pejo,Qiang Tang
4+阅读 · 2018年2月28日
相关VIP内容
【变分推断课件】Lectures on Variational Inference： Approximate Bayesian Inference in Machine Learning（附带pdf）
专知会员服务
19+阅读 · 2019年11月30日
专知会员服务
13+阅读 · 2019年10月17日
相关资讯
