Blogs (1) >>
ICSE 2019
Sat 25 - Fri 31 May 2019 Montreal, QC, Canada
Thu 30 May 2019 11:30 - 11:50 at Laurier - Unit Testing Chair(s): Martin Kropp

Reliable code coverage tools are critically important as it is heavily used to facilitate many quality assurance activities, such as software testing, fuzzing, and debugging. However, little attention has been devoted to assessing the reliability of code coverage tools. In this study, we propose a randomized differential testing approach to hunting for bugs in the most widely used C code coverage tools. Specifically, by generating random input programs, our approach seeks for inconsistencies in code coverage reports produced by different code coverage tools, and then identifies inconsistencies as potential code coverage bugs. To effectively report code coverage bugs, we addressed three specific challenges: (1) How to filter out duplicate test programs as many of them triggering the same bugs in code coverage tools; (2) how to automatically reduce large test programs to much smaller ones that have the same properties; and (3) how to determine which code coverage tools have bugs? The extensive evaluations validate the effectiveness of our approach, resulting in 42 and 28 confirmed/fixed bugs for gcov and llvm-cov, respectively. This case study indicates that code coverage tools are not as reliable as it might have been envisaged. It not only demonstrates the effectiveness of our approach, but also highlights the need to continue improving the reliability of code coverage tools. This work opens up a new direction in code coverage validation which calls for more attention in this area.

Thu 30 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Unit TestingTechnical Track / Software Engineering in Practice / Papers / Journal-First Papers at Laurier
Chair(s): Martin Kropp University of Applied Sciences Northwestern Switzerland
(SEIP Talk) Mythical Unit Test CoverageSEIPIndustry Program
Software Engineering in Practice
Vard Antinyan Volvo Car Group, Miroslaw Staron University of Gothenburg
Research paper
Hunting for Bugs in Code Coverage Tools via Randomized Differential TestingTechnical Track
Technical Track
Yibiao Yang Nanjing University, China, Yuming Zhou , Hao Sun Unaffiliated, Zhendong Su ETH Zurich, Zhiqiang Zuo Nanjing University, China, Lei Xu Nanjing University, Baowen Xu
Rotten Green TestsTechnical Track
Technical Track
Julien Delplanque University of Lille, Stéphane Ducasse INRIA Lille, Guillermo Polito Univ. Lille, CNRS, Centrale Lille, Inria, UMR 9189 - CRIStAL - Centre de Recherche en Informatique Signal et Automatique de Lille, Andrew P. Black Portland State University and INRIA, Anne Etien Ecole Polytechnique Universitaire de Lille
A comprehensive study of pseudo-tested methodsJournal-First
Journal-First Papers
Oscar Luis Vera Pérez INRIA, Benjamin Danglot University Lille 1 and INRIA, Martin Monperrus KTH Royal Institute of Technology, Benoit Baudry KTH Royal Institute of Technology, Sweden
Link to publication DOI Pre-print
Discussion Period