Blogs (1) >>
ICSE 2019
Sat 25 - Fri 31 May 2019 Montreal, QC, Canada
Thu 30 May 2019 14:50 - 15:10 at Laurier - Automated Repair 2 Chair(s): Hamid Bagheri

Current state-of-the-art automatic software repair (ASR) techniques rely heavily on incomplete specifications, or test suites, to generate repairs. This, however, may cause ASR tools to generate repairs that are incorrect and hard to generalize. To assess patch correctness, researchers have been following two methods separately: (1) Automated annotation, wherein patches are automatically labeled by an independent test suite (ITS) – a patch passing the ITS is regarded as correct or generalizable, and incorrect otherwise, (2) Author annotation, wherein authors of ASR techniques manually annotate the correctness labels of patches generated by their and competing tools. While automated annotation cannot ascertain that a patch is actually correct, author annotation is prone to subjectivity. This concern has caused an on-going debate on the appropriate ways to assess the effectiveness of numerous ASR techniques proposed recently.

In this work, we propose to assess reliability of author and automated annotations on patch correctness assessment. We do this by first constructing a gold set of correctness labels for 189 randomly selected patches generated by 8 state-of-the-art ASR techniques through a user study involving 35 professional developers as independent annotators. By measuring inter-rater agreement as a proxy for annotation quality – as commonly done in the literature – we demonstrate that our constructed gold set is on par with other high-quality gold sets. We then compare labels generated by author and automated annotations with this gold set to assess reliability of the patch assessment methodologies. We subsequently report several findings and highlight implications for future studies.

Thu 30 May
Times are displayed in time zone: Eastern Time (US & Canada) change

14:00 - 15:30: Automated Repair 2Papers / Journal-First Papers / Software Engineering in Practice / Technical Track at Laurier
Chair(s): Hamid BagheriUniversity of Nebraska-Lincoln, USA
14:00 - 14:20
SapFix: Automated End-to-End Repair at ScaleSEIPIndustry Program
Software Engineering in Practice
Alexandru MargineanUniversity College London, UK, Johannes BaderFacebook, Satish ChandraFacebook, Mark HarmanFacebook and University College London, Yue JiaUniversity College London, Ke MaoFacebook, Alexander MolsFacebook, Andrew ScottFacebook
14:20 - 14:40
VFix: Value-Flow-Guided Precise Program Repair for Null Pointer DereferencesArtifacts Evaluated ReusableTechnical Track
Technical Track
Xuezheng XuUNSW Sydney, Yulei SuiUniversity of Technology Sydney, Australia, Hua YanUniversity of New South Wales, Jingling XueUNSW Sydney
14:40 - 14:50
ARJA: Automated Repair of Java Programs via Multi-Objective Genetic ProgrammingJournal-First
Journal-First Papers
Yuan YuanMichigan State University, Wolfgang BanzhafMichigan State University
14:50 - 15:10
On Reliability of Patch Correctness AssessmentTechnical Track
Technical Track
Xuan Bach D. LeCarnegie Mellon University, Lingfeng Bao Zhejiang University City College, David LoSingapore Management University, Xin XiaMonash University, Shanping Li, Corina S PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center
15:10 - 15:20
Alleviating Patch Overfitting with Automatic Test Generation: A Study of Feasibility and Effectiveness for the Nopol Repair SystemJournal-First
Journal-First Papers
Zhongxing Yu, Matias MartinezUniversity of Valenciennes, Benjamin DanglotUniversity Lille 1 and INRIA, Thomas DurieuxINRIA, Martin MonperrusKTH Royal Institute of Technology
15:20 - 15:30
Discussion Period