Blogs (1) >>
ICSE 2019
Sat 25 - Fri 31 May 2019 Montreal, QC, Canada
Fri 31 May 2019 12:00 - 12:10 at Laurier - Defect Prediction Chair(s): Burak Turhan

Software defect data sets are typically characterized by an unbalanced class distribution where the defective modules are fewer than the nondefective modules. Prediction performances of defect prediction models are detrimentally affected by the skewed distribution of the faulty minority modules in the data set since most algorithms assume both classes in the data set to be equally balanced. Resampling approaches address this concern by modifying the class distribution to balance the minority and majority class distribution. However, very little is known about the best distribution for attaining high performance especially in a more practical scenario. There are still inconclusive results pertaining to the suitable ratio of defect and clean instances (Pfp), the statistical and practical impacts of resampling approaches on prediction performance and the more stable resampling approach across several performance measures. To assess the impact of resampling approaches, we investigated the bias and effect of commonly used resampling approaches on prediction accuracy in software defect prediction. Analyses of six resampling approaches on 40 releases of 20 open-source projects across five performance measures and five imbalance rates were performed. The experimental results obtained indicate that there were statistical differences between the prediction results with and without resampling methods when evaluated with the geometric-mean, recall(pd), probability of false alarms(pf ) and balance performance measures. However, resampling methods could not improve the AUC values across all prediction models implying that resampling methods can help in defect classification but not defect prioritization. A stable Pfp rate was dependent on the performance measure used. Lower Pfp rates are required for lower pf values whilst higher Pfp values are required for higher pd values. Random Under-Sampling and Borderline-SMOTE proved to be the more stable resampling method across several performance measures among the studied resampling methods. Performance of resampling methods are dependent on the imbalance ratio, evaluation measure and to some extent the prediction model. Newer oversampling methods should aim at generating relevant and informative data samples and not just increasing the minority samples.

Fri 31 May
Times are displayed in time zone: (GMT-04:00) Eastern Time (US & Canada) change

11:00 - 12:30: Papers - Defect Prediction at Laurier
Chair(s): Burak TurhanMonash University
icse-2019-Journal-First-Paper11:00 - 11:10
Zhiyuan WanZhejiang University, Xin XiaMonash University, Ahmed E. HassanQueen's University, David LoSingapore Management University, Jianwei Yin, Xiaohu Yang
icse-2019-Technical-Papers11:10 - 11:30
Suraj YatishThe University of Adelaide, Jirayus JiarpakdeeMonash University, Patanamon ThongtanunamThe University of Melbourne, Chakkrit (Kla) TantithamthavornMonash University, Australia
icse-2019-Technical-Papers11:30 - 11:50
George CabralUniversity of Birmingham, Leandro Minku , Emad ShihabConcordia University, Suhaib MujahidConcordia University
icse-2019-Journal-First-Paper11:50 - 12:00
Chakkrit (Kla) TantithamthavornMonash University, Australia, Ahmed E. HassanQueen's University, Kenichi MatsumotoNara Institute of Science and Technology
icse-2019-Journal-First-Paper12:00 - 12:10
Kwabena E. BenninBlekinge Institute of Technology, SERL Sweden, Jacky Keung, Akito Monden
Authorizer link
icse-2019-New-Ideas-and-Emerging-Reults12:10 - 12:20
Cristina MonniUniversità della Svizzera Italiana, Mauro PezzeUniversità della Svizzera italiana (USI) (Switzerland) and Università degli Studi di Milano Bicocca (Italy)
icse-2019-Paper-Presentations12:20 - 12:30