Blogs (1) >>
ICSE 2019
Sat 25 - Fri 31 May 2019 Montreal, QC, Canada
Fri 31 May 2019 14:50 - 15:10 at Viger - Crowdsourcing in Software Engineering Chair(s): Tayana Conte

Crowdtesting has grown to be an effective alternative to traditional testing, especially in mobile applications. However, crowdtesting is hard to manage in nature. Given the complexity of mobile applications and unpredictability of distributed crowdtesting process, it is difficult to estimate (a) the remaining number of bugs as yet undetected or (b) the required cost to find those bugs. Experience-based decisions may result in ineffective crowdtesting process, i.e., there are an average of 32% wasteful spending in current crowdtesting practice. This paper aims at exploring automated decision support to effectively manage crowdtesting process. The proposed iSENSE applies incremental sampling technique to process crowdtesting reports arriving in chronological order, organizes them into fixed-size groups as dynamic inputs, and predicts two test completion indicators in an incrementally manner. The two indicators are: 1)total number of bugs predicted with Capture-ReCapture (CRC) model, and 2) required test cost for achieving certain test objectives predicted with Auto Regressive Integrated Moving Average(ARIMA) model. We assess iSENSE using 46,434 reports of 218 crowdtesting tasks from one of the largest crowdtesting platforms in China. Its effectiveness is demonstrated through two applications for automating crowdtesting management, i.e., automation of task closing decision, and semi-automation of task closing trade-off analysis. The results show that decision automation using iSENSE will provide managers with greater opportunities to achieve cost-effectiveness gains of crowdtesting. Specifically, a median of 100% bugs can be detected with 30% saved cost based on the automated close prediction.

Fri 31 May
Times are displayed in time zone: Eastern Time (US & Canada) change

14:00 - 15:30
Crowdsourcing in Software EngineeringPapers / Software Engineering in Practice / Technical Track at Viger
Chair(s): Tayana ConteUniversidade Federal do Amazonas
14:00
30m
Talk
(SEIP Talk) Crowdsourcing in Software Engineering: Models, Motivations, and ChallengesSEIPIndustry Program
Software Engineering in Practice
Thomas LaTozaGeorge Mason University
14:30
20m
Talk
CTRAS: Crowdsourced Test Report Aggregation and SummarizationTechnical TrackIndustry Program
Technical Track
hao rui, Yang FengUniversity of California, Irvine, James JonesUniversity of California, Irvine, Yuying LiState Key Laboratory for Novel Software Technology, Nanjing University, Zhenyu ChenNanjing University
14:50
20m
Talk
iSENSE: Completion-Aware Crowdtesting ManagementACM SIGSOFT Distinguished Paper AwardTechnical TrackIndustry Program
Technical Track
Junjie WangInstitute of Software, Chinese Academy of Sciences, Ye YangStevens institute of technology, Rahul KrishnaNC State University, Tim MenziesNorth Carolina State University, Qing WangInstitute of Software, Chinese Academy of Sciences
15:10
20m
Talk
Discussion Period
Papers