Blogs (1) >>
ICSE 2019
Sat 25 - Fri 31 May 2019 Montreal, QC, Canada
Wed 29 May 2019 14:40 - 15:00 at Viger - Continuous Integration Chair(s): Ipek Ozkaya

Continuous Integration traditionally relies on testing every code commit with all impacted tests. This practice requires considerable computational resources, which at Google scale, results in delayed test results and high operational costs. To deal with this issue and provide fast feedback, test selection and prioritization methods aim to execute the tests which are most likely to reveal changes in test results as soon as possible. In this paper we present a simulation framework to support the study and evaluation, with real data, of such techniques. We propose a test selection algorithm evaluation method, and detail several practical requirements which are often ignored by related work, such as the detection of transitions, the collection and analysis of data, and the handling of flaky tests. Based on this framework, we design an experiment evaluating five potential regression test selection algorithms, based on simple heuristics and inspired by previous research. Our results show that algorithms based on the recent (transition) execution history do not perform as well as expected (given the previously reported results) and that the test selection problem remains largely open. We found that the best performing algorithms are based on the number of times a test has been triggered and the number of distinct authors committing code that triggers particular tests. More research is needed in order to close the gap between the current approaches and the optimal solution.

Wed 29 May

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:30
Continuous IntegrationSoftware Engineering in Practice / Technical Track / Papers at Viger
Chair(s): Ipek Ozkaya Carnegie Mellon Software Engineering Institute
14:00
20m
Talk
Big Bangs and Small Pops: On Critical Cyclomatic Complexity and Developer Integration BehaviorSEIPIndustry Program
Software Engineering in Practice
Daniel Ståhl Ericsson AB, Antonio Martini University of Oslo, Norway, Torvald Mårtensson Saab AB
14:20
20m
Talk
Predictive Test SelectionSEIPIndustry Program
Software Engineering in Practice
Mateusz Machalica Facebook, Inc., Alex Samylkin Facebook, Inc., Meredith Porth Facebook, Inc., Satish Chandra Facebook
Pre-print
14:40
20m
Talk
Assessing Transition-based Test Selection Algorithms at GoogleSEIPIndustry Program
Software Engineering in Practice
Claire Leong Google / UNSW, Abhayendra Singh Google, Inc, Mike Papadakis University of Luxembourg, Yves Le Traon University of Luxembourg, John Micco Netflix
Pre-print
15:00
20m
Talk
Automated Reporting of Anti-Patterns and Decay in Continuous IntegrationArtifacts AvailableArtifacts Evaluated ReusableTechnical Track
Technical Track
Carmine Vassallo University of Zurich, Sebastian Proksch University of Zurich, Harald Gall University of Zurich, Massimiliano Di Penta University of Sannio
Pre-print
15:20
10m
Talk
Discussion Period
Papers