Scalable Approaches for Test Suite Reduction

Technical TrackIndustry Program
Test suite reduction approaches aim at decreasing software regression testing costs by selecting a representative subset from large-size test suites. Most existing techniques are too expensive for handling modern massive systems and moreover depend on artifacts, such as code coverage metrics or specification models, that are not commonly available at high scale. We present a family of novel very efficient approaches for similarity-based test suite reduction that apply algorithms borrowed from the big data domain together with smart heuristics for finding an evenly spread subset of test cases. The approaches are very general since they only use as input the test cases themselves (test source code or command line input). We evaluate four approaches in a version that selects a fixed budget B of test cases, and also in an adequate version that does the reduction guaranteeing some fixed coverage. The results show that the approaches yield a fault detection loss comparable to state-of-the-art techniques, while providing huge gains in terms of efficiency. When applied to a suite of more than 500K real world test cases, the most efficient of the four approaches could select B test cases (for varying B values) in less than 10 seconds.
Wed 29 May
16:00 - 18:00: Papers - Test Selection and Prioritization at Duluth Chair(s): Robert FeldtChalmers University of Technology | ||||||||||||||||||||||||||||||||||||||||||
16:00 - 16:20 Talk | Armin NajafiConcordia University, Weiyi (Ian) ShangConcordia University, Canada, Peter RigbyConcordia University, Montreal, Canada | |||||||||||||||||||||||||||||||||||||||||
16:20 - 16:40 Talk | Adithya Abraham Philip, Ranjita BhagwanMicrosoft Research India, Rahul KumarMicrosoft, Chandra Sekhar MaddilaMicrosoft, Nachiappan NagappanMicrosoft Research | |||||||||||||||||||||||||||||||||||||||||
16:40 - 17:00 Talk | Emilio CrucianiGran Sasso Science Institute, L'Aquila, Italy, Breno MirandaFederal University of Pernambuco, Roberto VerdecchiaGran Sasso Science Institute, Vrije Universiteit Amsterdam, Antonia BertolinoCNR-ISTI Pre-print | |||||||||||||||||||||||||||||||||||||||||
17:00 - 17:20 Talk | Abhaya ParthyApple Inc., Leopold SilbersteinApple Inc., Emily KowalczykApple Inc., John Paul HighApple Inc., Adithya NagarajanApple Inc., Atif MemonApple Inc. | |||||||||||||||||||||||||||||||||||||||||
17:20 - 17:40 Talk | Chenguang ZhuUniversity of Texas, Austin, Owolabi LegunsenUniversity of Illinois at Urbana-Champaign, August ShiUniversity of Illinois at Urbana-Champaign, Milos GligoricUniversity of Texas at Austin | |||||||||||||||||||||||||||||||||||||||||
17:40 - 17:50 Talk | ||||||||||||||||||||||||||||||||||||||||||
17:50 - 18:00 Talk |