Blogs (1) >>
ICSE 2019
Sat 25 - Fri 31 May 2019 Montreal, QC, Canada

Online controlled experiments are the gold standard for evaluating improvements and accelerating innovations in online and app worlds. However, little is known about applicability, implementation, and efficacy of experimentation for operating systems (OS), where many features are non-user-facing. In this paper, we present the Windows Experimentation platform (WExp), and insights from implementation and execution of real-world experiments in the OS. We start by discussing the need for experimentation in OS, using real experiments to illustrate the benefits. We then describe the architecture of WExp, focusing on unique considerations in its engineering. Finally, we discuss learnings and challenges from conducting real-world experiments. Our experiences and insights can motivate practitioners that build OS, platforms, and other non-user-facing components to start experimenting, can help practitioners to successfully build an experimentation platform, can help guide experimenters with best-practices, and highlight for researchers promising avenues for future work.

Wed 29 May

11:00 - 12:30: Papers - Controlled Experiments of Production Software at St-Denis / Notre-Dame
Chair(s): Yvonne DittrichIT University of Copenhagen, Denmark
icse-2019-Software-Engineering-in-Practice11:00 - 11:20
Aleksander FabijanMicrosoft, Pavel, Helena Holmström OlssonMalmö University, Jan BoschChalmers University of Technology, Sweden, Lukas, Dylan LewisIntuit
icse-2019-Software-Engineering-in-Practice11:20 - 11:40
Tong XiaMicrosoft, Sumit BhardwajMicrosoft, Pavel, Aleksander FabijanMicrosoft
icse-2019-Software-Engineering-in-Practice11:40 - 12:00
Paul Luo LiMicrosoft, Pavel, Huibin Mary HuMicrosoft, Xiaoyu ChaiMicrosoft, Zoran DimovMicrosoft, Brandon PaddockMicrosoft, Ying LiMicrosoft, Alex KirshenbaumMicrosoft, Irina NiculescuMicrosoft, Taj ThoresenMicrosoft
icse-2019-Software-Engineering-in-Practice12:00 - 12:20
Ali BasiriNetflix, Lorin HochsteinNetflix, Nora JonesNetflix, Haley TuckerNetflix
icse-2019-Paper-Presentations12:20 - 12:30