Blogs (1) >>
ICSE 2019
Sat 25 - Fri 31 May 2019 Montreal, QC, Canada

Online controlled experiments are the gold standard for evaluating improvements and accelerating innovations in online and app worlds. However, little is known about applicability, implementation, and efficacy of experimentation for operating systems (OS), where many features are non-user-facing. In this paper, we present the Windows Experimentation platform (WExp), and insights from implementation and execution of real-world experiments in the OS. We start by discussing the need for experimentation in OS, using real experiments to illustrate the benefits. We then describe the architecture of WExp, focusing on unique considerations in its engineering. Finally, we discuss learnings and challenges from conducting real-world experiments. Our experiences and insights can motivate practitioners that build OS, platforms, and other non-user-facing components to start experimenting, can help practitioners to successfully build an experimentation platform, can help guide experimenters with best-practices, and highlight for researchers promising avenues for future work.

Wed 29 May

icse-2019-Paper-Presentations
11:00 - 12:30: Papers - Controlled Experiments of Production Software at St-Denis / Notre-Dame
Chair(s): Yvonne DittrichIT University of Copenhagen, Denmark
icse-2019-Software-Engineering-in-Practice11:00 - 11:20
Talk
Aleksander FabijanMicrosoft, Pavel DmitrievOutreach.io, Helena Holmström OlssonMalmö University, Jan BoschChalmers University of Technology, Sweden, Lukas VermeerBooking.com, Dylan LewisIntuit
icse-2019-Software-Engineering-in-Practice11:20 - 11:40
Talk
Tong XiaMicrosoft, Sumit BhardwajMicrosoft, Pavel DmitrievOutreach.io, Aleksander FabijanMicrosoft
icse-2019-Software-Engineering-in-Practice11:40 - 12:00
Talk
Paul Luo LiMicrosoft, Pavel DmitrievOutreach.io, Huibin Mary HuMicrosoft, Xiaoyu ChaiMicrosoft, Zoran DimovMicrosoft, Brandon PaddockMicrosoft, Ying LiMicrosoft, Alex KirshenbaumMicrosoft, Irina NiculescuMicrosoft, Taj ThoresenMicrosoft
icse-2019-Software-Engineering-in-Practice12:00 - 12:20
Talk
Ali BasiriNetflix, Lorin HochsteinNetflix, Nora JonesNetflix, Haley TuckerNetflix
Pre-print
icse-2019-Paper-Presentations12:20 - 12:30
Talk