Tue 27 Oct 2015 15:54 - 16:18 at Grand Station 3 - Session 4, Empirical Studies

Little is known about how software performance evolves across software revisions. The severity of this situation is high since (i) most performance variations seem to happen accidentally and (ii) addressing a performance regression is challenging, especially when functional code is stacked on it.

This paper reports an empirical study on the performance evolution of 19 applications, totaling over 19 MLOC. It took 52 days to run our 49 benchmarks. By relating performance variation with source code revisions, we found out that: (i) 1 out of every 3 application revisions introduces a performance variation, (ii) performance variations may be classified into 9 patterns, (iii) the most prominent cause of performance regression involves loops and collections. We carefully describe the patterns we identified, and detail how we addressed the numerous challenges we faced to complete our experiment.

Tue 27 Oct

15:30 - 17:30: DLS - Session 4, Empirical Studies at Grand Station 3
dls201515:30 - 15:54
Beatrice ÅkerblomStockholm University, Tobias WrigstadUppsala University
dls201515:54 - 16:18
dls201516:18 - 16:42
Madhukar KedlayaUniversity of California, Santa Barbara, Behnam RobatmiliQualcomm Research, Ben HardekopfUC Santa Barbara
dls201516:42 - 17:06
Lars FischerUniversity of Duisburg-Essen, Essen, Germany, Stefan HanenbergUniversity of Duisburg-Essen
dls201517:06 - 17:30
Camille TeruelINRIA, Stéphane DucasseINRIA, France, Damien CassouLille 1 University, Marcus Denker INRIA Lille