Not logged in.
Quick Search - Contribution
Contribution Details
| Type | Conference or Workshop Paper |
| Scope | Discipline-based scholarship |
| Published in Proceedings | Yes |
| Title | Dynamically reconfiguring software microbenchmarks: reducing execution time without sacrificing result quality |
| Organization Unit | |
| Authors |
|
| Presentation Type | paper |
| Item Subtype | Original Work |
| Refereed | Yes |
| Status | Published in final form |
| Language |
|
| ISBN | 9781450370431 |
| Page Range | 989 - 1001 |
| Event Title | 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering |
| Event Type | conference |
| Event Location | Virtual Event, USA |
| Event Start Date | December 8 - 2020 |
| Event End Date | December 13 - 2020 |
| Series Name | ESEC/FSE '20 |
| Place of Publication | New York, NY, USA |
| Publisher | ACM |
| Abstract Text | Executing software microbenchmarks, a form of small-scale performance tests predominantly used for libraries and frameworks, is a costly endeavor. Full benchmark suites take up to multiple hours or days to execute, rendering frequent checks, e.g., as part of continuous integration (CI), infeasible. However, altering benchmark configurations to reduce execution time without considering the impact on result quality can lead to benchmark results that are not representative of the software’s true performance. We propose the first technique to dynamically stop software microbenchmark executions when their results are sufficiently stable. Our approach implements three statistical stoppage criteria and is capable of reducing Java Microbenchmark Harness (JMH) suite execution times by 48.4% to 86.0%. At the same time it retains the same result quality for 78.8% to 87.6% of the benchmarks, compared to executing the suite for the default duration. The proposed approach does not require developers to manually craft custom benchmark configurations; instead, it provides automated mechanisms for dynamic reconfiguration. Hence, making dynamic reconfiguration highly effective and efficient, potentially paving the way to inclusion of JMH microbenchmarks in CI. |
| Free access at | Related URL |
| Related URLs |
|
| Digital Object Identifier | 10.1145/3368089.3409683 |
| Other Identification Number | merlin-id:20158 |
| PDF File |
Download from ZORA
|
| Export |
BibTeX
EP3 XML (ZORA)
|