Supplementals
RQ1. What is the impact of including static metrics on the detection performance?
Including static metrics might enhance the detection of performance regression. However, those metrics are known to be highly correlated. Autospearman approach is applied to find metrics correlation. In this section, we are applying feature selection over 35 static metrics. To study the impact of including those metrics to light-weight dynamic metrics for regression detection, we are showing the detection results of three datasets: light-weight dynamic metrics only, including all 35 static metrics and including only the non-correlated static metrics.
Hit Rate
Hit rate comparison between applied algorithms on the three datasets. The validation method applied is 10-fold cross-validation.
 |  |
NSGAII | IBEA |
 |  |
PEAS-II | SPEA2 |
 |  |
RVEA | DBEA |
Dismiss Rate
Dismiss rate comparison between applied algorithms on the three datasets. The validation method applied is 10-fold cross-validation.
 |  |
NSGAII | IBEA |
 |  |
PEAS-II | SPEA2 |
 |  |
RVEA | DBEA |
F-measure
F-measure comparison between applied algorithms on the three datasets. The validation method applied is 10-fold cross-validation.
 |  |
NSGAII | IBEA |
 |  |
PEAS-II | SPEA2 |
 |  |
RVEA | DBEA |
Jiarpakdee, J., Tantithamthavorn, C., & Treude, C. (2018, September). Autospearman: Automatically mitigating correlated software metrics for interpreting defect models. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME) (pp. 92-103). IEEE Computer Society.