Case Studies 

We are showing three case studies on commits detected by PRICE. Each case study is from different code change regression root-cause as discovered by Chen et al. Chen et al. [1]. We investigated 20 random commits from Git are known to have regression after benchmark testing and tested PRICE rules on those commits. We found that code changes related to the expensive variable are detected only if changes are conducted on top overhead function and will miss detecting regression otherwise. Changes related to functions or condition are mostly detected correctly as there are metrics reflecting those changes.

Figure 1: Regression code level root-cause of commits under study

As stated by the developers, the change of this commit is moving one variable to be a part of a struct. Accessing struct defined data memory is more expensive than accessing primitive data types. This code change was detected correctly because it is associated with many minor code changes related to this change in a function known to be from the top overhead.

Figure 2: An example of a commit using memory expensive variables.

Changing conditions or adding one implies additional logical operations or even includes function calls. These changes might also change the code branching in some cases.

Figure 3: An example of a commit changing conditions.

Changing function calls category includes adding, deleting, or modifying the function signature. Adding a new function and calling it from another code portion as the listed example in Figure 4 is actually adding overhead to the code execution. Regression detectors should detect those changes. PRICE was able to detect this change as metrics indicate a change in the code functions that were part of the detection rule.

Figure 4: An example of a commit adding a new function. File 1 contains the added new function. File 2 contains the call for the new function.