Code Review and Static Analysis

A Fusion of Complementary Disciplines Delivers a Complete Picture of Your Code Base

What’s the difference between static analysis and code review? Static analysis can tell you if you are violating best practices or have common errors in your code. Code review can tell you that your code conforms to customer requirements and is actually correct.  At SmartBear, we espouse that to ensure the most software quality with the least time investment, development teams can benefit from both techniques. Our view is that code review complements static analysis, because when conducted together, they provide development teams with a complete picture of their code base.

Recently, Capers Jones, the leading authority on defect levels inherent in major software projects, delivered a webinar for SmartBear about the State of Software Quality 2011 (click link to register for webinar on-demand). According to Capers, employing inspections,* static analysis and testing, result in a greater than 97% efficiency level at removing defects from the software. Defect prevention (inspections and static analysis) coupled with defect removal (inspections and testing) provide the best overall strategy for ensuring high software quality.  In essence, his message is that these two defect identification strategies work better together rather than alone.

*Inspections apply to code, design and requirements documents.

The value of code review combined with static analysis is that it lets reviewers focus on the issues that static analysis cannot catch.  Code reviews enable development teams to identify coding issues early during the development process, making the job of remediating those issues much easier because everything is still “fresh” in the author’s head.  The more quality that gets integrated into the code during development, the fewer bugs exist in the code when it gets released to QA, enabling QA to focus effort on strategic testing activities rather than on bug identification. Although many static analysis tools can supplement the process by identifying bugs, they really only tell you that you have a bug; they cannot convey either the danger inherent in the bug or its consequence. Code review provides insight into the impact of the bug – is it a show-stopper or is it trivial—so developers can evaluate the prioritization of the fix.

Code Review and Static Analysis: Key Distinctions

Axis Code Review Static Analysis
Code Formatting Boredom, Flamewars Precise identification of violations and, often, automatic fixes
Naming Conventions Have you named it the right thing? (is minval actually the minimum value) Have you named it the right way? (it should be named MINVAL)
Complexity Human metric of “is this simple enough to understand” Blunt metric of cyclomatic complexity
Capability Uncovers problems in new patterns Known patterns of bug that have been formalized
Correctness Understands intent Does not know what the code is supposed to do

 

Here’s a great example of a style guide:

Everything in that style guide that can possibly be enforced or checked by automated tools should be. It’s unrealistic to expect that reviewers will read the code with a checklist like that for each changed line.

Here’s a great example of what static analysis fails to detect.

Getting software to understand that a function named getRandomNumber should probably not always return the same value is beyond the current state of the art.

Here’s what a static analysis dashboard looks like.

There’s a bunch of numbers you can try to move up and down. You might introduce real bugs as you try to move the numbers. Moving the numbers may have little effect on product quality. Adding peer review inspection to your process identifies defects with a human evaluation of their gravity when you are best equipped to fix them: right after you wrote them.

Static analysis can inform a developer’s next actions. Code review drives them.