|
By Eugene Zakhareyev
February 2008 (originally published in February '08 issue of TFS Times newsletter)
The article discusses Static Code Analysis tools available from Microsoft, including stand-alone application
and capabilities integrated into Visual Studio 2005/2008 Team System editions.
Introduction
Static Code Analysis feature in Visual Studio is different from other "edition-specific"
feature because it is also available as stand-alone application for users of any
Visual Studio edition. Stand-alone static code analysis engine, called FxCop,
is free and currently available in two versions (version 1.35 and version 1.36).
Thus if you are not sure of what value static code analysis might bring for you,
it is easy to try FxCop first to get a feeling of code analysis featureset, and
then graduate to Visual Studio edition with integrated Code Analysis support (and
perhaps with Team Foundation Server integration as well).
In the current article, I shall try to provide an overview of the features and the
difference between different Static Code Analysis products from Microsoft - stand-alone
FxCop application, Visual Studio Team Developer edition and Visual Studio Team Developer
with Team Foundation Server support.
Using Code Analysis
The whole idea of the static code analysis is to find the potential issues in the
software component without actually executing it (and that includes unit tests as
form of execution as well). I shall assume forthwith that the concept is indisputably
useful and concentrate on managed code analysis; while code analysis for C++
is also available in Visual Studio, it requires extra effort during the coding that
few appear be willing to invest and therefore is not widely spread.
To give a short overview, Visual Studio Code Analysis (and FxCop, since both use
same core code) runs on compiled managed code binaries, and verifies the compiled
MSIL code against set of rules specified by the user. The rules available out of
the box with Visual Studio Team Developer edition (or FxCop) are separated into
several categories, covering such areas of software development as Design, Security,
Usage, Performance etc. Since the analysis is performed on compiled code, it is
impossible to enforce code syntax conventions (such as number of spaces between
type and variable name in declaration); but it is possible to implement your own
custom rules.
The resulting report will contain the list of rules violated, with additional metadata
available for every rule, such as rule unique identifier, rule description, rule
category etc. together with code specific data. Once the report is available, the
user may choose to ignore certain rules violations while fixing the rest of them.
Another typical use of code analysis is in an automated build, to make sure code
analysis rules violations on integration are identified early on (as in nightly
builds) and not propagated to released software (as in release builds). The command
line code analysis execution is supported both by FxCop and Visual Studio Static
Code Analysis.
Using Code Analysis
The natural start-up solution (unless you have already have VS Team Developer or
Team Suite edition handy) is FxCop. The stand-alone application has lean installer
and features simple yet sufficient user interface allowing user to define rule set
to execute, assemblies to analyze and rules violations review screen. While both
version 1.35 and the latest version (version 1.36 beta) may be used to analyze binaries
built with all versions of .Net Framework available (1, 2 and 3), version 1.36 also
supports .Net 3.5 specific constructs (lambda expressions), has better performance
and multiple bug fixes, and rules available are similar to those in Visual Studio
2008.
Since FxCop is a stand-alone application, active rules set and other definitions
as well as analysis results are stored in separate project file, unrelated to Visual
Studio solution/project.
The following are some of the advantages of FxCop over Visual Studio integrated
Static Code Analysis:
- The code analysis definitions are maintained in separate project and thus do not
affect VS projects of the analyzed components.
- FxCop provides convenient GUI for filtering of the analysis results, that shows
violated rules of certain type or category (especially useful when experimenting
with different code analysis rule sets)
- FxCop provides easy visualization of analysis rules in UI - Active rule violations,
rules violations excluded in project, rules violations excluded in source code (using
SupressMessage attribute) and absent rules (rules available but not activated
during analysis)
- The application lightweight engine does not require loading of solutions or projects;
that may be important for large solutions
On the disadvantages side it is worth to note the following:
- Since all definitions are in separate project, it cannot be maintained in same IDE
(read VS) and requires setting up external process for running code analysis and
rules definitions maintenance
- Source code navigation ("jumping" from rule violation to appropriate line in the
source and back) is awkward (though it can be partially integrated into VS to overcome
that)
- The results of the code analysis cannot be merged into Team Foundation Server data
store and not available for subsequent reporting
- There are additional rules available in Visual Studio as compared to stand-alone
rules set [9]
Static Code Analysis in Visual Studio 2005
In Visual Studio 2005 Team Developer (or Team Suite), Static Code Analysis is integrated
into IDE. That means that the code analysis settings are part of the managed (C#
or VB.NET) project definitions, and the analysis results are displayed in Error
List window.
The advantages of integrated Static Code Analysis over FxCop are
- The rule set definition is integral part of VS Project Properties dialog, thus providing
easy access to customization with no external tools involved
- The analysis report is in format familiar to developers, minimizing learning time
- The navigation between Static Code Analysis report (displayed in Errors List window)
and source code is simple and natural
However, there are still some areas where FxCop is better than integrated Static
Code Analysis:
- Since the rule set is defined per project, it may be not trivial to propagate same
rules definitions over all projects, if only selected rules are to be activated
for analysis (this issue is somewhat mitigated if using TFS – see below)
- The filtering options of the analysis report are much poorer; the large number of
violations can be easily overwhelming in simple Visual Studio interface
- The rule violations suppressed by user or by using SuppressMessage attribute are
not easily visible in Visual Studio interface
Altogether, the choice of more appropriate tool is dependent on the organization
policies related to static code analysis (more on that in Best Practices section
below). But that is true only if the picture does not include Team Foundation Server;
with TFS Visual Studio becomes way more attractive.
Static Code Analysis in Visual Studio 2005 with Team Foundation Server
Team Foundation Server with its version control, work item tracking, reporting and
build capabilities available in conjunction with Static Code Analysis integrated
in Visual Studio changes quite a lot of things to the better.
First, it becomes easy to enforce running the same set of rules throughout the organization
with Code Analysis check-in policy. The administrator may define a set of rules
for specific Team Project, and Static Code Analysis will be run with the rule set
on every check-in for that project. If the analysis displays rule violations, the
user will have to fix the issues before check in or provide explanation for overriding
the policy.
Additionally, Static Code Analysis may be run as part of the code review process
with the reviewer generating Work Item for certain warnings (the functionality is
integrated into Errors List window). That way only selected issues identified through
code analysis can be handled.
Static Code Analysis can be run as a part of Team Build project build, so that any
rule violations detected would result in breaking the build, thus ensuring the proper
quality. In addition to being available as part of the build report, the build results data may
be also published to TFS data warehouse and thus become available for reporting
using MS SQL Reporting Services.
Static Code Analysis in Visual Studio 2008
In Visual Studio 2008, the following changes occurred to Static Code Analysis:
- Code Analysis has become more prominent in Visual Studio IDE and now has its own
Analyze menu
- Additional rules available (including custom spelling dictionaries)
- Support for .Net Framework 3.x features (such as anonymous methods and lambda expressions)
- VS2008 Code Analysis engine has better detection abilities as well as improved performance
And finally, new feature called Code Metrics was introduced in Visual Studio 2008.
Analyzing the code with Code Metrics, the user is able to calculate such metrics
as Maintainability Index, Cyclomatic Complexity, Depth of Inheritance, Class Coupling
and Lines of Code. The metrics are calculated on the component, class or method
level, which makes it easy to identify the problematic areas in code. Dedicated
user interface is provided within Visual Studio to facilitate the detection of such
areas with filtering and easy navigation to source code.
The metrics are also available as rules (for example, Cycolmatic Complexity is "Avoid
Excessive Complexity" rule) thus allowing for metrics to be turned on/off using
uniform interface with the other rules and specified for external builds.
Static Code Analysis in Visual Studio 2008 with Team Foundation Server
Team Foundation Server 2008 provides improvements in using Code Analysis policy
(specifically, handling conflicts between policy code analysis settings and project
specific settings, and providing better means of synchronizing policy with projects
definitions).
Additionally, since Code Metrics are available as rules, they can be specified as
part of the policy definitions.
Static Code Analysis in Rosario and beyond
Rosario CTP from November 2007 included number of code analysis related features,
most notably the initial implementation of rule sets. The idea behind the rule sets
is to provide several out of the box sets of rules handling different development
scenarios, and also to allow users generate custom rule sets answering their internal
needs.
While current preview gives a glimpse of the intention behind the rule sets, the
Code Analisys team is eager for the feedback on predefined rule sets contents.
Static Code Analysis best practices
While both FxCop and integrated Code Analysis in Visual Studio provide a dearth
of features, the initial success of implementing code analysis practices in organization
will not be solely dependent on tool features (or lack thereof).
For most organizations, the code analysis will have to be implemented over the existing
code base, and without proper process in place the initial number of violations
can easily overwhelm the developers and management, and potential effort of clean
up overshadow any future gains.
It can be assumed that the safest way of implementing code analysis practices is
to establish at least two rule sets – one for new development and one for the legacy
code. The former rule set will be superset of the latter, containing all desired
rules. The legacy rules set would be minimal, with the goal of gradually making
it identical to the new development rules set, on rule by rule basis.
In such manner, legacy code will eventually become as compliant with organization
code analysis policies as the new code, without all-at-once push for compliance.
Additionally, the internal education of users as to the purpose of the rules may
be critical for the success of the effort; internal “advocates” for code analysis
can ensure that relevant rules are observed and the process is dynamically adjusted
(f. e., new rules are selected as needed).
Conclusion
I have tried to give a short survey of the features and differences between different
flavors of Microsoft Static Code Analysis with a hope of providing a starting point
for those considering implementation of static code analysis.
In my personal opinion, both FxCop and Visual Studio integrated Static Code Analysis
can and should coexist peacefully in the organization, with FxCop used for development
and testing of the code analysis policies and with Visual Studio being the main
tool used by every single developer applying these policies.
Integration with Team Foundation Server certainly may be useful, if the automated
build system is to be implemented, and the quality of the builds to be accessed
based on static code analysis results as well as successful build completion; and
check-in policies are to be used to enforce the usage of proper code analysis rule
sets across the organization.
In the next version of Visual Studio, one may well expect some serious inroads into
better management of rule definitions as well as new approaches to static code analysis.
One non-Microsoft tool displaying a host of interesting features is NDepend,
providing such features as query language on analysis results for better access
to problematic areas on large code bases and complexity graphic visualization; and
hopefully, Rosario will provide some comparable features in the near future. The
Code Analysis team is open for suggestions and can be contacted through their blog.
Static Code Analysis Resources
Here is the short list of most representative resources dedicated to Static Code
Analysis:
Acknowledgements
Great thanks to David M. Kean from Microsoft for technical review. SnagIt
is still the best software for screen capture.
About the author
Eugene Zakhareyev is a principal consultant at Attrice Corporation in San-Jose,
California. His primary focus is SCM and development methodologies using Visual
Studio Team System. He may be contacted through his email.
|
