Appendix

Source Code Metrics

Source code metrics are properties of the source code that can be measured to gain relevant information on the quality of the implementation analyzed. QualityGate measures source code at multiple levels of abstraction: implementation, file, unit and for many other technology specific concepts, like packages, classes, interfaces, etc.

QualityGate groups metrics into the following 6 categories:

  • Cohesion metrics: to measure the extent of how coherent source code elements are.
  • Complexity metrics: to measure the complexity of source code elements.
  • Coupling metrics: to measure the number of interdependencies between source code elements.
  • Documentation metrics: to measure how well the source code is documented.
  • Inheritance metrics: to measure the different aspects of the inheritance hierarchy of the implementation.
  • Size metrics: to measure the volume of the source code elements.

Code Duplication Metrics

Code cloning (or copy-paste programming) means the copying of an existing piece of source code, pasting it somewhere else, and typically performing smaller modifications on it. Based on the level of similarity between the copied code fragments, the following duplication types can be defined:

  • Type-1: the copied code parts are identical code fragments except for variations in whitespace, layout and comments.
  • Type-2: syntactically identical fragments except for variations in identifier names, literals, type references, whitespace, layout and comments.
  • Type-3: copied fragments with further modifications such as altered, added, or removed statements, in addition to variations in identifier names, literals, type references, whitespace, layout and comments.
  • Type-4: the copied code fragments are syntactically different, but they perform the same functionality, i.e. they are semantically the same.

QualityGate is capable of identifying Type-2 clones, i.e. code fragments that are structurally identical, but may differ in variable names, literals, identifiers, etc.

Two code segments correspond to each other if they are copies of each other. This relation is an equivalence relation and we use the notion of clone classes to the classes of the relation, and the members of the classes will be referred to as clone instances. Owing to the nature of the relation, each clone class must contain at least two clone instances.

Clones are tracked during the source code analysis of consecutive revisions of the analyzed software system. QualityGate detects suspicious, inconsistently changing code copies, which are called “clone smells”. The smells concerning clone classes are the following:

  • Disappearing Clone Class (DCC): The clone class existed in the previously analyzed revision but it does not exist anymore.
  • Appearing Clone Class (ACC): The clone class did not exist in the previously analyzed revision.

The smells concerning clone instances are issued only if the containing clone class does not contain any smell. These are the following:

  • Disappearing Clone Instance (DCI): The clone instance existed in the previously analyzed revision but it does not exist anymore.
  • Appearing Clone Instance (ACI): The clone instance did not exist in the previously analyzed revision.
  • Moving Clone Instance (MCI): The clone instance belonged to a different clone class in the previously analyzed revision than its current class.

VulnerabilityHunter Coding Rule Violations

The VulnerabilityHunter module of QualityGate can detect OWASP security vulnerabilities in Java web applications. The attackers can use these security vulnerabilities to get sensitive information. The majority of these types of attacks use the lack or improper implementation of the validation of the user inputs to achieve unwanted program behavior. The following table summarizes the security vulnerabilities QualityGate can detect.

The list of rulesets and rules contained in each ruleset is the following:

  • Vulnerability Rules: These rules deal with different security issues arise with tainting user inputs in web applications.

The following table contains the enabled rules and their priorities:

_images/vulnerabilityhunter.png

RTEHunter coding rule violations

The RTEHunter module of QualityGate can detect potential runtime errors in Java applications. These runtime errors are represented by the instances of Java class RuntimeException. The detection is based on an interprocedural symbolic execution engine.

The following table contains the enabled rules and their priorities:

Name Abbreviation Prio.
NullPointerException RH_NPE Blocker
DivisionByZero RH_DBZ Blocker
ArrayIndexOutOfBounds RH_IOB Blocker
NegativeArraySize RH_NAS Blocker

AndroidHunter coding rule violations

AndroidHunter is the module of QualityGate, which looks for Android specific coding rule violations. This module checks the Java sources of the Android projects for common Android specific faults and anti-patterns. AndroidHunter reimplements all the corresponding PMD rules and additionally it provides some checks that are not available in PMD. Such as the other modules of QualityGate, AndroidHunter also works on the precise Abstract Semantic Graph which results in higher precision and recall compared to other tools with a rougher syntactic analysis.

The list of rulesets and rules contained in each ruleset is the following:

  • Design Rules: Rules that help you discover design issues.
  • Error Prone Rules: Rules to detect constructs that are either broken, extremely confusing or prone to runtime errors.
  • Performance Rules: Rules that flag suboptimal code.

FaultHunter coding rule violations

FaultHunter is the coding rule violation checker module of QualityGate. This module makes it possible to identify common Java coding rule violations in the code (so-called bad practices) in a similar way as provided by the PMD tool. However, the algorithms implemented in the FaultHunter module work on the precise Abstract Semantic Graph of QualityGate which results in higher precision and recall compared to other tools with a rougher syntactic analyzer.

FaultHunter reimplements many PMD rule violation checks and provides less noise (false positives) and a number of real problems not detected by PMD (true negatives). It also provides additional checks that are not available in PMD.

The list of rulesets and rules contained in each ruleset is the following:

  • Best Practice Rules: Rules which enforce generally accepted best practices.
  • Code Style Rules: Rules which enforce a specific coding style.
  • Design Rules: Rules that help you discover design issues.
  • Documentation Rules: Rules that are related to code documentation.
  • Error Prone Rules: Rules to detect constructs that are either broken, extremely confusing or prone to runtime errors.
  • Multithreading Rules: Rules that flag issues when dealing with multiple threads of execution.
  • Performance Rules: Rules that flag suboptimal code.

PMD coding rule violations

QualityGate incorporates the PMD tool for coding rule violation checking and imports its results. SourceMeter also associates the issued rule violations with source code elements (i.e. methods, classes, packages, and components), and calculates metrics for the source code elements, which represent the amount of violations of each ruleset, rule, and priority groups, respectively. In addition, SourceMeter uses an optimized PMD setting, where the poorly performing PMD rule checks are disabled, and all other rules are reprioritized by our software developer and QA experts. Please note that if a valid FaultHunter license key is available, then the reimplemented PMD rules will be switched off automatically and FaultHunter will provide the results instead. If no license key is available (free version), then the original PMD rule violation checks will be executed.

QualityGate uses PMD “as is”, without any guarantees that the results of PMD are correct. All statements of the PMD license apply here as well. All texts describing the rulesets and the individual rules are copied from its official home page with some minor grammatical fixes.

The list of rulesets and rules contained in each ruleset is the following:

  • Best Practice Rules: Rules which enforce generally accepted best practices.
  • Code Style Rules: Rules which enforce a specific coding style.
  • Design Rules: Rules that help you discover design issues.
  • Documentation Rules: Rules that are related to code documentation.
  • Error Prone Rules: Rules to detect constructs that are either broken, extremely confusing or prone to runtime errors.
  • Multithreading Rules: Rules that flag issues when dealing with multiple threads of execution.
  • Performance Rules: Rules that flag suboptimal code.
  • Security Rules: Rules that flag potential security flaws.

SpotBugs coding rule violations

QualityGate applies the SpotBugs tool for coding rule violation checking and imports its results. It also associates the issued rule violations with source code elements (i.e. methods, classes, packages, and components), and calculates metrics for the source code elements, which represent the number of violations of each ruleset, rule, and priority groups, respectively.

QualityGate uses SpotBugs “as is”, without any guarantees that the results of SpotBugs are correct. All statements of the SpotBugs license apply here as well. All texts describing the rulesets and the individual rules are copied from the configuration files of SpotBugs with some minor grammatical fixes.

The list of rulesets and rules contained in each ruleset is the following:

  • Bad Practice Rules: Violations of recommended and essential coding practice. Examples include hash code and equals problems, cloneable idiom, dropped exceptions, Serializable problems, and misuse of finalize. We strive to make this analysis accurate, although some groups may not care about some of the bad practices.
  • Correctness Rules: Probable bug - an apparent coding mistake resulting in code that was probably not what the developer intended. We strive for a low false positive rate.
  • Dodgy Code Rules: code that is confusing, anomalous, or written in a way that leads itself to errors. Examples include dead local stores, switch fall through, unconfirmed casts, and redundant null check of value known to be null. More false positives accepted. In previous versions of SpotBugs, this category was known as Style.
  • Internationalization Rules: code flaws having to do with internationalization and locale
  • Multithreaded Correctness Rules: code flaws having to do with threads, locks, and volatiles
  • Optimization Rules: code that is not necessarily incorrect but may be inefficient
  • Security Code Guideline Rules: code that is vulnerable to attacks from untrusted code
  • Vulnerability Rules: A use of untrusted input in a way that could create a remotely exploitable security vulnerability.

Back: Benchmarks