Multi-year empirical study jointly published with the Software Engineering Institute at Carnegie Mellon

Defects are not reported correctly

• 26% of defect rules are incorrectly described
• Additional 35% of defect rules are mis-aligned

Defect coverage is an issue

• No tool has > 20% coverage – A major source of zero day vulnerabilities
• Less than 1% of defects found by 2 or more tools

The detailed analysis showed that the reported defects are noisy

• 43% have low confidence of being a real defect
• 76% are hard to exploit
• 13% have severe consequence

Reported severity levels are incorrect

• 97% of defects reported as severe are not, when all factors are considered
• 80% of our SATriage tool-determined severe defects are dismissed as minor by AST vendors