Benchmarks

Traditional security solutions are too slow. They were not designed for modern DevOps with fast CI/CD pipelines. Static Reviewer can scan a complex JAVA app made by 10,000 Files and 1,000,000 LOC in under 5 minutes with 4 GB RAM and 2 cores, 52X faster than other SAST tools. Our Code Inspection tool can analyze every inserted Pull Request (PR) or every build so that developers never have to wait for security results.

MTTR

By inserting fast and accurate code analysis into Pull Requests or builds, Security Reviewer ensures the developers have the right security information as soon as possible. Mean Time to Remediation (MTTR) is minimized by providing developers the complete data flow, the exact line number(s) where the flaw exists, runtime prioritization, and also the malicious payload(s) that successfully exploit the application.

Speed and Accuracy

Traditional SAST tools generate a lot of False Positives. This requires significant resources to triage and tune these tools increasing the TCO. Security Reviewer provides a dramatic reduction of False Positives applying its patenteed Dynamic Syntax Tree algorithm.

While maintaining our products we follow the 10 Golden Images Best Practices.

For Speed monitoring and Accuracy tests, Static Reviewer (SAST) and Dynamic Reviewer (DAST) are continuosly tested against public Benchmarks and scanning of publicy available buggy apps. Further, we continuosly follow the GitHub's Trending Projects they're somewhat close to real-world applications.

Acronyms used in the context of computing benchmarking are the following:

• TP = number of True Positives = issues expected and detected

• FP = number of False Positives = issues not expected but detected

• FN = number of False Negatives = issues expected but not detected

• True Positive Rate (TPR) = TP / (TP + FN)

• False Discovery Rate (FDR)  = FP / (FP + TP)

See details below.

See also: Compliance Modules

OWASP Benchmark 1.2

The OWASP Benchmark Project is a JAVA test suite designed to verify the speed and accuracy of vulnerability detection tools. It is a fully runnable open source web application that can be analyzed by any type of Application Security Testing (AST) tool, including SAST, DAST tools. The intent is that all the vulnerabilities deliberately included in and scored by the Benchmark are actually exploitable so its a fair test for any kind of application vulnerability detection tool. The Benchmark also includes scorecard generators for numerous open source and commercial AST tools, and the set of supported tools is growing all the time.

Open

The project documentation is all on the OWASP site at the OWASP Benchmark project pages.

The current latest release is v1.2.

Open

Security Reviewer has the Highest Score

TPR=97.1%

FPR=1.1%

Accuracy=96%

in both the OWASP Benchmarks

The ability to validate vulnerabilities in production also using our DAST solutions further improves the accuracy and reliability of Security Reviewer’s scan results.

OWASP Webgoat

WebGoat is a deliberately insecure JavaScript and JAVA web application maintained by OWASP designed to teach web application security lessons.

This program is a demonstration of common server-side JAVA application flaws. The exercises are intended to test application security and penetration testing techniques. More often is used to test SAST and DAST tools results accuracy.

Open

Static Reviewer SAST alone reaches:

TPR=92%

FPR=5.1%

Note it is very difficult for SAST tool to reach better accuracy. You need a correlation between SAST, SCA, IAST tools. By using our ASPM correlation between Static Reviewer (SAST), SCA Reviewer and third-party BlackDuck Seeker IAST tool, we reached TPR=98% and FPR=1.2%.

Spectral Goat

A security testbed, vulnerable by design for testing codesec pipeline solutions.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=1.3%

DVWA

Damn Vulnerable Web Application (DVWA) is a PHP/MySQL web application that is damn vulnerable. The aim of DVWA is to practice some of the most common web vulnerability, with various difficultly levels, with a simple straightforward interface. Please note, there are both documented and undocumented vulnerability with this software. This is intentional. You are encouraged to try and discover as many issues as possible.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=1.3%

GO-DVWA

The Go Damn Vulnerable Web App is a vulnerable application demonstration.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=1.3%

OWASP Security Shepherd

The OWASP Security Shepherd Project is a web and mobile application security training platform. Security Shepherd has been designed to foster and improve security awareness among a varied skill-set demographic. The aim of this project is to take AppSec novices or experienced engineers and sharpen their penetration testing skill set to security expert status. Seldom is used to test SAST+DAST correlation accuracy too.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=1.3%

Flowbot

Flowbot is an advanced multi-platform chatbot framework that provides intelligent conversation, workflow automation, and comprehensive LLM agent capabilities with extensive third-party integrations. It has several security holes.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=96%

FPR=2.3%

NIST-SAMATE Juliet Test Suite for C/C++

The Juliet Test Suite for C/C++ version 1.3 from https://samate.nist.gov/SARD/testsuite.php augmented with a build system for Unix-like OSes that supports automatically building test cases into individual executables and running those tests, organized by CWE.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=89%

FPR=3.3%

WebGoat.NET

WebGoat.NET contains generic security flaws that apply to most web applications. It also contains lessons that specifically pertain to the .NET framework in order to check the scan accuracy againt a checklist.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=92%

FPR=2.1%

DSVW

Damn Small Vulnerable Web (DSVW) is a deliberately vulnerable web application written in under 100 lines of code, created for educational purposes. It supports majority of (most popular) web application vulnerabilities together with appropriate attacks.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=91,4%

FPR=0.3%

DVGA

Damn Vulnerable GraphQL Application is an intentionally vulnerable implementation of Facebook's GraphQL technology, to learn and practice GraphQL Security.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=96,9%

FPR=0.7%

Security Knowledge Framework Labs

These labs are correlated to knowledge-base id's which are on their place again correlated to security controls such as from the ASVS or NIST, etc.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=96,3%

FPR=1.0%

DVNA

Damn Vulnerable NodeJS Application (DVNA) is a simple NodeJS application to demonstrate OWASP Top 10 Vulnerabilities and guide on fixing and avoiding these vulnerabilities. The fixes branch will contain fixes for the vulnerabilities.

The application is powered by commonly used libraries such as express, passport, sequelize, etc.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=0.3%

DVIA

Damn Vulnerable iOS App (DVIA) is an iOS Objective-C application that is damn vulnerable. This application covers all the common vulnerabilities found in iOS applications (following OWASP top 10.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=1.0%

DVIA-v2

DVIA-v2 is mainly re-written in Swift along with additional vulnerabilities respect then DVIA. Its main goal is to provide a platform to mobile security enthusiasts/professionals or students to test their iOS penetration testing skills in a legal environment.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=0.0%

DVAC

Damn Vulnerable Android Components (DVAC) is an educational Android application intentionally designed to expose and demonstrate vulnerabilities related to various Android components such as Activities, Intents, Content Providers, and Broadcast Receivers. It is structured as a password manager application to manage and store passwords securely.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=1.7%

Damn Vulnerable Bank

Damn Vulnerable Bank is designed to be an intentionally vulnerable android application. All the details are documented in the guide, here.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=97%

FPR=1.9%

OVAA

OVAA (Oversecured Vulnerable Android App) is an Android app that aggregates all the platform's known and popular security vulnerabilities.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=97%

FPR=1.4%

Allsafe - Android

Allsafe is an intentionally vulnerable Android application designed for security enthusiasts, pentesters, and developers to learn about Android application security. Unlike typical CTF-style apps, Allsafe simulates a real-world application using modern libraries and technologies, providing a practical learning experience for identifying and exploiting Android vulnerabilities.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=96%

FPR=1.1%

JBoss Legacy

JBoss Application Server 6.0-final community edition (now discontinued) is still suddenly used as example of big, vulnerable component. It was over 750 KLOC and 10K files and had about 1000 Security vulnerabilities and 500 Dead Code/ Best Practices violations in its legacy JAVA 6 code.

Open

Static Reviewer (SAST) correlated to Dynamic Reviewer (DAST) reaches:

TPR=98%

FPR=1.0%