This article discusses vulnerability scanning tools relevant to securing modern web applications, so we’re not talking about network security scanners that find network vulnerabilities such as open ports or exposed operating system services. When pointed at a website or application, network scanners can only identify a handful of external application security issues like web server misconfigurations or outdated server software, making up a tiny proportion of what a dedicated web vulnerability scanner can find.
What is a web vulnerability scanner?
Web vulnerability scanners are used to automatically test running applications for security vulnerabilities. This approach is called dynamic application security testing, or DAST, and since web applications make up the vast majority of today’s business software, web security scanners are also called DAST tools.
At the most basic level, a web vulnerability scanner interacts with a website, application, or API in similar ways that a human user or interfacing external system would. However, instead of simulating valid and expected operations, the tool simulates (safely) the actions of an attacker who is trying to find security flaws and exploit them to extract sensitive data or gain unauthorized access. You can think of a DAST scanner as an automatic penetration tester who works extremely fast, never gets tired, and has a wider arsenal of tricks than any individual tester.
Vulnerability scanning examines web applications from the outside without requiring source code access or any knowledge of their internal workings, so it’s also referred to as black-box security testing. Professional DAST tools are extremely versatile and can cover many use cases across information security and application security, from vulnerability assessments and automated penetration testing to dynamic testing at multiple points in the software development lifecycle.
How does vulnerability scanning work?
There are many vulnerability scanners out there, and each one will be slightly different in how it does things and what functionality it provides besides actual scanning, but there are three broad stages to any web application scanning process:
- Pre-scan: Before testing, you need to know what to test. This phase can include discovery, crawling, and scan target selection and prioritization.
- Vulnerability scanning: The scanner performs passive and active security checks on selected targets and returns scan results. This is typically the only functionality provided by pentesting tools and open-source scanners.
- Post-scan: Going from scan results to remediation decisions is where actual security improvements are made. This phase can include vulnerability management, workflow integrations, and fix retesting.
There are many ways to categorize vulnerability scans (see Types of vulnerability scans below), but the general process is for the scanner to send HTTP requests to a target URL, inserting test values (payloads) into identified parameters and then observing how the application reacts. In the most basic case, this could mean trying out various form values to see if the application is vulnerable to an injection attack like SQL injection or cross-site scripting (XSS). For each parameter on each page, a good scanner will test for multiple vulnerabilities, often trying out multiple payloads for each one. This gives you a way to safely and extremely quickly simulate cyberattacks and imitate the potential actions of malicious hackers trying to compromise your systems.
To add an extra layer of complexity, practically all web-facing business apps require authentication to access any valuable functionality, so authenticating the scanner is another prerequisite step in the vulnerability scanning process. Fully automated vulnerability scanning requires automated authentication, which is only possible with more advanced DAST tools.
What’s the difference between security weaknesses (CWE) and vulnerabilities (CVE)?
When it comes to vulnerabilities, terminology can get a little fuzzy. Strictly speaking, CWEs are potential weaknesses, while CVEs are reported vulnerabilities in specific products. The Common Weakness Enumeration (CWE) catalog lists software and hardware security weaknesses that could result in vulnerabilities if implemented in production. The Common Vulnerabilities and Exposures (CVE) database lists confirmed and publicly reported security defects.
In practice, it’s common to call any identified security weaknesses a vulnerability, especially when talking about security issues that have been verified and confirmed, whether manually or automatically.
How are vulnerabilities identified?
Any decent vulnerability scanner should be able to find both CWEs (security weaknesses in code that could result in new vulnerabilities) and CVEs (known vulnerable products and components), as well as security issues such as misconfigurations that don’t directly result from insecure code. Each class of security flaws requires a different approach to identify as many real issues as possible while avoiding false positives.
The ability to automatically find new vulnerabilities is what makes DAST tools unique among vulnerability scanners. The scanner needs to have an extensive collection of active security checks that allow it to probe for weaknesses (Invicti DAST has over a thousand), but it also needs smart and reliable ways of identifying vulnerable behaviors triggered by its mock attacks. Some vulnerabilities may be identified directly in server responses to test requests, while others will require indirect or out-of-band observation.
Application behaviors in response to testing can be ambiguous, so finding a way to automatically verify findings has been the holy grail of vulnerability scanning. The Invicti platform uses proof-based scanning to safely exploit many common vulnerabilities and extract proof that the issue is real and remotely exploitable. This clearly shows which vulnerabilities are definitely not false positives and can go straight to remediation.
Finding CVEs is a bit different because a CVE corresponds to a piece of software with a known vulnerability, so you’re looking for that component rather than probing for weak spots. To find a CVE, the vulnerability scanner needs two things: a list of vulnerable components to look out for and a way to identify application components for checking. The Invicti platform has its own vulnerability database, updated weekly with the latest CVEs, and a fingerprinter that lets it efficiently identify components to check against the database. This dynamic SCA functionality is augmented by tech stack analysis to flag outdated products.
Last but not least are passive security checks to find such crucial gaps as missing security headers and other misconfigurations. Having an automated scanner to check things like CSP rules or HSTS headers across thousands of pages is invaluable to save time and sanity on manual verification.
Some CVEs have their own additional active security checks on the Invicti platform, which is extremely useful for verifying whether a reported vulnerability is actually exploitable in your specific environment.
Types of vulnerability scans
There are several ways to categorize web vulnerability scans, but it’s worth keeping in mind that different types of scans don’t have to require separate tools. In fact, as application environments keep growing while also becoming more complex and technologically diverse, AppSec tool consolidation is becoming a major trend. An application security platform such as Invicti’s internally uses many different tools and processes to present a unified picture of your application and its security posture.
Passive vs. active vulnerability scanning
As already mentioned, the core original purpose of a web vulnerability scanner is to actively probe websites, applications, and APIs to try and uncover new vulnerabilities. Active scanning is the most difficult but also the most valuable part of application security testing, giving you a realistic security assessment of your applications in their runtime state. Passive checks, on the other hand, are used to detect many misconfigurations as well as identify vulnerable or outdated open-source libraries, application frameworks, and tech stack components.
Heuristic vs. signature-based vulnerability scanning
A closely related way to categorize vulnerability scans is by what they are looking for: suspicious behaviors or known patterns (signatures). Heuristic scanners perform security checks and analyze application reactions to detect vulnerable behaviors that may never have been observed before. A signature-based scanner, on the other hand, looks for known vulnerabilities by comparing against its internal database. What used to be separate tools can now be combined and integrated into modern AppSec platforms, as with Invicti’s combination of a heuristic scanner with dynamic SCA and outdated component analysis.
Internal vs. external vulnerability scanning
In past decades, internal and external scanning would have referred to literally scanning the internal corporate network behind a firewall versus externally scanning its outer perimeter. Today, especially in the context of application security, internal vulnerability scanning more often refers to automated testing performed while an application is still in internal development, with external scanning corresponding to testing at the production stage. Again, what used to require different scanners for each role can now be done on a single AppSec platform that integrates at multiple points into the CI/CD pipeline and general DevOps workflow.
What common vulnerabilities are detected by automated scanning?
A decent vulnerability scanner can detect hundreds of weaknesses (CWEs) and thousands of known vulnerabilities (CVEs). The most common classes of new vulnerabilities found during scanning include the following:
- Cross-site scripting (XSS): The most numerous type of web vulnerability, essentially script injection made possible by unsanitized inputs.
- SQL injection: A common vector for data breaches, caused by passing unsanitized database commands to a back-end database server.
- Directory traversal: Usually exploited in combination with other vulnerabilities, this allows attackers to access other directories on the web server.
- Misconfigurations: A catch-all term for runtime vulnerabilities caused by config-related issues such as bad or missing security headers.
- Command injection: Allows an attacker to trick the application into running operating system commands on the web server or application server.
What happens after a vulnerability scan?
Running a vulnerability scan is only the beginning. After all, the main reason you scan for vulnerabilities is to find and remediate security issues that could get you hacked if left untouched—but the actual steps you need to take can vary hugely depending on the tool, your environment, and your workflow.
Ad-hoc scanning with an inaccurate tool will typically require your security team to manually go through all the results to weed out false positives and only then triage and assign confirmed vulnerabilities for remediation. In such ad-hoc workflows, security engineers need to manually send security tickets to developers, clarify the required mitigation, monitor resolution, retest fixes, and more. This places a huge burden on the security team while also making it a potential release bottleneck when the process cannot keep up with development schedules.
To avoid these headaches, the recommended practice is to have a vulnerability management program and process, based on a reliable AppSec solution and deeply integrated into the software development lifecycle. Using the Invicti platform as an example, you can plug the vulnerability scanner directly into your Jira or other issue tracker and have developers receive automatic tickets when specific criteria are met, for example for confirmed high or critical vulnerabilities. Each vulnerability report includes full technical information and detailed remediation guidance—and thanks to proof-based scanning, everyone is confident that confirmed issues are not false positives but real vulnerabilities that need fixing.
Bottom line: Vulnerability scanning is the foundation of application security
Vulnerability scanners have evolved from basic pentesting tools to critical AppSec solutions that can run in continuous processes to help organizations take a more proactive approach to security. On the information security side, automated DAST can deliver real-time insights into your security posture, support remediation efforts, and help with risk management and compliance. At the same time, automated dynamic security testing in the development pipeline can greatly improve software security while also removing the process bottlenecks traditionally associated with security testing.
Vulnerability scanning is foundational to web application and API security—and an industry-grade DAST platform is the way to build it into your AppSec program. See how Invicti can help you level up your application security.
Frequently asked questions about vulnerability scanners
How reliable are vulnerability scanners at finding security bugs?
That depends on the quality of the specific tool and also its intended role. The latest web vulnerability scanners can reliably find the vast majority of common vulnerabilities and even test them for exploitability. Less advanced tools can struggle to access and test all parts of a modern web application, making them less reliable than dedicated solutions.
Do vulnerability scanners produce false positives?
All automatic testing can potentially produce false positives, and vulnerability scanners vary widely in the proportion of false alarms in their results. Basic scanners designed for manual testing (which includes popular open-source vulnerability scanners) may deliberately overreport potential vulnerabilities for the user to check manually. Enterprise-grade DAST tools are built for automation and use techniques such as proof-based scanning to clearly indicate which results are real and exploitable vulnerabilities.
Will different vulnerability scanners get different results?
Yes, and the differences can be extreme, depending on the tool, setup, and target environment. For example, a basic scanner that can only run unauthenticated scans may skip all but a handful of pages on a test site because it couldn’t access them or crawl them in full, so its results will only cover a tiny part of the environment. A quality DAST tool may be able to run thousands more tests in the same environment and with more accuracy, delivering far more actionable results.
Can web application vulnerability scanners scan APIs?
Yes, they can, but the level of coverage and accuracy heavily depends on the specific tool. The Invicti platform has full support for importing and testing REST, SOAP, and GraphQL APIs and can also perform REST API discovery. More basic DAST tools may be able to test some REST endpoints but lack the features for comprehensive API security testing.
The post What is vulnerability scanning and how do web vulnerability scanners work? appeared first on Invicti.
0 Comments