Automated Penetration Testing Tools: Strengths, Limits & When You Still Need a Human
A scanner has never once talked its way past a receptionist. It has never noticed that the price field on a checkout page was a hidden integer it could set to minus one and walk away with a credit. It has never chained three low-severity findings that every tool marked as informational into a full account takeover. Yet every quarter a client sends us a 400-page automated scan report and asks why we still charge for a manual test.
The honest answer is that automated tools and manual testers are not the same job wearing two price tags. They are different instruments that fail in different places. If you buy one believing it does the work of the other, you will pass your scan and fail your breach. This guide is the version of that conversation we have in the audit room, minus the diplomacy.
What the machines are actually good at
Start by giving automation its due, because plenty of manual testers oversell their own value by pretending scanners are useless. They are not. For breadth, speed and repeatability, nothing beats a well-tuned tool chain, and no auditor worth their empanelment does a large scope without one.
The two families you will meet are SAST and DAST. SAST is Static Application Security Testing: it reads your source code without running it, looking for dangerous patterns such as unsanitised input flowing into a SQL query. DAST is Dynamic Application Security Testing: it attacks the running application from the outside, like an attacker would, without seeing the code. There is a third cousin, IAST (Interactive Application Security Testing), which instruments the running app to watch data flow from the inside, but in Indian mid-market engagements you will mostly deal with SAST and DAST.
Here is where each earns its keep.
| Tool type | What it does well | What it cannot see |
|---|---|---|
| SAST (static / code) | Finds insecure patterns early, covers every code path, runs in CI on every commit, catches hardcoded secrets and weak crypto calls | Runtime behaviour, business logic, whether the vulnerable code is even reachable, config and infrastructure gaps |
| DAST (dynamic / running app) | Confirms exploitable input points from outside, no source needed, catches server misconfig and exposed endpoints | Anything behind authentication it cannot navigate, multi-step logic flaws, race conditions, findings needing chained steps |
| Network / infra scanners | Fast CVE and missing-patch detection across thousands of hosts, TLS and open-port hygiene, version fingerprinting | Whether a CVE is actually exploitable in your context, false positives from back-ported patches, segmentation logic |
That last column is not a footnote. It is the entire reason manual testing exists. A scanner tells you a door is unlocked. It cannot tell you the door leads to the vault, or that the lock is decorative and the real entrance is round the back.
The five things a scanner will never find
After a few hundred engagements the pattern is boringly consistent. The findings that actually get organisations breached, and the findings that fail a serious audit, cluster into five categories that no automated tool reliably catches. Call them the five blind spots.
1. Business logic flaws
A scanner has no idea what your application is supposed to do. It does not know that a loan-disbursal API should never approve an amount above the sanctioned limit, or that a discount coupon should apply once. We routinely find flows where a user can skip the payment step and still receive the goods, or replay a one-time-password because the server never invalidates it. These are the vulnerabilities that make the newspapers, and they are invisible to SAST and DAST because the code is doing exactly what it was told to do. It was just told the wrong thing.
2. Chained low-severity findings
Tools rate findings in isolation. A verbose error message here, a predictable object ID there, a missing rate limit somewhere else, each rated low or informational and each ignored by a busy team. A human links them: read the error to learn the ID format, enumerate the IDs because there is no rate limit, and pull every customer record. Three lows become one critical. No scanner does this arithmetic.
3. Authentication and authorisation logic
DAST can log in if you give it credentials, but it cannot reason about who should be allowed to do what. Broken object-level authorisation, where user A can read user B data by changing an ID in the URL, remains the single most common serious finding we report, and automated coverage of it is poor because the tool cannot tell an intended cross-tenant access from a bug.
4. Context and false positives
The inverse problem. Scanners cry wolf. A network scanner flags a CVE on a service where the patch was back-ported by the OS vendor, so the version string looks vulnerable but the code is not. Hand a raw 400-finding scan to an examiner and you will spend the audit defending noise instead of fixing risk. Triage is a human skill and it is most of the value.
5. The human layer
Social engineering, physical access, a helpdesk that resets passwords without verification. CERT-In and RBI both expect this to be in scope for a real assessment, and no tool tests it at all.
What actually happens in the room
A payments company in Pune, PCI DSS in scope, came to us confident. Their quarterly ASV scan (Approved Scanning Vendor, mandated under PCI DSS Requirement 11.3.2) was clean and green. Their internal SAST pipeline broke the build on anything above medium. On paper, unbreakable.
On day two of the manual test, one of our team noticed the refund endpoint accepted a negative amount. The scanner had tested that endpoint, confirmed it required authentication, found no injection, and moved on. It never occurred to the tool to send minus five thousand rupees. The negative refund credited the attacker card and debited the merchant float. From there, the transaction IDs were sequential, so every refund in the system could be enumerated and reversed. One business-logic flaw, invisible to two tool chains, would have drained the settlement account before the next ASV scan even ran.
That is the shape of nearly every real finding. The tools did their jobs. The jobs just did not include the flaw that mattered.
Where the Indian regulators actually stand on this
This is not merely good practice, it is written into the frameworks you are audited against. Every major Indian regulator expects both automated coverage and manual expert testing, and the seniority of the tester is increasingly explicit.
| Framework | What it expects on pentesting | The automation-only trap |
|---|---|---|
| RBI Cyber Security Framework (2016) & Master Direction on IT Governance (2023) | Periodic VAPT of critical systems, with vulnerability assessment and penetration testing named as distinct activities | A VA scan alone does not satisfy the PT limb; examiners ask specifically for manual testing evidence |
| CERT-In (2022 directions & empanelment) | Security audit by CERT-In empanelled auditors; manual verification of findings is part of the empanelment methodology | Tool output without auditor validation is not an empanelled audit report |
| PCI DSS v4.0 | Req 11.3.1 internal and 11.3.2 external penetration testing at least annually and after significant change; segmentation testing under 11.4 | ASV scans (11.3.2 quarterly) are separate from and do not replace the annual penetration test |
| SEBI CSCRF (2024) | VAPT for regulated entities on a defined cadence with reporting to the board and SEBI | Self-attested scans without qualified tester sign-off do not meet the assurance bar |
| IRDAI & DPDP context | Security safeguards proportionate to risk; VAPT expected for systems holding personal data | Personal-data breach liability is not reduced by a clean scan if logic flaws remained untested |
Notice the recurring word: penetration testing is named as a distinct activity from vulnerability assessment. Regulators drew that line deliberately. A vulnerability assessment is largely what a scanner produces, a list of potential weaknesses. A penetration test is what a human does with that list, proving which weaknesses are actually exploitable and chaining them. When your RBI examiner asks to see your VAPT report, they are asking for both halves. Submitting only the automated half is the most common gap we see, and it is an easy finding for them to raise.
So what should you actually run, and when
The mature answer is not tools versus humans. It is tools continuously, humans periodically and after change, with the humans reviewing what the tools produce. Here is the cadence that survives an audit and a real attacker.
| Activity | Frequency | Who / what |
|---|---|---|
| SAST in CI pipeline | Every commit / build | Automated, developers triage |
| DAST against staging | Weekly to monthly | Automated, security team triage |
| Network / infra vulnerability scan | Monthly (quarterly minimum for PCI ASV) | Automated ASV / internal scanner |
| Full manual VAPT of critical apps | Annually and after significant change | Senior CERT-In empanelled tester |
| Business-logic and auth deep dive | Annually, part of manual VAPT | Human, scenario-driven |
| Social engineering / red team | Annually or as risk dictates | Human, scoped exercise |
The cost conversation, honestly
Clients push back on manual pricing because a tool licence looks cheap by comparison. The comparison is real, but it is not the right one. You are not choosing between spending on a tool or a tester. You are choosing what residual risk you carry into your next incident.
| Option | Typical Indian cost range | What you are really buying |
|---|---|---|
| Commercial DAST / SAST licence | INR 3 to 15 lakh per year | Continuous breadth, false-positive noise, no logic coverage |
| Automated-only VAPT report | INR 40,000 to 1.5 lakh per app | A scan with a cover page; rarely survives a serious examiner |
| Senior manual VAPT (empanelled) | INR 1.5 to 6 lakh per app depending on scope | Exploited findings, chained attacks, logic and auth coverage, audit-ready sign-off |
| A material breach | INR crores plus DPDP penalties up to 250 crore | The bill you avoid by paying the one above |
The DPDP Act penalty ceiling of 250 crore rupees for failure to take reasonable security safeguards changes the arithmetic permanently. A regulator or the Data Protection Board asking whether your security safeguards were reasonable will not be impressed that you owned a scanner. They will ask whether a competent tester actually examined the systems holding personal data. That question has a yes or no answer, and the scanner cannot give it for you.
A checklist before you sign off on any pentest
Whether the work is in-house or outsourced, run the engagement against this list. If any item is a no, you have an automated scan wearing a penetration-test label, and an examiner will spot the difference.
- Does the report distinguish vulnerability assessment findings from penetration testing findings, as RBI expects?
- Is there evidence of manual verification for each high and critical finding, not just tool output?
- Were business-logic flows tested by scenario (payment bypass, negative amounts, replay, workflow skipping)?
- Was broken object-level authorisation tested by manipulating identifiers across accounts?
- Were low and informational findings reviewed for chaining into a higher-severity attack path?
- Are false positives triaged out with a stated reason, so the examiner sees signal not noise?
- For PCI scope, are the annual penetration test and the quarterly ASV scan both present and clearly separate?
- Is the tester CERT-In empanelled where empanelment is required, with named individuals and credentials?
- Does the report include a retest confirming the fixes actually closed the findings?
The bottom line
Buy the tools. Run them constantly. Break the build on real findings and scan every host every month. Automation is the only sane way to cover breadth at speed, and any tester who dismisses it is showing off. But do not mistake coverage for assurance. The scanner tells you which doors are unlocked. It takes a human to notice the door leads to the vault, to walk in, and to tell you honestly what they found on the other side.
At CyberSigma we run the tool chains and then do the part they cannot, as senior CERT-In empanelled auditors and PCI QSAs who still sit in the room and test by hand. If your last pentest was 400 pages of green with no chained findings and no logic testing, that is worth a second look before your next examiner takes one.
FAQs
Is a vulnerability assessment the same as a penetration test?
No, and Indian regulators treat them as distinct. A vulnerability assessment is largely automated, producing a list of potential weaknesses. A penetration test is a human proving which of those weaknesses are actually exploitable, chaining low findings into real attack paths, and testing business logic no scanner understands. RBI and PCI DSS both expect both activities, not one standing in for the other.
Can automated tools satisfy my PCI DSS penetration testing requirement?
Not on their own. PCI DSS v4.0 Requirement 11.3.1 and 11.3.2 mandate penetration testing at least annually and after significant change, performed to a documented methodology. The quarterly ASV scan under 11.3.2 is a separate, additional requirement. An automated scan is a vulnerability assessment, not the penetration test the standard demands.
What is the difference between SAST and DAST?
SAST, Static Application Security Testing, reads your source code without running it and finds insecure coding patterns across every code path. DAST, Dynamic Application Security Testing, attacks the running application from the outside like an attacker, without needing the source. They are complementary: SAST catches code-level flaws early, DAST confirms exploitability at runtime, and neither reliably finds business-logic or authorisation flaws.
Why do scanners miss business-logic vulnerabilities?
Because a scanner has no concept of what your application is meant to do. It cannot know that a refund should not accept a negative amount, or that a coupon should apply once. The vulnerable code runs exactly as written, so nothing looks anomalous to a tool. Only a human who understands the intended workflow can spot that the logic itself is the flaw.
How often should we run manual penetration testing?
Run automated SAST and DAST continuously and network scans monthly. Commission a full manual VAPT by a senior tester at least annually and after any significant change to the application or infrastructure, which is what RBI, PCI DSS and SEBI CSCRF expect. Business-logic, authorisation and social-engineering testing should be part of that annual manual engagement.
Does the DPDP Act change how we should approach pentesting?
Yes. With penalties up to 250 crore rupees for failing to take reasonable security safeguards, the question shifts from whether you scanned to whether your safeguards were actually reasonable for systems holding personal data. Demonstrating that a competent, ideally CERT-In empanelled, tester examined those systems by hand is far stronger evidence of reasonable safeguards than a clean automated report.
Liked the post? Share on:
Leave A Comment