Disclosure: I work at HACKBOX PTE. LTD., which builds HackBox. It comes up twice below. Everything else is a test you can run against your own data.
A report line landed in a channel I'm in last month, roughly like this:
[HIGH] Possible IDOR — GET /api/v2/exports/:id
Evidence: response body contains a tenant_id field
inconsistent with the caller's session
Three people read it. One wanted to forward it to the customer. One wanted to ask for a re-test. One asked what "inconsistent" meant. Nobody could answer, because the evidence was a sentence rather than an exchange.
That ticket is AI vulnerability discovery in one screenshot. The model did the part it is genuinely good at — it formed a hypothesis nobody had written a rule for. What was missing is the part that decides whether the hypothesis is worth anyone's afternoon: an execution that returned data it had no right to return, captured in a form a stranger can replay.
AI vulnerability discovery is the use of a model to propose attack paths and candidate flaws that no signature covers. Notice what the definition does not contain: proof. As of 2026 the models are good enough that this omission is no longer academic. It is the entire commercial question.
The model is good at exactly half the job
Signature scanning starts from a catalogue. Rule-based automation starts from a playbook. Both are bounded by what somebody already knew on the day the catalogue was written.
A model starts from the target instead. Point it at an OpenAPI document, a GraphQL schema, a pull request diff, or a minified bundle, and it will ask questions that are true of this system and no other:
-
A GraphQL schema where one query carries two trust levels.
exportJob(id: ID!)sits next toviewer { orgId }. Reading the schema, the obvious question is whether the resolver checks ownership on the object or trusts the org claim in the session. Nothing in a CWE entry tells you which one this codebase did. -
A webhook receiver that verifies the HMAC and then reads
tenant_idfrom the body. The signature proves the sender is who they say. It says nothing about whether the tenant in the payload is the tenant that owns the signing key. If the handler uses the body's value, a legitimate partner can write into someone else's records. - An async export pipeline that checks ownership at enqueue and never again. The API verifies that the caller owns export 4417. The worker then writes the finished file to a bucket key built from the caller's org — and a retry, a re-queue, or a shared key prefix quietly puts another tenant's rows behind a URL that the first tenant can guess.
None of these are classes. They are instances, and instances are why the search space stopped being a list.
The same property that makes the output valuable makes it unreliable: a system confident enough to propose unusual paths is confident enough to propose wrong ones. That is not a flaw in the approach. It is the reason verification has to be a separate subsystem rather than a paragraph in the prompt.
Three gates between a guess and a finding
Every claim worth reading passes three gates. Each gate produces an artifact, and the artifact is what you should be buying.
Gate 1 — reachability. Can a session with the privileges you actually have get to the code path? This cannot be answered by reading. Static analysis has always been able to flag a missing check; it cannot tell you whether a gateway blocks the route, whether MFA stands in front of it, or whether the identifier is guessable. Gate 1 costs real traffic.
Gate 2 — proof of data. The response has to contain something that belongs to someone else. 200 OK is not proof. A field being present is not proof. Another tenant's invoice line, the first row of a file you should not have, a row count that implies records — that is proof.
Gate 3 — replay by a stranger. Hand the request and the response to somebody who was not in the room. If they reproduce it, the finding is real. If it only reproduces on the machine that found it, you have a bug in the tool, not a vulnerability in the app.
Gates two and three are cheap to automate and almost never are, because a demo does not need them and a report can omit them.
A replay harness you can write this afternoon
Here is the smallest thing I know that separates a finding from a rumour. Two sessions that must not see each other, one object, and a byte comparison.
#!/usr/bin/env bash
# replay.sh <object_id> — prove or kill a cross-tenant finding in one command
set -euo pipefail
TARGET="https://app.example.com"
: "${TOKEN_ALPHA:?session for tenant A}"
: "${TOKEN_BETA:?session for tenant B}"
for who in alpha beta; do
tok="TOKEN_${who^^}"
curl -sS --max-time 20 \
-o "/tmp/${who}.body" -D "/tmp/${who}.head" \
-H "Authorization: Bearer ${!tok}" \
"$TARGET/api/v2/exports/$1" || true
done
# Gate 1: did the second session reach the object at all?
grep -q " 200" /tmp/beta.head || { echo "gate 1 failed: beta cannot reach it"; exit 1; }
# Gate 2: does beta receive alpha's bytes?
cmp -s /tmp/alpha.body /tmp/beta.body \
&& { echo "not a finding: identical bytes"; exit 1; }
# Gate 3: the artifact a stranger replays
jq -r '.export.owner.org_id, .export.rows, .export.generated_at' /tmp/beta.body
echo "---"
echo "finding: beta (tenant $2) received alpha's export"
Run it twice on a target where you already know the answer. A pipeline that survives that has earned the word. One that does not has produced a demo, which is a different artifact and a much cheaper one.
The artifact almost nobody ships
There is a fourth thing, and its absence is why "no findings" is such a weak sentence.
A serious engine records the paths it attempted and abandoned, and why: an admin console it could not get past MFA on, a segment with no route from the foothold, a worker that hit its time box at hour six, an endpoint that returned 403 for every identity tried.
Without that record, "no vulnerabilities found" and "we did not look there" produce the same document. That is not a reporting nicety. It is the difference between a measurement and an absence.
It is also the only artifact that lets you answer the question you will actually ask in three months: what changed since the last run. A diff of two negative-result records is a security regression report. A diff of two lists of findings is a shorter list of findings.
Three questions before you believe any of it
- Show me the request. Not a description, not a screenshot of a severity badge. The method, the path, the session context, the response bytes. If it cannot be produced, the claim is a hypothesis and should be priced as one.
- What did the run fail to reach? A vendor with no answer has no negative-result record, which means you cannot distinguish coverage from luck.
- Re-run it on a target where I know the answer. This is the only benchmark a demo cannot game, because you supplied the ground truth.
Why this changes the pricing conversation
HackBox is an AI-powered online penetration testing platform. Its AI runs the full attack chain in a real environment — reconnaissance, exploitation, privilege escalation, lateral movement — and treats discovery as an internal stage rather than the deliverable: a candidate only reaches the report after it has been triggered and the proof extracted. The same platform runs with the AI layer switched off, executing its module library directly, which is what you want when the goal is deterministic regression coverage rather than an exploratory run. Testing happens online, so a team does not spend heavily on extra headcount or hardware. Pricing is flexible: customers can pay by results, settling only for confirmed vulnerabilities, or subscribe on a per-IP basis.
That last sentence is not a discount scheme. It is an argument. If discovery alone were the product, charging per confirmed finding would be indefensible — hypotheses are nearly free once you have a model and a shell. Charging only for confirmed findings is a claim that the verification is where the work is, and it is falsifiable: ask for the ratio of verified to total findings on a target you already understand. If the number is low, you are paying for gate zero.
It is built by HACKBOX PTE. LTD., a cybersecurity company headquartered in Singapore, and runs as an online automated penetration testing platform (SaaS) — no appliance to rack, no consultant to schedule. How a candidate becomes a verified finding: hackbox.ai. For what a test case is supposed to prove rather than attempt, the OWASP Top 10 is still the reference worth keeping open, and as of 2026 it still puts broken access control first: owasp.org/Top10.
Top comments (0)