How a Security Department Reads an External Audit Report

A supplier’s security report does not get read the way it was written. It was written top to bottom, with an executive summary and a grade. It gets read as an adversarial sampling exercise: the reviewer is deciding whether the document can be used as evidence, and that decision is usually made before the second page.

Understanding that sequence is more useful than improving the prose.

The order of operations

1. Date. Not the report’s title, the date the measurement was taken. A document dated eight months ago describes a system that has since been deployed several hundred times. It is treated as historical, not current — and a report that carries no measurement date at all is treated as undated, which is worse.

2. Scope. Which hostnames were examined. A report on example.com while the product runs on app.example.com answers a question nobody asked. Reviewers look for the list of tested hosts early, because the rest of the document is only meaningful relative to it.

3. Method. Whether the finding came from an active probe, a passive lookup, or a self-declaration. These carry very different weight, and a report that blends them without labelling which is which forces the reviewer to assume the weakest.

4. One finding, checked. This is the step that decides the outcome. The reviewer picks an item — usually the most severe one — and verifies it independently. curl -I on the host, dig on the DMARC record, an SSL Labs run, opening the URL a finding claims is exposed.

If that check reproduces, the rest of the document inherits the credibility. If it does not, the document is set aside and the questionnaire is sent instead.

What makes a reviewer stop

A claim they can disprove in thirty seconds. The classic is a “critical exposure” on a path that returns the application’s own homepage — a soft-404 read as a file. One disproved critical does not get corrected in the reviewer’s mind; it reclassifies the whole document as unverified output.

A grade with no arithmetic. “Security score: 94/100” without the checks, weights and observations behind it is a number, not a measurement. Reviewers who cannot reconstruct a score do not argue with it — they ignore it and ask for the underlying findings.

Numbers without a source. “Vendors who do X close deals 3–4 weeks faster.” “62% of reviews fail on Y.” A security department reads uncited statistics in a supplier document as marketing that wandered into an evidence pack, and it lowers the credibility of the technical sections by association.

Absolutes. “No vulnerabilities found” is not a defensible sentence. “No finding across the 66 checks in this scope, on the date of measurement” is. The first invites the reviewer to find a counterexample; the second states what was actually observed.

A claimed equivalence to something else. A report that positions itself as a substitute for a penetration test, or implies certification by a body that issued nothing, gets read as a document that misrepresents its own nature. Everything it says about scope then becomes suspect too.

What a defensible finding looks like

The unit of trust is not the report. It is the individual finding, and it needs four elements:

• The observation, verbatim. The header as returned, the certificate chain as negotiated, the cipher suites offered, the DKIM selectors tested, the response excerpt. Not a paraphrase of the observation.
• The timestamp, on the finding itself rather than only on the cover page.
• The method, so the reviewer knows what was actually done — which host, which port, authenticated or not.
• The control reference, if the report claims a compliance mapping: NIS2 Article 21(2)(e) rather than “NIS2 compliant”.

A reviewer who can re-run a finding and get the same answer stops treating your document as a claim and starts treating it as data. That is the entire objective.

Say what was not covered

The instinct is to omit the limits. It reads as weakness in a sales document and as honesty in an evidence document, and the security department is reading the second kind.

An unauthenticated external scan does not exercise business logic, does not test authenticated flows, does not see internal networks, and does not evaluate your processes. Writing that down does not reduce the value of what was measured — it tells the reviewer that the boundary of the claim was drawn deliberately. The alternative is that they draw it for you, less generously, after finding a gap you did not mention.

The report that survives review is rarely the most impressive one. It is the one where every sentence is something the reader can check.

Related reading: red flags a CISO looks for in a vendor questionnaire, and what an external scan can and cannot prove.