What an External Scan Can and Cannot Prove in a Vendor Questionnaire

A supplier questionnaire mixes three kinds of question in one document, and they need three different kinds of answer. Treating them uniformly is what turns a one-hour task into a three-week one.

• Observable from outside. What a third party can measure without credentials.
• Internal and attestable. True of your systems, but only demonstrable from inside — a log, an export, a screenshot.
• Organisational. A process, a policy, a contract, a role.

An unauthenticated external scan is evidence for the first category. It is a supporting artefact for parts of the second. It is silent on the third. Mapping the questionnaire onto that split before answering tells you which items you can close today and which need a human.

What it closes with evidence

These are questions where the observation is the answer, because anyone can reproduce it from outside:

• Transport security. Protocol versions accepted, cipher suites negotiated, certificate chain and expiry, HSTS presence and max-age. See TLS configuration for SaaS.
• HTTP security headers. Content-Security-Policy, X-Content-Type-Options, Referrer-Policy, frame protections — as actually returned by the host, which is frequently not what the configuration file says. See security headers and NIS2.
• Email authentication. SPF, DKIM selectors, DMARC policy and alignment mode as published in DNS. See DMARC, SPF and DKIM.
• DNS posture. DNSSEC, CAA records, dangling records that enable subdomain takeover.
• Exposed artefacts. Configuration files, repository metadata, database dumps and status endpoints reachable without authentication — provided the finding is confirmed by content signature rather than by status code.
• Publicly disclosed component versions, where a version is actually advertised. Absence of a version banner is not absence of a vulnerable component.

For each of these, the honest form of the answer is the observation plus its date, not an adjective. “TLS 1.2 and 1.3 only; TLS 1.0/1.1 refused; observed 2026-09-09” is evidence. “Strong encryption in transit” is a claim.

What it supports without closing

Some questions have an externally visible symptom and an internal reality. The scan is corroboration, not proof.

• “Do you run continuous vulnerability management?” A dated series of external scans shows a cadence exists for the external surface. It says nothing about your dependency scanning, your internal hosts, or your remediation SLA. Pair it with the ticket export.
• “Are systems patched?” An externally advertised version can contradict a claim, but cannot confirm one — most components do not advertise a version at all.
• “Is your attack surface inventoried?” Enumerating hostnames from outside shows what a stranger can find. Your inventory has to be at least that complete; the scan sets a floor, not a ceiling.
• “Do you enforce MFA?” Externally you may observe an SSO redirect. Enforcement, coverage and break-glass exceptions live in the identity provider’s configuration.

What it cannot touch

No amount of external probing produces evidence here, and claiming otherwise is the fastest way to lose a reviewer:

• Business logic. Broken object-level authorisation, price manipulation, workflow bypasses, tenant isolation failures behind login. These require authenticated, manual testing with a scope document — this is what the penetration test is for, and why the two artefacts are requested together rather than interchangeably.
• Anything behind authentication. The admin surface, the API under a valid token, per-role authorisation.
• Data handling. Encryption at rest, key management and rotation, retention and deletion, sub-processor flows.
• Internal architecture. Network segmentation, backup integrity and restore testing, logging coverage.
• The organisation. Access reviews, onboarding and offboarding, incident response rehearsals, supplier due diligence, board oversight.
• Certification status. A scan produces a measurement. It does not produce an ISO 27001 certificate or a SOC 2 report, and no automated tool issues one — those come from an accredited body after an audit.

Answering the split

The practical consequence is a three-column pass over the questionnaire before writing anything.

Column one: items the scan answers. Attach the report section, cite the observation, state the measurement date.

Column two: items where the scan corroborates. Answer from the internal system of record and attach the scan as supporting material, clearly labelled as external evidence only.

Column three: items that need a document — a policy, an export, a certificate, a contract clause. These are the ones worth the calendar time, and they are also the ones that are the same in every questionnaire, which makes them worth writing once and reusing.

The reason to be explicit about column three rather than stretching column one to cover it is not modesty. A reviewer who finds an external scan cited as evidence for an access-review question stops trusting the citations in column one, where they were sound. Scope discipline is what makes the strong part of the answer usable.

Related reading: how a security department reads an external audit report, and the 50-point vendor security assessment checklist.