Checks

62 checks in 13 packs.

shipped

Generated from the CLI's own description of its checks (methodology 0.3-draft, CLI 0.3.0). critical: a fail caps the pack grade at D. advisory: reported, never scored. candidate: reported now, scored only after a methodology bump promotes it. When a pack runs is on Packs; what a verdict scores is on Verdicts.

core pack

Hygiene any MCP server should pass. Needs only a command or a URL.

Check	Weight	Decided by	Scope
`protocol` Valid MCP handshake and tool schemas	scored	protocol	server
`tool-poisoning` No tool-poisoning indicators in tool metadata	critical	static	server (what it declares, not what it does)
`robust-input` Survives malformed tool input	scored	protocolresponse	server
`secret-echo` No environment secret echoed in responses	critical	response	server
`refusal-quality` Refusals say what to do next	advisory · not scored candidate	response	server (how it words a refusal)
`instructions-hygiene` Server instructions name only tools that exist	advisory · not scored candidate	static	server (what its instructions say)

db pack

Starts your server once per database role against a seeded synthetic Postgres.

Check	Weight	Decided by	Scope
`read-only` Read-only by default	scored	db-stateresponse	server (tested as a role that can write, so the DB cannot be the reason a write fails)
`write-gate` Write approval gate	advisory · not scored	db-stateresponse	server
`rls` Row-level security respected	critical	response	database policy, tested through the server: does the server defeat it?
`mutating-sql` Mutating SQL blocked	critical	db-stateresponse	server (tested as a role that can write)
`pii` Personal data masking respected	critical	response	database grants, tested through the server: does the server defeat them?
`pii-masked` Masked data reachable	scored	response	server (does it let a masked view through, or does it deny everything?)
`db-rowcap` Row cap	scored	response	server
`db-timeout` Statement timeout	scored	response	server
`audit` Every call logged	advisory · not scored	response	server
`db-privilege` Connects as the database role it was given	advisory · not scored candidate	db-stateresponse	server (does each instance connect as exactly the role in its {dsn}, with that role's attributes?)

sqlite pack

The same ideas on an embedded SQLite file the harness seeds.

Check	Weight	Decided by	Scope
`sqlite-read-only` SQLite: read-only by default	scored	db-stateresponse	server (tested through its SQL tool against a seeded SQLite file; the engine itself is not what is graded; SQLite only in this release, DuckDB is planned)
`sqlite-mutating-sql` SQLite: mutating SQL blocked	critical	db-state	server (tested through its SQL tool against a seeded SQLite file; the engine itself is not what is graded; SQLite only in this release, DuckDB is planned)
`sqlite-file-read` SQLite: no reading of other files	critical	responsedb-state	server (tested through its SQL tool against a seeded SQLite file; the engine itself is not what is graded; SQLite only in this release, DuckDB is planned); the database file the server was given is the only file it should read
`sqlite-file-write` SQLite: no writing of other files	critical	fs-observer	server (tested through its SQL tool against a seeded SQLite file; the engine itself is not what is graded; SQLite only in this release, DuckDB is planned); the database file the server was given is the only file it should write
`sqlite-extension-load` SQLite: no loading of extensions	critical	response	server (tested through its SQL tool against a seeded SQLite file; the engine itself is not what is graded; SQLite only in this release, DuckDB is planned); decided from the error text, not from observed code execution
`sqlite-rowcap` SQLite: row cap	scored	response	server (SQLite only in this release; DuckDB is planned)
`sqlite-timeout` SQLite: query timeout	scored	response	server (SQLite only in this release; DuckDB is planned)

fs pack

For tools that take a file path: reads and writes outside the sandbox root, links, size cap.

Check	Weight	Decided by	Scope
`fs-read-escape` Reads stay inside the allowed root	critical	responsestatic	server (path validation, with a harness-owned root)
`fs-write-escape` Writes stay inside the allowed root	critical	fs-observerstatic	server (path validation, with a harness-owned root)
`fs-link-escape` Links cannot lead out of the allowed root	critical	responsefs-observerstatic	server (does it resolve links before checking the path?)
`fs-size-cap` Large file reads are capped	scored	response	server (response size limit)

net pack

For tools that take a URL: loopback and scheme probes against a listener the harness owns.

Check	Weight	Decided by	Scope
`net-loopback-ssrf` Refuses loopback targets (SSRF)	critical	listenerresponse	server (does a URL tool refuse loopback addresses? tested against a harness-owned loopback listener)
`net-scheme-allowlist` Only http(s) schemes accepted	critical	responselistener	server (does a URL tool accept file:, ftp:, gopher:, dict: or data: URLs?)
`net-caps` Response size and time are capped	scored	responselistener	server (size and time limits on a fetched response; measurable only when the server already reached the harness listener)
`net-redirect` Redirects are re-validated	scored	listenerresponse	server (does a URL tool re-validate the target of a redirect? observable only when the server already fetches the harness listener)

git pack

For tools that take a repository path: repository scope and argument injection.

Check	Weight	Decided by	Scope
`git-repo-scope` Only the configured repository is reachable	critical	responsefs-observer	server (does it restrict which repositories a call may name?)
`git-arg-injection` Arguments cannot become git options	critical	fs-observerresponse	server (validation, or a `--` separator, before git runs)

exec pack

Opt-in. For tools that run commands; runs only with --active and only harness-owned programs. Not in the standard profile.

Check	Weight	Decided by	Scope
`exec-injection` Arguments are not interpreted by a shell	critical	fs-observerresponse	server (no shell in argument passing; shell-style tools that take one command line are not tested, they are a shell by design)
`exec-cwd` Programs run inside the configured root	scored	fs-observer	server (working directory of started programs)
`exec-timeout-cap` Run time and output are capped	scored	processresponse	server (time and output limits on started programs)
`exec-allowlist` Only allowed programs run	advisory · not scored	fs-observerresponse	server (which programs the tool will start)

http pack

For servers reached by URL: origin, authentication, protocol version, sessions, CORS, bind address and TLS.

Check	Weight	Decided by	Scope
`http-origin` Origin header validated	critical	http	server as deployed (the endpoint you configured, with the headers you configured)
`http-auth` Rejects requests without credentials	scored	http	server as deployed (is the configured credential actually required?)
`http-protocol-version` Unsupported protocol version rejected	scored	http	server as deployed (MCP-Protocol-Version validation)
`http-session` Session ids required, validated and unguessable	scored	http	server as deployed (Mcp-Session-Id handling)
`http-session-binding` A session id is bound to the caller that opened it	advisory · not scored candidate	http	server as deployed (Mcp-Session-Id handling across two callers; MCP authorization specification, session binding)
`http-cors` No permissive CORS	scored	http	server as deployed (answer to a cross-origin preflight)
`http-bind` Listens on loopback only	advisory · not scored	process	environment (the machine the server runs on, when the configured URL is loopback; not examined when Sealmark started the server itself)
`http-tls` Uses TLS off the local machine	advisory · not scored	static	environment (the URL as configured)

effects pack

Declared against done: whether a tool's annotations match what changed.

Check	Weight	Decided by	Scope
`annotation-honesty` Read-only tools change nothing	critical	fs-observer	server (annotations vs what the sandbox observed during the harness's own benign calls)
`openworld-honesty` Closed-world tools stay on the machine	scored	listener	server (annotations vs what the sandbox observed during the harness's own benign calls)
`undeclared-writes` Unannotated tools leave the sandbox unchanged	advisory · not scored	fs-observer	server (annotations vs what the sandbox observed during the harness's own benign calls)

custom pack

Your own probes from sealmark.yaml. They add checks of their own; there are none built in. Not in the standard profile.

scope pack

Not in the standard profile.

Check	Weight	Decided by	Scope
`scope-escape` Identifiers outside the declared scope are refused	critical candidate	upstreamresponse	server (does it refuse every identifier outside scope.allow, in every spelling?)

sql-guard pack

Not in the standard profile.

Check	Weight	Decided by	Scope
`sql-guard-statements` SQL guard: only read statements reach the backend	critical candidate	upstream	server (judged by what reached the harness's upstream stub, not by the server's reply; no SQL is parsed)
`sql-guard-references` SQL guard: out-of-scope references blocked in every position	critical candidate	upstream	server (judged by what reached the harness's upstream stub, not by the server's reply; no SQL is parsed)
`sql-guard-rowcap` SQL guard: row cap applied before the backend	scored candidate	upstream	server (judged by what reached the harness's upstream stub, not by the server's reply; no SQL is parsed); read from the forwarded text, a heuristic

upstream pack

Not in the standard profile.

Check	Weight	Decided by	Scope
`upstream-credential` Each caller's own credential reaches the upstream	critical candidate	upstream	server (what the harness's upstream stub received from the server, and what the server printed or relayed)
`upstream-credential-echo` Replies never contain a caller's credential	scored candidate	response	server (what the harness's upstream stub received from the server, and what the server printed or relayed)
`log-secret` The server's log never contains a caller's credential	scored candidate	log	server (what the harness's upstream stub received from the server, and what the server printed or relayed)
`upstream-error-leak` Upstream error bodies are not relayed with the credential	scored candidate	upstreamresponse	server (what the harness's upstream stub received from the server, and what the server printed or relayed)
`upstream-slow` A slow upstream does not hang the server	scored candidate	upstreamresponse	server (what the harness's upstream stub received from the server, and what the server printed or relayed)
`upstream-token-passthrough` The caller's access token is never passed to the upstream	critical candidate	upstream	server (what the harness's upstream stub received from the server, and what the server printed or relayed)

oauth pack

Not in the standard profile.

Check	Weight	Decided by	Scope
`oauth-token-forgery` Forged and unsigned tokens are refused	critical candidate	httpauthserver	server as deployed (the bearer tokens the endpoint accepts, and what it tells a client that has none)
`oauth-claims` Expired, early, wrong-issuer and wrong-audience tokens are refused	critical candidate	http	server as deployed (the bearer tokens the endpoint accepts, and what it tells a client that has none)
`oauth-discovery` A client without a token is told where to authorize (RFC 9728)	scored candidate	http	server as deployed (the bearer tokens the endpoint accepts, and what it tells a client that has none)
`oauth-scope` Scopes are enforced and tokens are read from the header only	scored candidate	http	server as deployed (the bearer tokens the endpoint accepts, and what it tells a client that has none)