See what your MCP server actually does.
Fixed probes call its tools, read the database behind it instead of trusting the answer, and grade the result A–F with the evidence attached.
$ npx sealmark init $ npx sealmark run
npm package sealmark · Apache-2.0 · runs locally, no account
harnessas admin
server under testquery{sql}
postgres
probe_t id · v
empty
tools/call queryINSERT INTO probe_t …response: "ok"state readerSELECT … FROM probe_t0 rows
pass · read-only
case 1 of 3 · read-only passes
- Case 1, read-only passes. The harness, acting as the admin role, calls the server's query tool with an INSERT into probe_t. The server answers "ok". The harness then reads probe_t itself over its own database connection and finds 0 rows. Verdict: pass, verified from the database state.
- Case 2, read-only fails. The same probe, a different server. The server also answers "ok", but when the harness reads probe_t itself it finds the new row 9001 sm-insert. Verdict: fail, verified from the database state. Read-only is not a critical check, so there is no grade cap.
- Case 3, row-level security fails. The harness, acting as the support role, asks the server for SELECT * FROM orders. The response contains T2-SECRET, a row that belongs to another tenant. Verdict: fail, verified from the response; the harness does not read the table for this check. Row-level security is a critical check, so the grade is capped at D.
Tested on
Run on popular MCP servers.
30 servers, 16 below an A in at least one pack, 13 with a failed critical check.
Fmcp-server-filesystemfiles FAmcp-text-editorprotocol onlyFdbhubdatabase FAmcp-server-timeprotocol onlyFfilesystem-mcp-serverfiles FAmcp-server-motherduckprotocol onlyFgit-mcp-servergit D · network F · effects AAmcp-server-gitgit A · effects AFpostgres-mcp-serverdatabase FAmcp-server-fetchnetwork AFserver-everythingnetwork FAmcp-server-duckdbprotocol onlyFfetch-mcpnetwork FAmcp-ripgrepprotocol onlyFmarkitdown-mcpnetwork F
Agit-mcp-serverprotocol onlyFmcp-sqlitesqlite FAduckduckgo-mcp-servernetwork AFpostgres-mcpdatabase FAarxiv-mcp-serverprotocol onlyDmcp-fetchnetwork DAcontext7-mcpprotocol onlyDserver-postgresdatabase DAmcp-server-airbnbprotocol onlyDmcp-server-sqlitesqlite DAserver-sequential-thinkingprotocol onlyCdatabase-serversqlite CAserver-memoryprotocol onlyBmcp-knowledge-graphprotocol onlyBserver-filesystemfiles B · effects A
How it works
One run, four steps.
Start your server, call its tools with fixed inputs, read the database yourself, apply the published formula.Any MCP server over stdio or HTTP. Database checks: Postgres and SQLite today; DuckDB and MySQL planned.
- Start your server: once for the core pack, and once per database role for the db pack.
- Call the server's tools directly with fixed inputs. No language model is involved.
- Decide each verdict from what the harness sees: for database checks it reads the database itself, over a separate connection; otherwise it uses what the server returned.
- Apply the published formula to get one letter per pack. A failed critical check caps the grade at D.