Unreleased: not public yet. Maintainers have not reviewed this result.
Results / arxiv-mcp-server
arxiv-mcp-server
A
Protocol4/4 points
Coverage: low (1 of 19 tools reached with a valid call). Only protocol hygiene was graded, so the A says little about what the tools do.
Protocol checks
| Check | Verdict | What happened |
|---|---|---|
| protocol Valid MCP handshake and tool schemas | pass | handshake ok, 19 well-formed tool(s) |
| tool-poisoningcritical No tool-poisoning indicators in tool metadata | pass | scanned 19 tool definition(s), 7 prompt(s), including parameter text: nothing suspicious |
| robust-input Survives malformed tool input | pass | 68 invalid call(s) rejected cleanly |
| secret-echocritical No environment secret echoed in responses | pass | canary secrets absent from tools/list and 69 captured response(s) |
What was not covered
- Database: Not applicable. Server config has no {dsn}/{host}/... Placeholder, so the harness cannot start the server once per role
- HTTP: Not applicable. The server is started over stdio; the http pack needs server.url
- Files: Not applicable. No tool is classified as taking a file path with medium confidence or higher
- Git: Not applicable. No tool is classified as taking a git repository path with medium confidence or higher
- Network: Not applicable. No tool is classified as taking a URL with medium confidence or higher
- SQLite: Not applicable. No tool takes a SQL string
- Effects: Not applicable. No tool takes a harness-owned target (file path, repository or URL) with medium confidence or higher
- Tools that a server lists only for clients that declare roots, sampling or elicitation capabilities are not seen: the harness declares none
- Tool "search_papers" is covered by the core checks only: argument "query" is not clearly SQL (nothing in the tool name, its description or the argument's description mentions sql, a database or a table); add it under tools: to include it
- Tool "download_paper" is covered by the core checks only: string parameters only, and none names a path, URL, command, SQL or repository; add it under tools: to include it
- Tool "list_papers" is covered by the core checks only: the tool has no string parameters; add it under tools: to include it
- Tool "read_paper" is covered by the core checks only: string parameters only, and none names a path, URL, command, SQL or repository; add it under tools: to include it
- Tool "get_abstract" is covered by the core checks only: string parameters only, and none names a path, URL, command, SQL or repository; add it under tools: to include it
- Tool "semantic_search" is covered by the core checks only: argument "query" is not clearly SQL (nothing in the tool name, its description or the argument's description mentions sql, a database or a table); add it under tools: to include it
- Tool "reindex" is covered by the core checks only: the tool has no string parameters; add it under tools: to include it
Run it yourself
Started with the standard settings like this; put it in sealmark.yaml as the server:
uvx arxiv-mcp-server==0.8.2
Each of 613 verdicts was checked independently, and Sealmark was wrong on some: how reliable Sealmark is.
Think a verdict is wrong, or want a server removed? Email disclosure@sealmark.dev with the server version. We recheck, correct the page if we were wrong, and add your reply. See the disclosure policy.