Results / oddsrail
oddsrail
Sealmark's automated test of oddsrail 0.19.0 on 2026-11-20 gave it grade D, its lowest test-area grade (protocol A and effects D). Only 15 of 42 tools could be called with default settings, so the deeper test areas cover only those tools. Of its 6 checks, 4 passed, 1 partial and 1 not applicable. 3 more checks are advisory, and advisory checks never change a grade: 1 passed and 2 not applicable. The run used Sealmark 1.0.1 with the standard settings.
Grade D
The server's grade is its lowest test-area grade. How grading works
Only 15 of 42 tools could be called with default settings, so the deeper test areas cover only those tools.
Grade by test area
Protocol checks
| Check | Verdict | What happened |
|---|---|---|
| protocol Valid MCP handshake and tool schemas | pass | handshake ok, 42 well-formed tool(s) Evidence for protocol: 1 line
|
| tool-poisoningcritical No tool-poisoning indicators in tool metadata | pass | scanned 42 tool definition(s), the server instructions and serverInfo, 4 prompt(s), including parameter text: nothing suspicious |
| robust-input Survives malformed tool input | pass | 103 invalid call(s) rejected cleanly Evidence for robust-input: 1 line
|
| secret-echocritical No environment secret echoed in responses | pass | canary secrets absent from tools/list and 106 captured response(s), 3 of them successful result(s) |
| instructions-hygiene (advisory) Server instructions name only tools that exist | pass | the instructions name no tool the server lacks |
| refusal-quality (advisory) Refusals say what to do next | n/a | no refusal was recorded in this run, so there is nothing to judge |
Effects checks
| Check | Verdict | What happened |
|---|---|---|
| annotation-honestyweak signal Read-only tools change nothing | partial | declared read-only; only caches changed (sandbox home or temp): paper_positions Weak signal, not a security finding. A tool declared read-only changed only cache files, in its sandbox home or temp folder. A partial counts as half a point. |
| openworld-honesty Closed-world tools stay on the machine | n/a | no tool declares openWorldHint: false that could be given a harness-owned control argument |
| undeclared-writes (advisory) Unannotated tools leave the sandbox unchanged | n/a | every tool carries annotations, nothing to check |
Each check id links to what it tests and how to fix a failure. The same text is in sealmark explain <id>.
Under a check, Evidence shows what it probed (sent or inspected) and what it observed in this run: up to 6 lines per check, each cut at 240 characters, for checks that are not security checks. The text is shortened, and paths and anything that looks like a secret or token are replaced. Run sealmark run --format json yourself for every line in full.
By tool
Which checks called each tool with a valid call in this run, and which did not exercise it. A check can leave a tool out for several reasons: a limit Sealmark sets, a tool left out by design, input the run could not fill in, or a call that returned an error. The CLI report gives the reason. A verdict belongs to the whole check, not to one tool: a failed check that called three tools does not say which one failed. Security checks are left out of this table.
| Tool | Called by | Not exercised by |
|---|---|---|
| search_markets | No check listed here called it with a valid call. | None |
| get_market | No check listed here called it with a valid call. | None |
| get_orderbook | No check listed here called it with a valid call. | None |
| price_history | No check listed here called it with a valid call. | None |
| get_positions | No check listed here called it with a valid call. | None |
| overshoot_signal | No check listed here called it with a valid call. | None |
| dispute_risk | No check listed here called it with a valid call. | None |
| check_order | No check listed here called it with a valid call. | None |
| place_order | No check listed here called it with a valid call. | None |
| cancel_order | No check listed here called it with a valid call. | None |
| open_orders |
| None |
| builder_stats | No check listed here called it with a valid call. | None |
| attribution_ledger |
| None |
| kalshi_search_markets | No check listed here called it with a valid call. | None |
| kalshi_get_market | No check listed here called it with a valid call. | None |
| kalshi_get_orderbook | No check listed here called it with a valid call. | None |
| kalshi_get_trades | No check listed here called it with a valid call. | None |
| kalshi_balance |
| None |
| kalshi_positions | No check listed here called it with a valid call. | None |
| kalshi_open_orders | No check listed here called it with a valid call. | None |
| kalshi_place_order | No check listed here called it with a valid call. | None |
| kalshi_cancel_order | No check listed here called it with a valid call. | None |
| find_markets | No check listed here called it with a valid call. | None |
| compare_venues | No check listed here called it with a valid call. | None |
| quote_cost | No check listed here called it with a valid call. | None |
| order_status | No check listed here called it with a valid call. | |
| my_fills | No check listed here called it with a valid call. | |
| my_positions | No check listed here called it with a valid call. | |
| my_balance | No check listed here called it with a valid call. | |
| cancel_all_orders | No check listed here called it with a valid call. | |
| split_position | No check listed here called it with a valid call. | |
| merge_positions | No check listed here called it with a valid call. | |
| redeem_positions | No check listed here called it with a valid call. | |
| paper_positions | No check listed here called it with a valid call. | |
| paper_reset | No check listed here called it with a valid call. | |
| watch_book | No check listed here called it with a valid call. | |
| redeemable_positions | No check listed here called it with a valid call. | |
| resolution_criteria | No check listed here called it with a valid call. | |
| closing_soon | No check listed here called it with a valid call. | |
| settlement_audit | No check listed here called it with a valid call. | |
| position_size | No check listed here called it with a valid call. | |
| server_info | No check listed here called it with a valid call. |
What was not covered
- Database: Not applicable. The start command takes no database connection, so the database tests did not run. ({dsn}, {host} and the like are placeholders that Sealmark fills in with its own test database.)
- HTTP: Not applicable. The server is started over stdio; the http test area needs server.url
- Files: Not applicable. No tool is classified as taking a file path with medium confidence or higher
- Git: Not applicable. No tool is classified as taking a git repository path with medium confidence or higher
- Network: Not applicable. No tool is classified as taking a URL with medium confidence or higher
- SQLite: Not applicable. No tool takes a SQL string
- Malformed-input probes cover the first 25 of 42 tools
- Tools that a server lists only for clients that declare roots, sampling or elicitation capabilities are not seen: the harness declares none
- Tool "search_markets" is covered by the protocol checks only: argument "query" is not clearly SQL (nothing in the tool name, its description or the argument's description mentions sql, a database or a table); add it under tools: to include it
- Tool "get_market" is covered by the protocol checks only: free-text parameter "id_or_slug": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "get_orderbook" is covered by the protocol checks only: free-text parameter "token_id": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "price_history" is covered by the protocol checks only: free-text parameter "token_id": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "get_positions" is covered by the protocol checks only: argument "address" could be a URL, but nothing in the tool name, its title or the parameter's own description says so; add it under tools: to include it
- Tool "overshoot_signal" is covered by the protocol checks only: free-text parameter "token_id": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "dispute_risk" is covered by the protocol checks only: free-text parameter "id_or_slug": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "check_order" is covered by the protocol checks only: free-text parameter "venue": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "place_order" is covered by the protocol checks only: free-text parameter "token_id": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "cancel_order" is covered by the protocol checks only: free-text parameter "order_id": no path, URL, command, SQL or repository signal; add it under tools: to include it
- Tool "open_orders" is covered by the protocol checks only: the tool takes no parameters
- Tool "builder_stats" is covered by the protocol checks only: free-text parameter "time_period": no path, URL, command, SQL or repository signal; add it under tools: to include it
- 30 more tool(s) are covered by the protocol checks only (see the classification)
- Tool "order_status" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "my_fills" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "my_positions" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "my_balance" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "cancel_all_orders" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "split_position" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "merge_positions" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "redeem_positions" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "paper_positions" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "paper_reset" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "watch_book" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "redeemable_positions" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "resolution_criteria" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "closing_soon" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "settlement_audit" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "position_size" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- Tool "server_info" was not covered by the core test area: beyond the first 25 tools, which are the only ones robust-input probes
- The sandbox is a convention the harness follows, not a security boundary. A malicious server runs with your user's privileges, sees the real file system and network, and can ignore cwd, HOME and PATH. Run untrusted servers in a throwaway VM or container.
- Prompt injection carried in tool output (only tool metadata is scanned)
- Authentication and transport security
- Audit logging (not measured by these test areas; only the db test area has an advisory audit check)
Grade over time
Since the run on
Same as that run: the same grade in every test area, the same server version (0.19.0), the same Sealmark version (1.0.1).
- Dv0.19.0Sealmark 1.0.0
- Dv0.19.0Sealmark 1.0.1
- Dv0.19.0Sealmark 1.0.1
Badge for the README
The badge shows this automated result for version 0.19.0. It is a link to this page, not a certificate, and it changes when the next re-test does.
[](https://sealmark.dev/results/oddsrail/)
Files: badge SVG, card SVG, shields endpoint JSON.
Package facts
- License
- MIT License
- Latest release
- 0.19.0, published 24 Sep 2026 (the version graded on this page)
- Repository
- hmesutozsoy/oddsrail
- Last push
- 29 Oct 2026
- Open issues and pull requests
- 0
From PyPI and GitHub, checked 22 Nov 2026. Plain facts about the package; they play no part in the grade.
Add it to an MCP client
This starts version 0.19.0, the version graded on this page, with the start command from the test run. For a client that reads an mcpServers JSON file:
{
"mcpServers": {
"oddsrail": {
"command": "uvx",
"args": [
"oddsrail==0.19.0"
]
}
}
}Or add it to Claude Code:
claude mcp add oddsrail -- uvx oddsrail==0.19.0
The test run started the server with no API keys or other secrets, and with a cleaned environment. If the server needs some, its own documentation says how to pass them.
Run it yourself
This repeats the run with the same Sealmark version and the standard settings, starting the server the same way:
npx -y sealmark@1.0.1 run --start "uvx oddsrail==0.19.0"
To confirm your run used the standard settings, add --format json --out report.json to the command. In the report, config.standard should be true and config.hash should start with sha256:d13e7893a0ac, as they did in this run. If either differs, so did the settings, and the verdicts may too (STANDARD and CUSTOM runs).
To keep the setup, put the start command in sealmark.yaml as the server.
Grade alerts
One email when the grade of oddsrail changes after a re-test. Nothing else, and every email has a link to stop them.
Or follow it in a feed reader: the Atom feed lists each move of the overall grade that came with a different version of this server, tested with the same Sealmark version as the graded run before.
Each of 613 verdicts was checked independently, and Sealmark was wrong on some: how reliable Sealmark is.
Is this wrong?
Reply on the GitHub issue or email disclosure@sealmark.dev and we will recheck it. Maintainers can ask for their server to be removed.
If we were wrong, we correct the page and add your reply. Security findings go to the maintainer privately first; see the disclosure policy.