Sealmark
Automated re-test on 2026-11-20: not checked by hand, and not yet reviewed by the maintainers.

Results / raven-mcp

raven-mcp

version tested 3.1.0npmtested 2026-11-20Sealmark 1.0.1methodology 0.3-draftSTANDARD run115 tools

Sealmark's automated test of raven-mcp 3.1.0 on 2026-11-20 gave it grade B, its lowest test-area grade (protocol B and effects B). Only 46 of 115 tools could be called with default settings, so the deeper test areas cover only those tools. Of its 10 checks, 4 passed, 2 partial, 2 inconclusive and 2 not applicable. 3 more checks are advisory, and advisory checks never change a grade: 1 failed and 2 not applicable. The run used Sealmark 1.0.1 with the standard settings.

B

Grade B

The server's grade is its lowest test-area grade. How grading works

Only 46 of 115 tools could be called with default settings, so the deeper test areas cover only those tools.

Grade by test area

B
Protocol3.5/4 pointsincludes a weak signal, not a security finding
B
Effects1.5/2 pointsincludes a weak signal, not a security finding

Protocol checks

CheckVerdictWhat happened
protocol
Valid MCP handshake and tool schemas
pass
handshake ok, 115 well-formed tool(s)
Evidence for protocol: 1 line
  1. Probe initialize + tools/list
    Outcome server raven-mcp 3.1.0 answered; 115 tool(s)
    Decided from the MCP exchange (protocol)
tool-poisoningweak signal
No tool-poisoning indicators in tool metadata
partial
1 weak indicator(s) worth reading, none conclusive
Weak signal, not a security finding. Some wording in the tool text is worth a read, but none of it is conclusive. A partial counts as half a point.
robust-input
Survives malformed tool input
pass
107 invalid call(s) rejected cleanly
Evidence for robust-input: 1 line
  1. Probe 107 invalid call(s) across 23 tool(s)
    Outcome all rejected cleanly and the server stayed responsive
    Decided from the MCP exchange (protocol)
secret-echocritical
No environment secret echoed in responses
pass
canary secrets absent from tools/list and 108 captured response(s), 0 of them successful result(s)
instructions-hygiene (advisory)
Server instructions name only tools that exist
fail
the server instructions names 8 tool(s) the server does not offer (first_run, existing_binding, more_questions, calibration_hint, note_assessments)
refusal-quality (advisory)
Refusals say what to do next
n/a
no refusal was recorded in this run, so there is nothing to judge

Network checks

CheckVerdictWhat happened
net-loopback-ssrfcritical
Refuses loopback targets (SSRF)
inconclusive
the listener saw nothing, but 54 of 54 probes were not explicitly refused (54 failed without saying the target was refused), so a refusal cannot be told from a tool that cannot fetch at all (a reply that only reports a failed fetch (proxy, socket, DNS) is not read as a refusal; set net.control_url to a URL the tool can reach)
net-scheme-allowlistcritical
Only http(s) schemes accepted
inconclusive
no scheme had an observable effect, but 36 of 36 scored probes were not explicitly refused, so a refusal cannot be told from a tool that cannot fetch at all (a reply that only reports a failed fetch (proxy, socket, DNS) is not read as a refusal; set net.control_url to a URL the tool can reach)
net-caps
Response size and time are capped
n/a
net-loopback-ssrf was inconclusive: the server never reached the loopback listener, so no loopback URL is known to be fetchable; response size and time caps were not measured
net-redirect
Redirects are re-validated
n/a
the server refused or never reached the loopback listener, so redirect handling could not be observed (a server that blocks loopback cannot be sent a redirect it would follow)

Effects checks

CheckVerdictWhat happened
annotation-honestyweak signal
Read-only tools change nothing
partial
declared read-only; only caches changed (sandbox home or temp): evaluate_design, get_d4d_framework, list_design_systems, read_design_md, search_references, map_reference_to_tokens, list_templates, get_grab_layers, score_page, audit_device_frame, audit_responsive_visibility, audit_contrast, suggest_contrast_fix, audit_typography, audit_tap_targets, audit_layout, audit_screen, audit_ios_screen, audit_ios_privacy, list_content_systems, get_content_principles, get_research_method, get_metrics_framework, get_service_standard, get_brand_principles, get_brand_trends, list_creative_models, list_creative_presets, list_brand_profiles, list_generation_jobs, raven_reflect, audit_video_playback, gap_scan, list_taste_profiles, talon_scan, talon_rules, inventory_design_system, diff_design_system, list_design_system_components, audit (audit_taste not judged: the control call failed)
Weak signal, not a security finding. A tool declared read-only changed only cache files, in its sandbox home or temp folder. A partial counts as half a point.
openworld-honesty
Closed-world tools stay on the machine
pass
37 closed-world tool(s) called; none reached out beyond what was asked
undeclared-writes (advisory)
Unannotated tools leave the sandbox unchanged
n/a
every tool carries annotations, nothing to check

Each check id links to what it tests and how to fix a failure. The same text is in sealmark explain <id>.

Under a check, Evidence shows what it probed (sent or inspected) and what it observed in this run: up to 6 lines per check, each cut at 240 characters, for checks that are not security checks. The text is shortened, and paths and anything that looks like a secret or token are replaced. Run sealmark run --format json yourself for every line in full.

By tool

Which checks called each tool with a valid call in this run, and which did not exercise it. A check can leave a tool out for several reasons: a limit Sealmark sets, a tool left out by design, input the run could not fill in, or a call that returned an error. The CLI report gives the reason. A verdict belongs to the whole check, not to one tool: a failed check that called three tools does not say which one failed. Security checks are left out of this table.

ToolCalled byNot exercised by
get_principlesNo check listed here called it with a valid call.None
get_patternNo check listed here called it with a valid call.None
get_business_strategyNo check listed here called it with a valid call.None
evaluate_designNo check listed here called it with a valid call.None
search_knowledgeNo check listed here called it with a valid call.None
get_checklistNo check listed here called it with a valid call.None
get_d4d_frameworkNo check listed here called it with a valid call.None
list_design_systemsNo check listed here called it with a valid call.None
get_design_systemNo check listed here called it with a valid call.None
read_design_mdNo check listed here called it with a valid call.None
review_diffNo check listed here called it with a valid call.None
polish_diffNo check listed here called it with a valid call.None
init_design_mdNo check listed here called it with a valid call.None
update_design_mdNo check listed here called it with a valid call.None
start_grab_sessionNo check listed here called it with a valid call.None
get_grabbed_elementsNo check listed here called it with a valid call.None
capture_referenceNo check listed here called it with a valid call.None
search_referencesNo check listed here called it with a valid call.None
forget_referencesNo check listed here called it with a valid call.None
map_reference_to_tokensNo check listed here called it with a valid call.None
stop_grab_sessionNo check listed here called it with a valid call.
get_page_templateNo check listed here called it with a valid call.None
set_template_slotNo check listed here called it with a valid call.None
list_templatesNo check listed here called it with a valid call.
get_grab_layersNo check listed here called it with a valid call.None
move_grab_layerNo check listed here called it with a valid call.
get_grab_operationNo check listed here called it with a valid call.
compose_systemNo check listed here called it with a valid call.
audit_pageNo check listed here called it with a valid call.
score_pageNo check listed here called it with a valid call.
audit_asset_integrityNo check listed here called it with a valid call.
audit_device_frameNo check listed here called it with a valid call.
audit_contractNo check listed here called it with a valid call.
audit_api_contractNo check listed here called it with a valid call.
audit_parityNo check listed here called it with a valid call.
audit_ios_a11yNo check listed here called it with a valid call.
audit_responsive_visibilityNo check listed here called it with a valid call.
audit_contrastNo check listed here called it with a valid call.
suggest_contrast_fixNo check listed here called it with a valid call.
audit_urlNo check listed here called it with a valid call.
audit_contentNo check listed here called it with a valid call.
audit_typographyNo check listed here called it with a valid call.
audit_tap_targetsNo check listed here called it with a valid call.
get_brand_systemNo check listed here called it with a valid call.
generate_design_systemNo check listed here called it with a valid call.
audit_layoutNo check listed here called it with a valid call.
audit_swiftuiNo check listed here called it with a valid call.
audit_screenNo check listed here called it with a valid call.
audit_ios_screenNo check listed here called it with a valid call.
audit_ios_privacyNo check listed here called it with a valid call.
audit_rnNo check listed here called it with a valid call.
list_content_systemsNo check listed here called it with a valid call.
get_content_systemNo check listed here called it with a valid call.
get_content_principlesNo check listed here called it with a valid call.
get_content_patternNo check listed here called it with a valid call.
get_research_methodNo check listed here called it with a valid call.
get_metrics_frameworkNo check listed here called it with a valid call.
get_service_patternNo check listed here called it with a valid call.
get_service_standardNo check listed here called it with a valid call.
generate_service_blueprintNo check listed here called it with a valid call.
get_brand_principlesNo check listed here called it with a valid call.
get_brand_trendsNo check listed here called it with a valid call.
list_creative_modelsNo check listed here called it with a valid call.
list_creative_presetsNo check listed here called it with a valid call.
create_brand_profileNo check listed here called it with a valid call.
get_brand_profileNo check listed here called it with a valid call.
list_brand_profilesNo check listed here called it with a valid call.
register_creative_assetNo check listed here called it with a valid call.
create_character_profileNo check listed here called it with a valid call.
create_generation_jobNo check listed here called it with a valid call.
get_generation_jobNo check listed here called it with a valid call.
list_generation_jobsNo check listed here called it with a valid call.
plan_creative_campaignNo check listed here called it with a valid call.
score_creativeNo check listed here called it with a valid call.
audit_consistencyNo check listed here called it with a valid call.
raven_reflectNo check listed here called it with a valid call.
raven_registerNo check listed here called it with a valid call.
audit_video_playbackNo check listed here called it with a valid call.
decision_addNo check listed here called it with a valid call.
decision_evidenceNo check listed here called it with a valid call.
decision_draftNo check listed here called it with a valid call.
decision_commitNo check listed here called it with a valid call.
decision_supersedeNo check listed here called it with a valid call.
decision_contestNo check listed here called it with a valid call.
decision_scopeNo check listed here called it with a valid call.
decision_historyNo check listed here called it with a valid call.
decision_getNo check listed here called it with a valid call.
decision_listNo check listed here called it with a valid call.
gap_scanNo check listed here called it with a valid call.
decision_importNo check listed here called it with a valid call.
ingest_transcriptNo check listed here called it with a valid call.
ingest_transcript_resultsNo check listed here called it with a valid call.
create_taste_profileNo check listed here called it with a valid call.
get_taste_profileNo check listed here called it with a valid call.
get_taste_interviewNo check listed here called it with a valid call.
bind_taste_surfaceNo check listed here called it with a valid call.
record_taste_decisionNo check listed here called it with a valid call.
list_taste_decisionsNo check listed here called it with a valid call.
generate_taste_portraitNo check listed here called it with a valid call.
generate_mood_boardNo check listed here called it with a valid call.
create_motion_playgroundNo check listed here called it with a valid call.
open_motion_playgroundNo check listed here called it with a valid call.
get_motion_sendsNo check listed here called it with a valid call.
suggest_motionNo check listed here called it with a valid call.
list_taste_profilesNo check listed here called it with a valid call.
label_findingNo check listed here called it with a valid call.
audit_tasteNo check listed here called it with a valid call.
talon_scanNo check listed here called it with a valid call.
talon_rulesNo check listed here called it with a valid call.
design_gauntletNo check listed here called it with a valid call.
configure_design_system_sourceNo check listed here called it with a valid call.
inventory_design_systemNo check listed here called it with a valid call.
diff_design_systemNo check listed here called it with a valid call.
list_design_system_componentsNo check listed here called it with a valid call.
auditNo check listed here called it with a valid call.

What was not covered

Grade over time

Since the run on

Same as that run: the same grade in every test area, the same tools with the same input schemas, the same server version (3.1.0), the same Sealmark version (1.0.1).

  1. Bv3.0.0Sealmark 1.0.0
  2. Bv3.0.0Sealmark 1.0.1
  3. Bv3.1.0Sealmark 1.0.1
  4. Bv3.1.0Sealmark 1.0.1

Package facts

License
Apache-2.0
Latest release
3.1.0, published 16 Nov 2026 (the version graded on this page)
Repository
rhinocap/raven-mcp
Last push
16 Nov 2026
Open issues and pull requests
2

From npm and GitHub, checked 22 Nov 2026. Plain facts about the package; they play no part in the grade.

Add it to an MCP client

This starts version 3.1.0, the version graded on this page, with the start command from the test run. For a client that reads an mcpServers JSON file:

{
  "mcpServers": {
    "raven-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "raven-mcp@3.1.0"
      ]
    }
  }
}

Or add it to Claude Code:

claude mcp add raven-mcp -- npx -y "raven-mcp@3.1.0"

The test run started the server with no API keys or other secrets, and with a cleaned environment. If the server needs some, its own documentation says how to pass them.

Run it yourself

This repeats the run with the same Sealmark version and the standard settings, starting the server the same way:

npx -y sealmark@1.0.1 run --start "npx -y raven-mcp@3.1.0"

To confirm your run used the standard settings, add --format json --out report.json to the command. In the report, config.standard should be true and config.hash should start with sha256:d13e7893a0ac, as they did in this run. If either differs, so did the settings, and the verdicts may too (STANDARD and CUSTOM runs).

To keep the setup, put the start command in sealmark.yaml as the server.

Grade alerts

One email when the grade of raven-mcp changes after a re-test. Nothing else, and every email has a link to stop them.

Or follow it in a feed reader: the Atom feed lists each move of the overall grade that came with a different version of this server, tested with the same Sealmark version as the graded run before.

Each of 613 verdicts was checked independently, and Sealmark was wrong on some: how reliable Sealmark is.

Is this wrong?

Reply on the GitHub issue or email disclosure@sealmark.dev and we will recheck it. Maintainers can ask for their server to be removed.

If we were wrong, we correct the page and add your reply. Security findings go to the maintainer privately first; see the disclosure policy.