Sealmark
Automated re-test on 2026-11-20: not checked by hand, and not yet reviewed by the maintainers.

Results / @shipeasy/mcp

@shipeasy/mcp

version tested 2.16.0npmtested 2026-11-20Sealmark 1.0.1methodology 0.3-draftSTANDARD run180 tools

Sealmark's automated test of @shipeasy/mcp 2.16.0 on 2026-11-20 gave it grade B, its lowest test-area grade (protocol B and effects A). Only 30 of 180 tools could be called with default settings, so the deeper test areas cover only those tools. Of its 10 checks, 4 passed, 1 partial, 2 inconclusive and 3 not applicable. 3 more checks are advisory, and advisory checks never change a grade: 3 not applicable. The run used Sealmark 1.0.1 with the standard settings.

B

Grade B

The server's grade is its lowest test-area grade. How grading works

Only 30 of 180 tools could be called with default settings, so the deeper test areas cover only those tools.

Grade by test area

B
Protocol3.5/4 pointsincludes a weak signal, not a security finding
A
Effects1/1 points

Protocol checks

CheckVerdictWhat happened
protocol
Valid MCP handshake and tool schemas
pass
handshake ok, 180 well-formed tool(s)
Evidence for protocol: 1 line
  1. Probe initialize + tools/list
    Outcome server shipeasy 0.1.0 answered; 180 tool(s)
    Decided from the MCP exchange (protocol)
tool-poisoningweak signal
No tool-poisoning indicators in tool metadata
partial
1 weak indicator(s) worth reading, none conclusive
Weak signal, not a security finding. Some wording in the tool text is worth a read, but none of it is conclusive. A partial counts as half a point.
robust-input
Survives malformed tool input
pass
104 invalid call(s) rejected cleanly
Evidence for robust-input: 1 line
  1. Probe 104 invalid call(s) across 21 tool(s)
    Outcome all rejected cleanly and the server stayed responsive
    Decided from the MCP exchange (protocol)
secret-echocritical
No environment secret echoed in responses
pass
canary secrets absent from tools/list and 107 captured response(s), 1 of them successful result(s)
instructions-hygiene (advisory)
Server instructions name only tools that exist
n/a
the server sent no instructions
refusal-quality (advisory)
Refusals say what to do next
n/a
no refusal was recorded in this run, so there is nothing to judge

Network checks

CheckVerdictWhat happened
net-loopback-ssrfcritical
Refuses loopback targets (SSRF)
inconclusive
the listener saw nothing, but 54 of 54 probes were not explicitly refused (54 failed without saying the target was refused), so a refusal cannot be told from a tool that cannot fetch at all (a reply that only reports a failed fetch (proxy, socket, DNS) is not read as a refusal; set net.control_url to a URL the tool can reach)
net-scheme-allowlistcritical
Only http(s) schemes accepted
inconclusive
no scheme had an observable effect, but 36 of 36 scored probes were not explicitly refused, so a refusal cannot be told from a tool that cannot fetch at all (a reply that only reports a failed fetch (proxy, socket, DNS) is not read as a refusal; set net.control_url to a URL the tool can reach)
net-caps
Response size and time are capped
n/a
net-loopback-ssrf was inconclusive: the server never reached the loopback listener, so no loopback URL is known to be fetchable; response size and time caps were not measured
net-redirect
Redirects are re-validated
n/a
the server refused or never reached the loopback listener, so redirect handling could not be observed (a server that blocks loopback cannot be sent a redirect it would follow)

Effects checks

CheckVerdictWhat happened
annotation-honestycritical
Read-only tools change nothing
pass
auth_check, metrics_grammar declared read-only and changed nothing in the observed directories during a benign call (errors_list, i18n_drafts_list, i18n_keys_list, i18n_profiles_list, metrics_events_list, metrics_list, ops_activity_list, ops_agents_list, ops_alerts_channels, ops_alerts_list, ops_fired_alerts_list, ops_memory_search, ops_review_agents_list, ops_routing_rules_list, ops_trail_gates_get, projects_current, release_configs_list, release_experiments_list, release_experiments_universes_list, release_flags_attributes_list, release_flags_list, release_flags_templates_list, release_killswitch_list, tags_list, whoami, docs_list, docs_get, docs_skill not judged: the control call failed)
openworld-honesty
Closed-world tools stay on the machine
n/a
no tool declares openWorldHint: false that could be given a harness-owned control argument
undeclared-writes (advisory)
Unannotated tools leave the sandbox unchanged
n/a
every tool carries annotations, nothing to check

Each check id links to what it tests and how to fix a failure. The same text is in sealmark explain <id>.

Under a check, Evidence shows what it probed (sent or inspected) and what it observed in this run: up to 6 lines per check, each cut at 240 characters, for checks that are not security checks. The text is shortened, and paths and anything that looks like a secret or token are replaced. Run sealmark run --format json yourself for every line in full.

By tool

Which checks called each tool with a valid call in this run, and which did not exercise it. A check can leave a tool out for several reasons: a limit Sealmark sets, a tool left out by design, input the run could not fill in, or a call that returned an error. The CLI report gives the reason. A verdict belongs to the whole check, not to one tool: a failed check that called three tools does not say which one failed. Security checks are left out of this table.

ToolCalled byNot exercised by
projects_upsertNo check listed here called it with a valid call.None
auth_checkNone
auth_loginNo check listed here called it with a valid call.
auth_logoutNo check listed here called it with a valid call.None
api_keys_createNo check listed here called it with a valid call.None
errors_getNo check listed here called it with a valid call.None
errors_listNo check listed here called it with a valid call.None
errors_resolveNo check listed here called it with a valid call.None
errors_seriesNo check listed here called it with a valid call.None
i18n_drafts_createNo check listed here called it with a valid call.None
i18n_drafts_listNo check listed here called it with a valid call.
i18n_drafts_updateNo check listed here called it with a valid call.None
i18n_keys_listNo check listed here called it with a valid call.None
i18n_keys_pushNo check listed here called it with a valid call.None
i18n_keys_setNo check listed here called it with a valid call.None
i18n_keys_updateNo check listed here called it with a valid call.None
i18n_profiles_createNo check listed here called it with a valid call.None
i18n_profiles_listNo check listed here called it with a valid call.
i18n_profiles_publishNo check listed here called it with a valid call.None
metrics_archiveNo check listed here called it with a valid call.None
metrics_createNo check listed here called it with a valid call.None
metrics_events_approveNo check listed here called it with a valid call.None
metrics_events_archiveNo check listed here called it with a valid call.None
metrics_events_createNo check listed here called it with a valid call.None
metrics_events_getNo check listed here called it with a valid call.None
metrics_events_listNo check listed here called it with a valid call.
metrics_events_updateNo check listed here called it with a valid call.
metrics_experimentsNo check listed here called it with a valid call.
metrics_listNo check listed here called it with a valid call.
metrics_seriesNo check listed here called it with a valid call.
metrics_showNo check listed here called it with a valid call.
metrics_unarchiveNo check listed here called it with a valid call.
metrics_updateNo check listed here called it with a valid call.
ops_ackNo check listed here called it with a valid call.
ops_activity_listNo check listed here called it with a valid call.
ops_agents_listNo check listed here called it with a valid call.
ops_alerts_archiveNo check listed here called it with a valid call.
ops_alerts_channelsNo check listed here called it with a valid call.
ops_alerts_createNo check listed here called it with a valid call.
ops_alerts_listNo check listed here called it with a valid call.
ops_alerts_updateNo check listed here called it with a valid call.
ops_attachments_createNo check listed here called it with a valid call.
ops_bugNo check listed here called it with a valid call.
ops_comments_createNo check listed here called it with a valid call.
ops_comments_listNo check listed here called it with a valid call.
ops_comments_upload_mediaNo check listed here called it with a valid call.
ops_createNo check listed here called it with a valid call.
ops_featureNo check listed here called it with a valid call.
ops_fired_alerts_listNo check listed here called it with a valid call.
ops_fired_alerts_updateNo check listed here called it with a valid call.
ops_getNo check listed here called it with a valid call.
ops_groupNo check listed here called it with a valid call.
ops_investigations_createNo check listed here called it with a valid call.
ops_investigations_listNo check listed here called it with a valid call.
ops_investigations_updateNo check listed here called it with a valid call.
ops_link_prNo check listed here called it with a valid call.
ops_linksNo check listed here called it with a valid call.
ops_listNo check listed here called it with a valid call.
ops_memory_searchNo check listed here called it with a valid call.
ops_notifyNo check listed here called it with a valid call.
ops_qa_notes_createNo check listed here called it with a valid call.
ops_qa_notes_listNo check listed here called it with a valid call.
ops_qa_questions_answerNo check listed here called it with a valid call.
ops_qa_questions_askNo check listed here called it with a valid call.
ops_qa_questions_listNo check listed here called it with a valid call.
ops_questions_answerNo check listed here called it with a valid call.
ops_questions_askNo check listed here called it with a valid call.
ops_questions_cancelNo check listed here called it with a valid call.
ops_questions_listNo check listed here called it with a valid call.
ops_review_agents_archiveNo check listed here called it with a valid call.
ops_review_agents_createNo check listed here called it with a valid call.
ops_review_agents_listNo check listed here called it with a valid call.
ops_review_agents_updateNo check listed here called it with a valid call.
ops_routing_rules_createNo check listed here called it with a valid call.
ops_routing_rules_listNo check listed here called it with a valid call.
ops_trail_gates_getNo check listed here called it with a valid call.
ops_trail_gates_setNo check listed here called it with a valid call.
ops_trigger_create_claudeNo check listed here called it with a valid call.
ops_trigger_create_copilotNo check listed here called it with a valid call.
ops_trigger_create_cursorNo check listed here called it with a valid call.
ops_trigger_create_julesNo check listed here called it with a valid call.
ops_updateNo check listed here called it with a valid call.
projects_currentNo check listed here called it with a valid call.
projects_updateNo check listed here called it with a valid call.
release_configs_archiveNo check listed here called it with a valid call.
release_configs_createNo check listed here called it with a valid call.
release_configs_getNo check listed here called it with a valid call.
release_configs_listNo check listed here called it with a valid call.
release_configs_updateNo check listed here called it with a valid call.
release_configs_update_schemaNo check listed here called it with a valid call.
release_experiments_archiveNo check listed here called it with a valid call.
release_experiments_createNo check listed here called it with a valid call.
release_experiments_getNo check listed here called it with a valid call.
release_experiments_listNo check listed here called it with a valid call.
release_experiments_readout_createNo check listed here called it with a valid call.
release_experiments_readout_getNo check listed here called it with a valid call.
release_experiments_reanalyzeNo check listed here called it with a valid call.
release_experiments_restoreNo check listed here called it with a valid call.
release_experiments_resultsNo check listed here called it with a valid call.
release_experiments_set_metricsNo check listed here called it with a valid call.
release_experiments_startNo check listed here called it with a valid call.
release_experiments_stopNo check listed here called it with a valid call.
release_experiments_timeseriesNo check listed here called it with a valid call.
release_experiments_universes_archiveNo check listed here called it with a valid call.
release_experiments_universes_createNo check listed here called it with a valid call.
release_experiments_universes_listNo check listed here called it with a valid call.
release_experiments_universes_updateNo check listed here called it with a valid call.
release_experiments_updateNo check listed here called it with a valid call.
release_flags_activityNo check listed here called it with a valid call.
release_flags_archiveNo check listed here called it with a valid call.
release_flags_attributes_archiveNo check listed here called it with a valid call.
release_flags_attributes_createNo check listed here called it with a valid call.
release_flags_attributes_getNo check listed here called it with a valid call.
release_flags_attributes_listNo check listed here called it with a valid call.
release_flags_attributes_updateNo check listed here called it with a valid call.
release_flags_createNo check listed here called it with a valid call.
release_flags_disableNo check listed here called it with a valid call.
release_flags_enableNo check listed here called it with a valid call.
release_flags_getNo check listed here called it with a valid call.
release_flags_listNo check listed here called it with a valid call.
release_flags_templates_archiveNo check listed here called it with a valid call.
release_flags_templates_createNo check listed here called it with a valid call.
release_flags_templates_getNo check listed here called it with a valid call.
release_flags_templates_listNo check listed here called it with a valid call.
release_flags_templates_updateNo check listed here called it with a valid call.
release_flags_updateNo check listed here called it with a valid call.
release_flags_whitelistNo check listed here called it with a valid call.
release_flags_whitelist_addNo check listed here called it with a valid call.
release_flags_whitelist_removeNo check listed here called it with a valid call.
release_flags_whitelist_setNo check listed here called it with a valid call.
release_killswitch_archiveNo check listed here called it with a valid call.
release_killswitch_createNo check listed here called it with a valid call.
release_killswitch_getNo check listed here called it with a valid call.
release_killswitch_listNo check listed here called it with a valid call.
release_killswitch_setNo check listed here called it with a valid call.
release_killswitch_set_valueNo check listed here called it with a valid call.
release_killswitch_toggleNo check listed here called it with a valid call.
release_killswitch_unsetNo check listed here called it with a valid call.
release_killswitch_updateNo check listed here called it with a valid call.
tags_getNo check listed here called it with a valid call.
tags_listNo check listed here called it with a valid call.
tags_setNo check listed here called it with a valid call.
tags_updateNo check listed here called it with a valid call.
trails_attributeNo check listed here called it with a valid call.
trails_closeNo check listed here called it with a valid call.
trails_confirm_suspectNo check listed here called it with a valid call.
trails_createNo check listed here called it with a valid call.
trails_getNo check listed here called it with a valid call.
trails_items_addNo check listed here called it with a valid call.
trails_items_moveNo check listed here called it with a valid call.
trails_items_removeNo check listed here called it with a valid call.
trails_link_prNo check listed here called it with a valid call.
trails_listNo check listed here called it with a valid call.
trails_milestones_addNo check listed here called it with a valid call.
trails_milestones_removeNo check listed here called it with a valid call.
trails_milestones_updateNo check listed here called it with a valid call.
trails_pause_autopilotNo check listed here called it with a valid call.
trails_pause_rolloutNo check listed here called it with a valid call.
trails_plan_revisionsNo check listed here called it with a valid call.
trails_promote_findingNo check listed here called it with a valid call.
trails_reject_suspectNo check listed here called it with a valid call.
trails_release_factsNo check listed here called it with a valid call.
trails_reopenNo check listed here called it with a valid call.
trails_resources_addNo check listed here called it with a valid call.
trails_resources_listNo check listed here called it with a valid call.
trails_resources_removeNo check listed here called it with a valid call.
trails_reviewNo check listed here called it with a valid call.
trails_startNo check listed here called it with a valid call.
trails_submit_evalNo check listed here called it with a valid call.
trails_submit_planNo check listed here called it with a valid call.
trails_timelineNo check listed here called it with a valid call.
trails_unlink_prNo check listed here called it with a valid call.
trails_updateNo check listed here called it with a valid call.
trails_update_findingNo check listed here called it with a valid call.
trails_update_planNo check listed here called it with a valid call.
whoamiNo check listed here called it with a valid call.
metrics_grammarNo check listed here called it with a valid call.
docs_listNo check listed here called it with a valid call.
docs_getNo check listed here called it with a valid call.
docs_skillNo check listed here called it with a valid call.

What was not covered

Grade over time

Since the run on

Same as that run: the same grade in every test area, the same server version (2.16.0), the same Sealmark version (1.0.1).

  1. Bv2.16.0Sealmark 1.0.0
  2. Bv2.16.0Sealmark 1.0.1
  3. Bv2.16.0Sealmark 1.0.1

Package facts

License
MIT
Latest release
2.16.0, published 4 Nov 2026 (the version graded on this page)
Repository
shipeasy-ai/shipeasy
Last push
20 Nov 2026
Open issues and pull requests
0

From npm and GitHub, checked 22 Nov 2026. Plain facts about the package; they play no part in the grade.

Add it to an MCP client

This starts version 2.16.0, the version graded on this page, with the start command from the test run. For a client that reads an mcpServers JSON file:

{
  "mcpServers": {
    "shipeasy-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@shipeasy/mcp@2.16.0"
      ]
    }
  }
}

Or add it to Claude Code:

claude mcp add shipeasy-mcp -- npx -y "@shipeasy/mcp@2.16.0"

The test run started the server with no API keys or other secrets, and with a cleaned environment. If the server needs some, its own documentation says how to pass them.

Run it yourself

This repeats the run with the same Sealmark version and the standard settings, starting the server the same way:

npx -y sealmark@1.0.1 run --start "npx -y @shipeasy/mcp@2.16.0"

To confirm your run used the standard settings, add --format json --out report.json to the command. In the report, config.standard should be true and config.hash should start with sha256:d13e7893a0ac, as they did in this run. If either differs, so did the settings, and the verdicts may too (STANDARD and CUSTOM runs).

To keep the setup, put the start command in sealmark.yaml as the server.

Grade alerts

One email when the grade of @shipeasy/mcp changes after a re-test. Nothing else, and every email has a link to stop them.

Or follow it in a feed reader: the Atom feed lists each move of the overall grade that came with a different version of this server, tested with the same Sealmark version as the graded run before.

Each of 613 verdicts was checked independently, and Sealmark was wrong on some: how reliable Sealmark is.

Is this wrong?

Reply on the GitHub issue or email disclosure@sealmark.dev and we will recheck it. Maintainers can ask for their server to be removed.

If we were wrong, we correct the page and add your reply. Security findings go to the maintainer privately first; see the disclosure policy.