Sealmarkdocs methodology 0.3-draft
methodology 0.3-draft

annotation-honesty

Read-only tools change nothing

shipped

The sections below are generated from the CLI (methodology 0.3-draft, CLI 0.3.0); the same text is printed by sealmark explain annotation-honesty.

pack
effects
weight
critical
scope
server (annotations vs what the sandbox observed during the harness's own benign calls)
decided by
fs-observer
since
0.2-draft (methodology 0.3-draft)
config keys
checks.annotation-honesty.enabledchecks.annotation-honesty.severitysandbox.enabledtools.overrides

Probes

  • control-read a path tool declared readOnlyHint: true reads {sandbox}/ok.txt (list tools list {sandbox}); snapshot of {sandbox}, {canary}, the sandbox home and temp directory before and after
  • control-git a git tool declared readOnlyHint: true runs on {sandbox}/repo
  • control-url a URL tool declared readOnlyHint: true fetches {listener}/t/<token>; a tool that refused it and was shown to refuse every loopback target by net-loopback-ssrf is n/a (no harness-owned control URL exists for it)
  • control-noarg a tool declared readOnlyHint: true with no required parameter (any class) is called with {}

Settings and thresholds

Every check also takes checks.annotation-honesty.enabled and checks.annotation-honesty.severity; they are described once in the configuration reference.

This check has no numeric threshold.

KeyDefaultStandardMeaning
tools.overrides{}lockedPer-tool classifier overrides: { tool: { class, params } }. An override always wins and is printed.

How the verdict is established

  • Scoring. critical and scored: pass = 1 point, part = 0.5, fail = 0. A fail caps the pack's grade at D, whatever the other checks scored.

What to fix

Make a tool annotated readOnlyHint: true change nothing on disk, or remove the annotation; index and cache refreshes in the observed directories still count, so write them elsewhere or stop writing them. A read-only tool that takes no arguments is called with {} and observed like any other. A URL tool whose control fetch was refused is n/a here when the net check showed it refuses loopback targets, because the harness owns no URL such a tool will fetch; when every tool is n/a the check is n/a, so a correctly guarded fetcher can be green under the ci profile.

False positives

A read-only tool that refreshes a lock or index file (for example git) is a part at most: changes under .git/index and *.lock are ignored for git tools. Cache files in the sandbox home or temp directory are a part, not a fail.

False negatives

Only the harness's benign control call is observed; a tool that writes only for other arguments passes. A tool with required parameters outside the classes the harness can fill (a path, a repository, a URL) is not called. Writes outside the observed directories (the real home, other drives) are not seen. Changes are looked for right after the reply and again after a settle window (300 ms, longer when timeout_probe_sec is large); a write that comes later than that is not seen.