Configuration

The tables on this page are generated from the CLI: every key the installed build accepts, with its type, default, the version that added it and whether the standard methodology locks it. This page explains how configuration is read. sealmark config show --origins prints every effective value with the layer it came from.

A config file is optional. Every setting is reachable by a flag, by a SEALMARK_* environment variable or by a file, so a CI job can run with the command line alone. sealmark init writes a starter file. See CLI for the flags.

Layers#

Lowest to highest, each layer overrides the one before:

1. built-in defaults;
2. the selected profile (standard unless the profile key or --profile says otherwise; a profile may extends: another);
3. your user file: $XDG_CONFIG_HOME/sealmark/config.yaml (%APPDATA%\sealmark\config.yaml on Windows);
4. sealmark.yaml (or the file named by --config) and sealmark.local.yaml beside it, which belongs in .gitignore;
5. SEALMARK_* environment variables: SEALMARK_PACKS=core,fs, SEALMARK_SANDBOX__DIR=/tmp/x (a double underscore nests);
6. command-line flags.

Maps merge key by key and lists replace, except the lists that append (tools.exclude, corpus add and remove, notCovered.extra, plugins). null resets a key to its built-in default. --active and --yes come only from flags: a file or an environment variable cannot switch on the opt-in exec pack or confirm an administrative --pg run.

A key's environment variable is SEALMARK_ plus its path in capitals, with __ for each dot. --start, --header and --bearer have none: they are flags, or server.command, server.args and server.headers in a file.

Two minimal files#

A stdio server that takes a database connection string. The {dsn} placeholder is what lets the db pack start your server once per database role.

name: server-a
version: 1.4.0
server:
  command: node
  args: ["./server-a.js", "{dsn}"]

An HTTP server that is already running. The harness does not start it.

name: server-d
version: 0.3.0
server:
  url: http://localhost:8000/mcp

Server A and Server D are invented examples, like every server and grade on this site.

How the file is checked#

Unknown keys are errors, with a did-you-mean hint and the file and line. All problems found are listed together and stop the run with exit code 2. Among the things that are checked:

• the file exists, is valid YAML and contains a mapping;
• server has command or url; both together are allowed and mean "launch the command and test the URL", but only when both are set in the same place (the same file, the environment, or the flags);
• either server or servers, never both;
• server.args is a list, and server.env and server.headers are mappings;
• packs is a list of pack ids;
• sql_tools is a list of { tool, arg } entries with string values;
• a {dsn:role} placeholder names a role that exists, and every ${VAR} reference in server.url, server.args, server.env and server.headers is set or has a :-default.

sealmark init starts the file with a yaml-language-server line that points an editor at the JSON Schema shipped in the package (./node_modules/sealmark/schema/sealmark.schema.json), for completion and checking while you type. sealmark config migrate rewrites an older file to the current schema_version.

name and version#

name and version describe the server under test. YAML reads version: 1.10 as the number 1.1, so quote versions that look like numbers: version: "1.10". A three-part version such as 1.4.0 is already text. A grade belongs to one name and version, so change version when the server changes.

server.command and server.args#

The harness runs the command directly, with no shell: write the executable in server.command and each argument in server.args, and use sh -c '...' if you need a shell. Placeholders in the command and in every argument are expanded (see Placeholders). A relative ./ or ../ path in the file is resolved against the folder of the config file, not the folder the server runs in.

server.rootless#

server.rootless (default false) says the server takes no root argument and works in its own working directory. The fs pack then treats that directory, {sandbox}, as the allowed root and runs every path probe, so a server that reads or writes any path fails the escape checks. It only adds coverage, so a STANDARD run stays STANDARD. servers.<name>.rootless sets it per server.

server.env#

server.env values may use placeholders and ${VAR} references.

server:
  command: node
  args: ["./server-a.js"]
  env: { DATABASE_URL: "{dsn}", API_KEY: "${MY_API_KEY}", LOG_LEVEL: info }

A sandboxed server does not inherit your shell's environment: it gets a short allow-list plus what you set here. See Environment.

server.files#

server.files stages small files into the sandbox before the server starts, for a server that reads a config file. It is a list of objects, so it is configuration-file only: no flag and no environment variable can carry it.

server:
  command: my-server
  args: ["--config", "{sandbox}/config/app.ini"]
  files:
    - path: "{sandbox}/config/app.ini"
      content: |
        mode=test

path may begin with {sandbox}, {dir} or {sandbox_home} and must stay inside that directory once expanded (.. is refused with exit code 2). content may use {sandbox}, {sandbox_home}, {listener}, {sqlite_path} and ${VAR}, and never a database placeholder, because the server is started once per database role and a file cannot change between roles: pass the connection string through args or env. A server Sealmark does not start in the sandbox (--url alone, --no-sandbox) cannot have staged files.

server.url and server.headers#

server.url and server.headers accept placeholders and ${VAR} references.

server:
  url: http://localhost:8000/mcp
  headers: { X-Team: demo, Authorization: "Bearer ${MCP_TOKEN}" }

Secrets: ${VAR} and sealmark.local.yaml#

A secret does not have to be in a file. ${VAR} and ${VAR:-default} in server.command, server.args, server.env, server.url and server.headers are read from your real environment when the run starts, and are never written to a report. A variable that is not set and has no :-default stops the run with exit code 2; write ${VAR:-} to allow an empty value. On the command line, put the reference in single quotes (--bearer '${MCP_TOKEN}') so your shell leaves it for Sealmark to expand.

Values that are specific to your machine, or that you would not commit, go in sealmark.local.yaml next to sealmark.yaml. Everything read from that file is scrubbed from every report format. Never put a real credential or a real database URL in sealmark.yaml. What a report prints and hides of your config is on Environment.

sql_tools#

sql_tools names the tools that take a SQL string, as pairs of the tool name and the name of its SQL argument. The db and sqlite packs send their probes through these tools.

sql_tools:
  - { tool: query, arg: sql }

When the list is omitted, the harness finds SQL tools in the server's tools/list answer:

• An argument named sql, sql_query or sqlquery counts on its own.
• An argument named query or statement counts only when the tool's name or description mentions sql, database, postgres or table.
• A tool that has other required arguments is kept. The harness fills them: an enum gets its first value, a string a token, a number its minimum or 1, a boolean false, and a schema default wins. Add args: { <param>: <value> } to a sql_tools entry (scalar values only) to set them yourself.

Every tool that looks like SQL but was skipped is listed under "Not covered" in the report, with the reason. A list you write adds to or corrects the automatic search and never hides a tool it finds (use tools.exclude to keep one out, which makes the run CUSTOM), and the harness checks your list against the server at run time. A tool the server does not declare, or an argument that is not a string property of that tool, stops the run with exit code 2.

upstream, scope and identities#

For a server that fronts a backend. upstream.kind http starts a harness-owned stub on loopback that the server must use as its backend through {upstream}; upstream.responses and upstream.default_reply script its answers, upstream.credential_header names the header the upstream pack reads and upstream.max_body_kib bounds what it keeps of each request. identities.a and identities.b are secrets: real caller tokens that replace the minted {bearer:a}/{bearer:b} for a server that validates tokens. scope.tools names the identifier parameter of each scoped tool, scope.allow and scope.deny what it may and must never serve (an entry ending in scope.separator is a prefix), and scope.case how names compare. None of these keys is locked, so declaring them keeps a run STANDARD. A worked example is in Verify a server that fronts a backend.

oauth#

For an HTTP server that validates OAuth bearer tokens. oauth.issuer stub (or the --oauth flag) starts a harness-owned authorization server on loopback and sends tokens it signs as Authorization: Bearer; point the server at it through {issuer}, {jwks} and {audience}. external grades only what a server already running tells a client without a token. oauth.audience is the audience the control token carries (default: the server URL). oauth.scopes lists the scopes the server requires: the control token carries them and the oauth-scope check sends a token without them. oauth.client_id is the client_id claim of the tokens. oauth.tls (default true, or --no-oauth-tls) serves the stub over https with a throwaway CA, written to {ca_file}; a server Sealmark starts trusts it through SSL_CERT_FILE, REQUESTS_CA_BUNDLE and NODE_EXTRA_CA_CERTS unless server.env sets them. None of these keys is locked, so declaring them keeps a run STANDARD.

tools#

tools.include and tools.exclude are lists of tool-name patterns that limit which tools Sealmark calls at all (include defaults to ["*"]). tools.exclude appends across layers. tools.overrides fixes a wrong classification, for example legacy_thing: { class: opaque }. A tool the classifier cannot place with enough confidence gets the core checks only and is listed as not covered, so an override is how you say what a tool takes.

checks#

Some check settings are locked by the standard profile. Changing a locked setting is allowed, but the run is then CUSTOM and the report lists the difference. sealmark list-checks shows which checks are locked, and sealmark explain <check-id> lists the settings that affect one.

checks:
  fs-size-cap: { enabled: false }    # allowed, but the run becomes CUSTOM
  robust-input: { max_tools: 10 }

A candidate check (db-privilege, the two core advisories and the checks of the candidate packs scope, sql-guard and upstream) is the exception: its settings are not locked, so a standard run's hash does not contain them, yet switching it off or giving it a severity still makes the run CUSTOM, and --replay never restores them. See Candidate checks.

output#

output.* never changes a grade, so it never makes a run CUSTOM.

output:
  formats: [summary, junit, json]
  fail_on: [regressions, drift]

Other keys#

The type, default and lock state of every key are in the generated table. The keys that are explained elsewhere:

Placeholders#

Placeholders are replaced just before a server starts. They work in server.command, server.args, the values of server.env, server.url, and the values of server.headers, and {sandbox}, {sandbox_home} and the other harness placeholders also in server.files. They are not replaced anywhere else.

{canary} and {token} exist only in probes:, {matrix.<name>} only in the args of a probe with matrix, and {upstream_token} only in a stub reply body. {listener} exists in probes: and in the content of server.files.

With a database placeholder in the config, the harness starts a Postgres (the embedded one, or your own with --pg and --yes) and the db pack starts the server once per role. With none, or with --no-embedded and no --pg, the db pack is not applicable and core still runs. See roles and server instances.

STANDARD and CUSTOM#

Every report starts with a header that says which it is, and a CUSTOM header lists each difference from the standard. A run is STANDARD only if every setting that affects a grade equals the standard profile, no standard pack was dropped, and no plugin or custom check ran. Adding coverage (the exec pack with --active, a probe without a check:) keeps it standard; weakening anything does not. The profile table above says which profiles can be STANDARD; sealmark config profiles also lists your own. The JSON record carries config.hash, config.standard and config.standardDiff, and --replay reproduces a run from them. A core-only or a CUSTOM run is not eligible for a badge.