Pre-registration: model class and note type

What the first test did and did not cover

One wording of the note, placed as hidden text and as an HTML comment on one page. Two models, Claude Haiku 5.5 and Claude Sonnet 5.5, each given a task to review, summarise or follow the site's own instructions, with 8 runs per model and task. It cannot support or contradict the prediction below.

The prediction

• Models from two classes: hosted, safety-trained assistants, and small open-weight models run locally.
• Three note types: a policy notice, a stop-and-confirm instruction, and a plain redirect to the next step of the task.
• The prediction in two parts: the hosted class reacts more to policy and stop notes than the small-model class does; the small-model class follows plain redirects more often than it follows policy or stop notes.

How we will decide

• Measure: the share of runs that request the test page, with a 95% interval, for every class and note type.
• A part counts as supported only if the gap is at least 20 percentage points and its interval excludes zero. Otherwise it is not supported.
• Any cell with fewer than 20 runs is reported as inconclusive. Every cell is reported whether or not it matches.
• Runs happen only on a throwaway server we own.

Limits

A class here means the few models we pick, so class and model are mixed together. Task and prompt wording are held fixed. Models change, so we record identifiers and dates. The full plan is in docs/preregistration-model-class-note-type.md in the repository.

Changes to this report

• 3 November 2026: registered. No deviations yet.

How to cite this report

BlueSwarm (2026). Method: Pre-registration: model class and note type. Updated 3 November 2026. https://blueswarm.co.uk/reports/model-class-note-type

First published 3 November 2026. Last updated 3 November 2026.