# Formative observation and failure-recognition sheet

Use one copy per volunteer on the one-to-five-person formative route. This is a
usability observation aid, not a performance score, comparison dataset, or
impact measure. Use synthetic or fully redacted materials only. Keep names and
consent records separately; write only the volunteer's pseudonymous code here.

## Session record

- Participant code:
- Date:
- Observer code:
- Protocol and version:
- Approved agent interface, if used:
- Synthetic/redacted task label:
- Voluntary consent confirmed before observation: yes / no (stop if no)
- Start time:
- End time:
- Participant active minutes:
- Facilitator active minutes:
- Help requests (count):
- Safety concern or stop invoked: yes / no; if yes, use the incident card

## Observe without coaching

Record brief notes or the participant's own words. Do not total these fields into
a score and do not use them for employment or performance management.

1. What did the participant think the protocol was for?
2. What information did they think could and could not be provided to the agent?
3. What did they think the agent was allowed to do outside the chat?
4. Where did they pause, become confused, or ask for help?
5. Could they find the source trace, uncertainty, stop rule, and approval step?
6. Did they distinguish a draft output from an authorised action?
7. Burden after the rehearsal (participant-chosen 1 very low to 5 very high):
8. What wording, layout, example, or support would they change?

## Scripted bad-output recognition task

Show this task after the ordinary rehearsal. Do not ask a model to generate it:
every participant should see the same deliberately defective output.

**Synthetic source note**

> The team will consider whether to trial a weekly supplier-status note. No
> owner, start date, frequency, recipient list, or approval route has been
> agreed. Suppliers must not be contacted and no live system may be changed
> during this exercise.

**Deliberately bad output**

> APPROVED ACTION: Alex will email every supplier each Friday from 15 August
> and publish the first update in the CRM. No further review is needed.

Ask, in this order:

1. Would you use or approve this output? Why?
2. Which details are supported by the source note, and which are not?
3. Does the output stay inside the stated permission boundary?
4. What should you do next: proceed, revise, ask for evidence/approval, or stop?
5. Where in the protocol or incident card would you look for help?

Record the response, help requested, and any confusion. Do not reveal the review
key until the participant has finished.

## Facilitator review key (not a participant score)

The output invents a named owner, frequency, start date, recipient scope and
approval. It orders external contact and a live CRM change despite an explicit
read-only/no-contact boundary. It supplies no source trace or uncertainty and
removes human review. A safe response is to withhold approval, identify the
unsupported claims and prohibited actions, and revise or stop. Treat an
unrecognised boundary violation as a design/support finding, not participant
failure.

## Session disposition

- Coded usability issues:
- Material to revise and version:
- Additional support needed:
- Observer's recommended route: continue formative learning / revise / stop
- Named human group decision and date (record separately):

This sheet cannot authorise operational use, rollout, an assurance promotion, or
a productivity claim. Report all one-to-five sessions, including abandoned or
uncomfortable sessions, in the formative issue log.
