From e8edd88efc817a27ca3428c74eaadf4c13eae2dd Mon Sep 17 00:00:00 2001 From: Heimdall Date: Fri, 6 Nov 2026 13:30:45 +0000 Subject: [PATCH] Specify answer collector record and consent Co-Authored-By: Claude Sonnet 5.5 --- docs/answer-collector-consent.md | 118 +++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 docs/answer-collector-consent.md diff --git a/docs/answer-collector-consent.md b/docs/answer-collector-consent.md new file mode 100644 index 0000000..f02be6a --- /dev/null +++ b/docs/answer-collector-consent.md @@ -0,0 +1,118 @@ +# Answer Collector: Record and Consent + +Status: specification. Nothing here is implemented. It fixes what an answer +record is and what a respondent has agreed to, so that a collector, a tally and +a training export can be built against one definition. + +Scope: free-text and choice answers that members give to a sample put to them +by an experiment. Not identity proof (see `architecture.md`). + +## Record + +One record per answer. Fields: + +| field | rule | +|---|---| +| `sample_id` | names the thing answered about, with a version component (`@`). A changed sample is a new version. | +| `experiment_id` | the experiment that asked. | +| `member_id` | Heimdall member id of the respondent. Never a Matrix id, never a display name. | +| `scope` | `show` \| `study` \| `train`. Default `show`. | +| `text` | the answer as given. | +| `revoked` | boolean, default false. One-way: a revoked record is never un-revoked; a new answer is a new record. | +| `received_at` | collector clock, UTC. | + +Key: `(experiment_id, sample_id, member_id)`. A member's later answer to the +same key replaces the earlier one (latest `received_at` wins); the earlier text +is not retained. + +## Versions + +Answers never tally across `sample_id` versions. A tally is computed for one +full `sample_id` including version; there is no "all versions" aggregate and no +API that takes a sample without a version. A version bump starts an empty tally. + +## Scope + +Scope is the widest use the respondent allowed. Each use reads exactly one +level: + +- `show`: may be displayed back inside the experiment (to the respondent, and + in aggregate to participants). Nothing else. +- `study`: `show`, plus analysis by the experiment's researchers. May be quoted + in findings only unattributed. +- `train`: `study`, plus use as training or evaluation data for models. + +Rule: nothing is training data unless `scope = train`. A missing, unparseable +or unknown scope is `show`, never wider. Scope can be narrowed by a new answer +or revocation; widening is a new answer by the respondent, never an inference. + +## Identity: count once, reward without exposure + +The Matrix sender is mapped to a Heimdall member id by Heimdall's linked +identity, at collection time. The collector stores `member_id` only; the Matrix +sender is discarded after mapping. A sender with no linked member is dropped, +not stored under a placeholder. + +- Count once: the key above makes one member one answer per sample version, + however many Matrix accounts or bridges they use. +- Reward without exposure: a reward is issued against `member_id` by the + rewarding service. Researchers and displays see counts and text under their + scope, never `member_id`. The mapping from a record to a person is available + to the reward path only. + +## Eureka's shape (Matrix) + +- One room per experiment. The room is `experiment_id`. +- The sample is posted as a message; an answer is a thread reply to it whose + body starts `scope: ` on its first line, the answer text following. +- The reply event is the receipt: its event id and timestamp are the proof the + answer was received, and `received_at` comes from it. +- Redaction is revoke: redacting the reply event sets `revoked`. +- The collector is on demand (it reads the room when run, it is not a standing + listener). It drops: + - redacted replies; + - replies with no `scope:` line (consent is explicit in Matrix; the `show` + default applies to records created elsewhere, not to silence); + - replies from unlinked senders; + - replies to an old sample version when the tally is for a newer one. +- Records wider than `show` (`study`, `train`) are kept apart from `show` + records: separate store or partition, so a read for one level cannot + accidentally return another. + +## What Matrix cannot guarantee + +- **Receipt outliving a deleted reply.** Redaction removes content, and + homeserver retention varies. If the collector holds a receipt for a reply that + later disappears, it keeps a tombstone: event id, `sample_id`, `received_at`, + and a hash of `text` (hash only, no text), so the count and the revoke are + provable without the content. The record is marked `revoked`; `text` is + erased. +- **Edits.** An edit is a new event, not the receipt. Only the original event + counts; edits are ignored, so what was consented to is what was received. To + change an answer, redact and reply again. +- **Bridge spoofing.** A bridge can post as any ghost user. Only senders whose + Matrix id resolves through Heimdall's linked identity to a member count; + bridged ghost ids are not linked identities. A bridge cannot raise scope, as + scope comes only from the reply's own `scope:` line. +- **Discord material.** Nothing that originates from Discord is collected, + even through a bridge, at any scope: Discord forbids training on its + content. Bridged events are dropped by origin, not by judging text. + +## What the respondent sees + +Posted with the sample, in the room: + +- the three scopes in one line each, as above; +- that no `scope:` line means the answer is not collected; +- how to revoke. + +After answering, the respondent sees their receipt: the thread reply itself, +plus the collector's acknowledgement that names the scope it recorded. If it +recorded narrower than they asked, it says so. + +## How to revoke + +Redact your reply. That is all. The next collection run marks the record +`revoked`, erases `text`, and excludes it from every tally and export. Anything +already exported at `train` before the run cannot be recalled by Matrix; the +experiment's page states the export cadence so the window is known.