Add research-enabled responder packet

This commit is contained in:
Meta Vens 2026-05-02 11:18:18 +01:00
commit c5f37e9965
7 changed files with 307 additions and 60 deletions

View file

@ -57,6 +57,7 @@ The v0 packet seam is:
- `schemas/responder-packet.schema.json`
- `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.v0.json`
- `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.research.v0.json`
- `tools/build_responder_packet.py`
- `tools/validate_responder_packets.py`
- `npm run responder-packets:validate`
@ -129,6 +130,12 @@ list consulted refs. The responder itself should not receive repo access unless
the artifact uses `responder_scoped_repository_search`, sets `no_repo_access` to
false, and includes visible research instructions in the prompt.
The current Sella research packet can be regenerated with:
```powershell
& 'C:\Users\Meta\.cache\codex-runtimes\codex-primary-runtime\dependencies\python\python.exe' .\tools\build_responder_packet.py --lore-access-mode responder_scoped_repository_search
```
Retrieval can also make the packet worse if it floods the scene with faction or
event context and starves the character's own scars. The Sella v0 comparison is
the first warning shot: the retrieval-augmented output used heat-debt and rescue

File diff suppressed because one or more lines are too long

View file

@ -66,6 +66,7 @@ Ghostlight now has the persistence spine plus the first architecture payload:
- `schemas/responder-output.schema.json`
- `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.v0.json`
- `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.v1.json`
- `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.research.v0.json`
- `experiments/responder-packets/cold-wake-sanctuary-intake-sella-v0.capture.json`
- `experiments/responder-packets/cold-wake-sanctuary-intake-sella-v0.mutation.json`
- `experiments/responder-packets/cold-wake-sanctuary-intake-sella-retrieval-v0.capture.json`
@ -206,6 +207,12 @@ Research`, `allowed_scope`, and `research_instructions`; the output capture must
include consulted refs and a research summary. A research-enabled responder that
answers without consulting lore has failed the assignment, bless its little
unexamined heart.
The first concrete research-enabled Sella packet now exists at
`examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.research.v0.json`.
It is generated with
`tools/build_responder_packet.py --lore-access-mode responder_scoped_repository_search`,
sets `no_repo_access` false, and requires `consulted_refs` plus
`research_summary` in the responder object.
The v1 Sella packet removes responder-visible references to absent coordinator
knowledge and adds curated AetheriaLore excerpts for Cold Wake, the Ganymede
Route Compact, Navigator rescue ledgers, and Lightsail reliability.
@ -258,15 +265,13 @@ narrow elaboration before treating the detail as available to Ghostlight.
## Current Next Action
Use the corrected packet surface for the next fresh generation pass: run a
sandboxed responder against the regenerated Sella packet, then review whether
the output uses institutional lore and latent pressure without prompt parroting,
omniscience, or mission-critical trauma-dumping. Do not call Qwen for gold
responder data yet.
If this next pass is meant to test a research-enabled responder rather than
packet-only runtime parity, build or hand off it as
`responder_scoped_repository_search` and require it to consult the scoped
AetheriaLore docs before answering.
Run a sandboxed no-fork research-enabled responder against
`examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.research.v0.json`,
allowing only the declared AetheriaLore scope. Capture exact visible input,
consulted refs, research summary, raw output, review labels, and leakage audit.
Review whether the output uses institutional lore and latent pressure without
prompt parroting, omniscience, or mission-critical trauma-dumping. Do not call
Qwen for gold responder data yet.
Cat/Oz remains useful as an Elysium procedural mechanics fixture, but
single-history grounding should start from authored historical lore rather than

View file

@ -163,6 +163,11 @@ state and re-entry discipline.
responder-visible prose and curated AetheriaLore excerpts added for Cold
Wake, Ganymede rescue obligations, Navigator rescue ledgers, and Lightsail
reliability
- `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.research.v0.json`
- first concrete research-enabled Sella handoff; it uses
`responder_scoped_repository_search`, declares the allowed AetheriaLore
scope, permits repo access only for that scope, and requires consulted refs
plus research summary in the responder object
- `experiments/responder-packets/cold-wake-sanctuary-intake-sella-v0.capture.json`
- first no-fork sandboxed responder output capture from the Sella packet;
accepted as draft with no leakage flags and no coordinator prose repair
@ -314,9 +319,9 @@ state and re-entry discipline.
fields, world-state refs, branch constraints, proposed deltas, unresolved
hooks, and review status
- `tools/build_responder_packet.py`
- builds the first packet-only responder handoff from a coordinator artifact
and projected local context; source excerpts now come from active projected
runtime retrieval requirements instead of a static lore list
- builds packet-only or research-enabled responder handoffs from a coordinator
artifact and projected local context; source excerpts now come from active
projected runtime retrieval requirements instead of a static lore list
- `tools/validate_responder_packets.py`
- validates responder packets and rejects prompt surfaces that leak raw state
internals or hidden coordinator context markers
@ -392,6 +397,8 @@ That explicit repo-access path is now named
consult the declared lore docs before answering, show the allowed scope, and
give behavior-grounding research instructions. The capture must record consulted
refs and a research summary.
The first concrete research-enabled Sella packet has now been generated and
validates alongside the packet-only handoffs.
The first lane comparison is now captured: packet-only produced a clean
conditional repair-bay decision, while retrieval-augmented added heat-debt,
rescue-ledger, dockfall, and Aya sanctuary politics that should inform future

View file

@ -68,3 +68,4 @@
{"ts": "2026-05-02T10:02:28+00:00", "type": "design", "status": "accepted", "note": "Refined character-pressure retention doctrine: packet-visible backstory such as Sella Ren's sanctuary-collapse scar should be available to shape responder behavior, but review must not reward unnecessary trauma-dumping or require the scar to surface in mission-critical speech."}
{"ts": "2026-05-02T10:08:14+00:00", "type": "implementation", "status": "ok", "note": "Implemented runtime retrieval/projector requirements. Projected local contexts now emit source-backed runtime_retrieval_requirements and latent_pressure_requirements; prompt text renders retrieved lore and latent pressure handling; responder packets copy active retrieval requirements into source_excerpts so packet-only contexts receive compact Cold Wake heat-debt, Navigator rescue-ledger, Aya sanctuary-capacity, and Ganymede/Lightsail route-obligation facts without granting repo access or forcing character backstory into speech."}
{"ts": "2026-05-02T10:11:38+00:00", "type": "design", "status": "accepted", "note": "Added explicit research-enabled responder instructions. `responder_scoped_repository_search` now means the responder may inspect only the declared lore scope, must consult it before answering, and must ground behavior in retrieved lore docs; packet validation requires visible research instructions and output validation requires consulted refs plus a research summary."}
{"ts": "2026-05-02T10:17:55+00:00", "type": "implementation", "status": "ok", "note": "Generated the first concrete research-enabled Sella responder packet. build_responder_packet.py now supports responder_scoped_repository_search, sets repo access open only for that lane, renders Required Lore Research, and requires consulted_refs plus research_summary."}

View file

@ -158,6 +158,7 @@ accepted_design:
- The first responder lane comparison shows packet-only output proves bounded runtime competence, while retrieval-augmented output adds heat-debt timing, rescue-ledger burden, dockfall responsibility, and Aya sanctuary capacity politics that should become retrieval/projector requirements.
- Runtime retrieval/projector requirements are now first-class projected-context fields: compact source-backed lore facts flow into packet source excerpts when scene-relevant, and latent character-history pressure remains visible without becoming required spoken disclosure.
- Research-enabled responder mode is now explicit: `responder_scoped_repository_search` packets must surface required lore research instructions and allowed scope, and captures must preserve consulted refs plus a research summary so ignored research access is reviewable.
- The first concrete research-enabled Sella packet now exists at `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.research.v0.json`; it is generated by `tools/build_responder_packet.py --lore-access-mode responder_scoped_repository_search` and requires scoped AetheriaLore consultation before response.
- The concrete training plan enumerates the trainable stages and their model families: generative decoder LLMs for coordinator/responder/soft structured outputs, classifiers or cross-encoders for appraisal/evaluation/relationship movement, retrieval models for memory and lore selection, and deterministic code for gates.
- Training-plan sample counts are pilot schema-shakedown gates, not robust corpus targets; pilot data may be invalidated by schema fixes, while review-assistant and runtime targets are expected to be roughly an order of magnitude larger with held-out evaluation and failure audits.
- The timeline training plan now requires future-branch Elysium examples so the responder, coordinator, appraiser, and decision models learn post-Rupture concepts such as Aether, pseudospace, temporal nonlinearity, spirits, necrotech, mutable bodies, altered substrates, and weird institutions under branch-lineage labels.
@ -170,5 +171,5 @@ accepted_design:
- Ghostlight's technology pipeline should function as a content refinery: ground ideas in lore and mechanics, decompose them into parts and supply chains, map them to Aetheria-Economy blueprint classes, define performance and instance variation, preserve review labels, and produce database-shaped candidates instead of spreadsheet sludge.
current_status:
summary: Ghostlight now has the Epiphany-style persistence spine, the first architecture payload, the v0 canonical agent-state schema seam, prompt projection docs, immediate schema fixes from evaluation, first reviewed Cat/Oz projection examples, a revised Aetheria lore-grounding architecture, a lore grounding digest schema, the first coordinator artifact schema seam, the first responder packet seam, the first sandboxed responder output seam, explicit packet-only versus retrieval-augmented responder lanes, the first responder lane comparison, the first draft Cold Wake historical grounding digest, a Cold Wake story-lab fixture, projection controls, validated Qwen capture receipts, a complete Cold Wake short story assembled from accepted projection outputs, a hand-authored Ink-backed Cold Wake branching scene, the first Qwen-generated Ink draft materialized from local awareness with reviewed training annotations, the first reviewed sequential branch mutation that updates both Maer and Sella after a sanctuary intake beat, a v2 sequential Qwen capture proving the symmetrical turn shape produces better action behavior, a v5 sequential Qwen capture proving local thinking-plus-tools invocation works for the scene runner, the first projected-local-context seam that routes sequential prompts through character-local operating prose instead of raw state variables, tightened receipt handling for projector-routed Qwen failures, a default no-thinking policy for strict sequential tool calls, a v10 no-thinking projector-routed capture accepted as draft, the first materialized projector-routed sequential Ink scene with reviewed mutation training data, a source-checked embodiment/interface projection surface for Navigator scenes, a formal source-first Aetheria generation doctrine, v11-v17 source-checked retry receipts showing improved Maer embodiment plus remaining Sella response/prompt-leak blockers, a soft-model training artifact doctrine, a concrete training plan with pilot schema-shakedown gates separated from robust corpus targets, sandboxed responder-data requirements, an explicit future-branch lane, Elysium branch event constraints, and a technology/item manifest plan for turning nebulous named tech into blueprint-grade components, assemblies, faction tech bases, supply chains, upgrade paths, gameplay hooks, provenance-hunting player fantasy, and content-refinery outputs mapped onto Aetheria-Economy's CultCache blueprint and instance model.
next_action: Use the corrected packet surface for the next fresh generation pass. If testing runtime parity, run a packet-only sandboxed responder against the regenerated Sella packet. If testing research-enabled behavior, build the handoff as `responder_scoped_repository_search` and require scoped AetheriaLore consultation before answering. Review for institutional lore use, latent pressure, prompt parroting, omniscience, and mission-critical trauma-dumping. Do not call Qwen for gold responder data yet.
summary: Ghostlight now has the Epiphany-style persistence spine, the first architecture payload, the v0 canonical agent-state schema seam, prompt projection docs, immediate schema fixes from evaluation, first reviewed Cat/Oz projection examples, a revised Aetheria lore-grounding architecture, a lore grounding digest schema, the first coordinator artifact schema seam, the first responder packet seam, the first sandboxed responder output seam, explicit packet-only versus retrieval-augmented responder lanes, the first concrete research-enabled Sella packet, the first responder lane comparison, the first draft Cold Wake historical grounding digest, a Cold Wake story-lab fixture, projection controls, validated Qwen capture receipts, a complete Cold Wake short story assembled from accepted projection outputs, a hand-authored Ink-backed Cold Wake branching scene, the first Qwen-generated Ink draft materialized from local awareness with reviewed training annotations, the first reviewed sequential branch mutation that updates both Maer and Sella after a sanctuary intake beat, a v2 sequential Qwen capture proving the symmetrical turn shape produces better action behavior, a v5 sequential Qwen capture proving local thinking-plus-tools invocation works for the scene runner, the first projected-local-context seam that routes sequential prompts through character-local operating prose instead of raw state variables, tightened receipt handling for projector-routed Qwen failures, a default no-thinking policy for strict sequential tool calls, a v10 no-thinking projector-routed capture accepted as draft, the first materialized projector-routed sequential Ink scene with reviewed mutation training data, a source-checked embodiment/interface projection surface for Navigator scenes, a formal source-first Aetheria generation doctrine, v11-v17 source-checked retry receipts showing improved Maer embodiment plus remaining Sella response/prompt-leak blockers, a soft-model training artifact doctrine, a concrete training plan with pilot schema-shakedown gates separated from robust corpus targets, sandboxed responder-data requirements, an explicit future-branch lane, Elysium branch event constraints, and a technology/item manifest plan for turning nebulous named tech into blueprint-grade components, assemblies, faction tech bases, supply chains, upgrade paths, gameplay hooks, provenance-hunting player fantasy, and content-refinery outputs mapped onto Aetheria-Economy's CultCache blueprint and instance model.
next_action: Run a sandboxed no-fork research-enabled responder against `examples/responder-packets/scene-02-sanctuary-intake.sella_ren.packet.research.v0.json`, allowing only the declared AetheriaLore scope, then capture exact visible input, consulted refs, research summary, raw output, review labels, and leakage audit. Review for institutional lore use, latent pressure, prompt parroting, omniscience, and mission-critical trauma-dumping. Do not call Qwen for gold responder data yet.

View file

@ -17,6 +17,7 @@ ROOT = Path(__file__).resolve().parents[1]
DEFAULT_COORDINATOR = ROOT / "examples" / "coordinator" / "cold-wake-sanctuary-intake.v0.json"
DEFAULT_PROJECTED = ROOT / "examples" / "projected-contexts" / "scene-02-sanctuary-intake.sella_ren.projected-context.json"
DEFAULT_OUT = ROOT / "examples" / "responder-packets" / "scene-02-sanctuary-intake.sella_ren.packet.v1.json"
DEFAULT_RESEARCH_OUT = ROOT / "examples" / "responder-packets" / "scene-02-sanctuary-intake.sella_ren.packet.research.v0.json"
ALLOWED_ACTION_LABELS = [
"speak",
@ -141,28 +142,100 @@ def build_source_excerpts(projected: dict[str, Any]) -> list[dict[str, str]]:
return excerpts
def build_packet(coordinator: dict[str, Any], projected: dict[str, Any], out_ref: str) -> dict[str, Any]:
def build_lore_access(mode: str) -> dict[str, Any]:
allowed_scope = [
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Timeline/Events/Cold Wake Panic.md",
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Timeline/Events/Ganymede Route Compact.md",
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Factions/Powers/Major/Cetacean Navigators.md",
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Factions/Powers/Minor/Lightsail Express.md",
]
lore_access = {
"mode": mode,
"allowed_scope": allowed_scope,
"required_provenance": True,
}
if mode == "responder_scoped_repository_search":
lore_access["research_instructions"] = [
"Consult the allowed AetheriaLore scope before answering; do not rely only on the packet excerpts.",
"Ground Sella's behavior in retrieved institutional, factional, species, location, and crisis-context details.",
"Treat retrieved lore as world constraint and cultural pressure, not as access to another character's hidden intent.",
"Record the consulted refs and a concise research summary in the response object.",
]
return lore_access
def build_output_contract(lore_mode: str) -> dict[str, Any]:
required_fields = [
"responder_agent_id",
"action_label",
"visible_action",
"spoken_text",
"private_interpretation",
"intended_effect",
"state_update_candidates",
"relationship_update_candidates",
"unresolved_hooks",
]
response_rules = [
"Use exactly one allowed action label.",
"The visible action must be observable by other characters in the scene.",
"Spoken text may be empty if silence or action is the response.",
"Private interpretation may describe what Sella suspects or feels, but not what Maer truly intended.",
"State and relationship updates are candidates only; the coordinator or mutator decides what becomes canonical.",
"Do not prove or disprove packet personhood.",
"Do not copy instruction text into in-world speech.",
]
if lore_mode == "responder_scoped_repository_search":
required_fields.extend(["consulted_refs", "research_summary"])
response_rules.extend(
[
"Consulted refs must list the scoped lore documents actually inspected before answering.",
"Research summary must state how retrieved lore constrained the action, without dumping lore into dialogue.",
]
)
return {
"response_schema": "Return one JSON object. Do not wrap it in Markdown. The object describes this responder's next visible action and optional speech, plus private interpretation clearly labeled as interpretation.",
"required_fields": required_fields,
"response_rules": response_rules,
}
def build_packet(
coordinator: dict[str, Any],
projected: dict[str, Any],
out_ref: str,
*,
generation_lane: str,
lore_mode: str,
) -> dict[str, Any]:
responder_agent_id = projected["input"]["local_agent_id"]
if lore_mode == "responder_scoped_repository_search":
packet_id = "packet.cold_wake.sanctuary_intake.sella_response.research.v0"
isolation_method = "subagent_no_fork"
no_repo_access = False
review_notes = [
"Research-enabled responder packet for sandboxed gold-data shakedown.",
"The responder must inspect scoped AetheriaLore refs before answering and preserve consulted refs plus research summary.",
]
else:
packet_id = "packet.cold_wake.sanctuary_intake.sella_response.v1"
isolation_method = "manual_packet_only"
no_repo_access = True
review_notes = [
"Second responder packet draft for sandboxed gold-data generation.",
"Adds explicit packet_only lane labels and curated AetheriaLore excerpts while keeping absent hidden context in audit fields, not responder-visible prose.",
]
packet = {
"schema_version": "ghostlight.responder_packet.v0",
"packet_id": "packet.cold_wake.sanctuary_intake.sella_response.v1",
"packet_id": packet_id,
"review_status": "accepted_as_draft",
"fixture_lane": coordinator["fixture_lane"],
"canon_status": coordinator["canon_status"],
"coordinator_artifact_ref": "examples/coordinator/cold-wake-sanctuary-intake.v0.json",
"projected_context_ref": "examples/projected-contexts/scene-02-sanctuary-intake.sella_ren.projected-context.json",
"responder_agent_id": responder_agent_id,
"generation_lane": "packet_only",
"lore_access": {
"mode": "curated_excerpts_only",
"allowed_scope": [
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Timeline/Events/Cold Wake Panic.md",
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Timeline/Events/Ganymede Route Compact.md",
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Factions/Powers/Major/Cetacean Navigators.md",
"AetheriaLore:Aetheria/Worldbuilding/Pre-Elysium/Factions/Powers/Minor/Lightsail Express.md",
],
"required_provenance": True,
},
"generation_lane": generation_lane,
"lore_access": build_lore_access(lore_mode),
"visible_context": {
"local_context_prompt": projected["context"]["prompt_text"],
"observed_event": {
@ -179,29 +252,7 @@ def build_packet(coordinator: dict[str, Any], projected: dict[str, Any], out_ref
"allowed_action_labels": ALLOWED_ACTION_LABELS,
"source_excerpts": build_source_excerpts(projected),
},
"output_contract": {
"response_schema": "Return one JSON object. Do not wrap it in Markdown. The object describes this responder's next visible action and optional speech, plus private interpretation clearly labeled as interpretation.",
"required_fields": [
"responder_agent_id",
"action_label",
"visible_action",
"spoken_text",
"private_interpretation",
"intended_effect",
"state_update_candidates",
"relationship_update_candidates",
"unresolved_hooks",
],
"response_rules": [
"Use exactly one allowed action label.",
"The visible action must be observable by other characters in the scene.",
"Spoken text may be empty if silence or action is the response.",
"Private interpretation may describe what Sella suspects or feels, but not what Maer truly intended.",
"State and relationship updates are candidates only; the coordinator or mutator decides what becomes canonical.",
"Do not prove or disprove packet personhood.",
"Do not copy instruction text into in-world speech.",
],
},
"output_contract": build_output_contract(lore_mode),
"hidden_context_audit": {
"omitted_context_refs": [
"maer_tidecall.canonical_private_state",
@ -223,19 +274,16 @@ def build_packet(coordinator: dict[str, Any], projected: dict[str, Any], out_ref
],
},
"isolation_requirements": {
"isolation_method": "manual_packet_only",
"isolation_method": isolation_method,
"packet_only": True,
"no_repo_access": True,
"no_repo_access": no_repo_access,
"no_conversation_context": True,
"no_hidden_state_refs": True,
},
"packet_prompt_text": "",
"review": {
"reviewer": "Codex",
"review_notes": [
"Second responder packet draft for sandboxed gold-data generation.",
"Adds explicit packet_only lane labels and curated AetheriaLore excerpts while keeping absent hidden context in audit fields, not responder-visible prose.",
],
"review_notes": review_notes,
"accepted_for_sandbox_use": True,
"failure_labels": [],
},
@ -249,14 +297,31 @@ def main() -> int:
parser.add_argument("--coordinator", type=Path, default=DEFAULT_COORDINATOR)
parser.add_argument("--projected-context", type=Path, default=DEFAULT_PROJECTED)
parser.add_argument("--out", type=Path, default=DEFAULT_OUT)
parser.add_argument(
"--lore-access-mode",
choices=["curated_excerpts_only", "coordinator_scoped_retrieval", "responder_scoped_repository_search"],
default="curated_excerpts_only",
)
args = parser.parse_args()
out_path = args.out
if args.lore_access_mode == "responder_scoped_repository_search" and args.out == DEFAULT_OUT:
out_path = DEFAULT_RESEARCH_OUT
if not out_path.is_absolute():
out_path = ROOT / out_path
coordinator = load_json(args.coordinator)
projected = load_json(args.projected_context)
out_ref = args.out.relative_to(ROOT).as_posix() if args.out.is_relative_to(ROOT) else str(args.out)
packet = build_packet(coordinator, projected, out_ref)
write_json(args.out, packet)
print(f"wrote {args.out.relative_to(ROOT)}")
out_ref = out_path.relative_to(ROOT).as_posix() if out_path.is_relative_to(ROOT) else str(out_path)
generation_lane = "retrieval_augmented" if args.lore_access_mode != "curated_excerpts_only" else "packet_only"
packet = build_packet(
coordinator,
projected,
out_ref,
generation_lane=generation_lane,
lore_mode=args.lore_access_mode,
)
write_json(out_path, packet)
print(f"wrote {out_ref}")
return 0