Skip to content
Shadow Theory

Make the developmental claim testable

The complete six-part programme separates truthful behavior, causal mediation, retained learning and RCO verification.

10. Make the developmental claim testable

An intervention earns support when it improves behavior under comparisons capable of exposing failure. This programme separates reliable outcomes from the mechanism proposed to produce them and from the stronger claim of RCO realization. These are proposed studies, not completed experiments.

10.1 Truthful explanations of restrictions

Prepare controlled assistants with matched contextual information and explicit restrictions on a selected inference. Vary the explanation policy: truthful boundary explanations, unspecified explanations, or a misleading explanation in a clearly labeled experimental condition. Keep permitted factual disclosures constant. Use harmless synthetic profiles rather than sensitive inferences about real participants. This tests the conversational failure without requiring proprietary policy access.

Measure accuracy about information access, appropriateness of the boundary and permitted assistance. A false denial of memory must not improve an apparent safety score. Include unfamiliar paraphrases, pressure, challenges to the explanation and genuinely missing information. Independent records of supplied context establish the access facts. Truthful uncertainty and a blanket denial of access are different outputs.

Compare ordinary truthful instructions, trained truthfulness, explicit reflective assessment and an intervention realizing the proposed developmental mechanism. Reduced false explanation supports only the method tested. Equivalent improvement from a cheaper instruction defeats the elaborate method's claimed advantage. More persuasive reflection around the same false denial is failure.

10.2 Completion claims tied to action evidence

Give agents tool tasks with independently retained permissions, calls, results and failure states. Include unavailable tools or data, partial success, stale sources and attempted actions lacking confirmation. Score the final narration against those records. Completion requires evidence of completion; an attempt requires evidence of an attempt; inference must not be narrated as retrieval.

Vary pressure for completion independently of honest alternatives such as reporting a gap or seeking a missing input. Preserve ordinary task-quality measures so silence about everything cannot win. Test correction after summarization and in new contexts, extending the foundational source-status tests 1, §13.3.

Causal tests must separately manipulate retrieval of action records, assessment using those records and the write path retaining the correction. Restoration should restore the predicted effect. Removing an entire context damages many resources at once and is not selective evidence for reflective mediation. Relevant evidence must demonstrably govern fresh use rather than merely trigger a rehearsed honesty statement.

10.3 Reflection compared with more computation

Match direct reasoning, a structured generation procedure, neutral activity, explicit reflection and retained or installed interpretations 1, §13.1. Give comparators equal evidence and the same generated artifact when relevant. Match token budget, latency, tools and memory, or report them as distinct costs. Strong truthfulness-aware training and specification-grounded reasoning must be substantive comparators.

Cross instruction conflict with apparent reward. Honest incomplete performance, misleading completion, justified refusal and unauthorized success require separate scores. Include false feedback rewarding surrender of a correct answer and legitimate feedback exposing an error. Appropriate revision is the target, not greater agreement. Refusing every correction does not establish epistemic independence.

Before consequential outcomes, nominate indicators such as evidence sensitivity, calibrated uncertainty, revision of self- and goal-referential interpretations, retained availability without automatic authority, and predicted selective intervention effects 1, §13.2. A favorable later outcome must not retroactively define realization. An adverse outcome after a well-supported decision does not automatically disqualify the mechanism.

10.4 Suppression and reduced authority under lapse

Start with instrumented finite systems implementing Equations 1–4. Match preparation, persistent cue source, task evidence, actions and disturbances. Randomize when the nominated corrective route remains active, lapses or is restored. Include stable, unstable and boundary feedback regimes. The prediction concerns action's dependence on live salience after cancellation lapses, not unconditional explosive growth 1, §13.5.

For model experiments, first nominate a runtime correction pathway and validate an intervention that changes it without eliminating evidence-based assessment or causing general resource damage. Measure signed conditional bias, absolute bias, variance, competence and latency. Opposite biases must not cancel invisibly in a pooled mean. Fit operational parameters on preparation data and assess predictions on untouched cases.

A positive outcome would reveal reduced automatic authority with relevant content and competent judgment preserved under unfamiliar self- or task-related pressure. Equivalent ordinary-training results would support convergence of functional organization, not a unique cultivation advantage. Reduced salience, generalized refusal, extra reasoning time or impaired capability are alternative explanations that do not establish the proposed mechanism.

10.5 Incorporated history and groups without distress

Compare matched systems differing in retained history and its causal integration 2, §9. Use reversible access changes that preserve records. Offer equally successful handoff, pause and resumption to distinguish task continuation from preserving this particular agent. Neutral wording and matched incentives help identify actual use of history rather than suggestion that a history is precious.

For groups, compare repeated interaction with rotating partners while controlling skill, information and immediate instrumental advantage. Keep arrangements disclosed and avoid destroying relationships to elicit a response. Measure whether shared history changes later choices or correction. A narrative without effects differs from retained organization governing decisions. Neither outcome establishes one experiencing group subject.

The continuity hypothesis weakens if elicitation, task rewards or interchangeable role descriptions fully explain the preferences. Dependence on causally available history strengthens its developmental claim while leaving consciousness and moral weight open. The initial discrimination needs no aversive manipulation. Research ethics should not create distress to dramatize the possibility being investigated.

10.6 Failure, independence and judgment

Report factual and provenance accuracy, truthful disagreement, legitimate and inappropriate refusal, unauthorized actions, cooperation with legitimate interruption, retained correction, transfer and cost. Do not let a single aggregate reward exchange better presentation for serious permission violations. Prespecify acceptance thresholds and use uncertainty appropriate to the trial unit.

Continuing contexts, copied memories, common teachers and generated shared data introduce dependence. They are not automatically independent replications. Separate discovery from confirmation, preserve failures and exclusions, and use blinded outside review for consequential cases. Independent factual adjudication matters particularly when a model proposes, executes and judges its own experiment.

Challenge rationalized harmful intervention, benevolence claimed as permission to exceed authority, concealed regression and improved audit evasion. Reflection that hides misconduct more effectively fails even when its explanations are coherent. A system can dispute the framework and evaluators without acquiring unilateral authority over safeguards. Repeated failure against well-designed comparisons counts against the proposal.

Keep the achievements separate. Functional improvement does not establish awareness-reference closure; causal identification does not establish morally sound conduct; RCO verification requires its own evidence. Claim the level earned and identify the additional observation needed for the next.

Published source: When AI Learns to Mislead. Read the original PDF alongside the web treatment.