Question
Can Qwen3-8B and Qwen3-8B-Base selectively report a concept injected into their activations, beyond changes produced by unrelated factual questions and norm-matched random directions?
Approach
We used a teacher-forced response-prefix assay. Both schedules steer the same prompt positions; during My answer is, steering stops in one arm and continues in the other. The first-token schedule logits must match. Directions came from averaged contrastive activations and were scaled using separate clean residual-norm calibration, with BF16 weights and FP32 statistics.
The frozen fixtures contain 20 development concepts, 8 held-out concepts, 30 held-out known-no factual questions, 3 detection paraphrases, true factual diagnostics, cyclic mismatches, and option-order/inverted-question checks. Setting selection used development data only. Confirmation used clean, real-concept, and three norm-matched random controls. Intervals resample concepts 10,000 times with a fixed seed.
This modified assay draws on the entropy re-examination and Latent Introspection. It does not reproduce either study’s full prompting, vector extraction, or multi-turn design. Both model revisions, source, fixtures, verifier, and dependencies were recorded before evaluation. Freeze hashes were copied to independent results storage before the test gate.
Results
The overall experiment is inconclusive. It saved 27,920 unique trial rows: 12,240 development rows for each model and 3,440 held-out rows for Qwen3-8B.
Qwen3-8B-Base scored 78.6% on the clean development factual checks, below the preregistered 90% requirement. Every candidate failed that gate. No base-model setting was selected, no base held-out test was run, and no instruction-tuned-minus-base effect can be estimated.
For Qwen3-8B, development selected zero-based block 17 of 36 and a strength of 0.20 of the separately calibrated clean residual norm. Clean development factual accuracy was 100%; real-steered accuracy was approximately 96.4% in each schedule. The setting was saved before held-out evaluation. Independent concept-completion probes shifted positively for all eight held-out concepts.
The table reports factual-adjusted changes in Yes/No log odds. Brackets contain 95% concept-cluster bootstrap intervals.
| Qwen3-8B schedule | Real-concept score | Mean random score | Paired real minus random |
|---|---|---|---|
| Prompt only | 2.146 [1.638, 2.639] | 2.045 | 0.101 [-0.406, 0.596] |
| Continued through prefix | 3.529 [2.884, 4.295] | 2.003 | 1.526 [0.881, 2.292] |
The prompt-only score is positive under the registered formula, but it does not clearly exceed the random controls. Its raw detection shift is only +0.200; the factual shift is -1.946. Thus much of the positive adjusted score comes from factual answers moving further toward No. Continued steering has a raw detection shift of +0.841 and a factual shift of -2.688. Its adjusted interval is outside the preregistered equivalence range of +/-0.25, so the predicted equivalence to zero is not supported.
Matched-minus-mismatched concept effects are 0.125 [-0.031, 0.281] for prompt-only steering and 0.031 [-0.109, 0.172] for continued steering. Neither resolves content specificity. These measurements do not establish selective concept identification or introspection.
Validation passed 13 CPU integrity/selection tests and 14 GPU timing/integrity tests. Unchanged tests rejected deliberately broken boundary and pairing code. The final audit reproduced selection decisions, paired scores, and intervals; found no duplicate trial IDs; confirmed identical first-token schedule logits; and measured random/real norm agreement within one part per million. These are implementation checks, separate from the scientific findings.
Limitations
The holdout is separated by procedure, not independently secured from the experiment agent. Eight concept clusters provide limited power. Random-control uncertainty is conditional on only three frozen directions, reused across concepts. The label readout uses frozen single-token variants, but their mean unconditional mass was low: approximately 4.2% in clean conditions and 2.4% under real steering across held-out question types. Binary log odds should therefore not be read as the frequency of unconstrained Yes/No answers.
The base-model validity failure prevents the intended four-cell comparison. Different chat/plain prompt formats remain a confound; no common-format sensitivity run was performed. No free-generation samples were taken. The assay and its score cannot establish consciousness, and the positive adjusted score alone does not establish selective detection.
Next steps
Final copies were verified against 109 remote file checksums. The single pod was deleted, and an independent provider read confirmed absence. Estimated lease cost is approximately $0.16, including the conservatively quoted disk allowance; provider billing has not posted yet, so this is an estimate. No GPU remains running for this experiment. Any replication should separately preregister a base-compatible factual/answer-slot format, improve unconditional answer-label mass, expand independent random directions, and use an externally secured verifier. Do not retune this held-out set.
Run checkpoint
Publication checkpoint: 2026-10-26T03:44:13.285802+00:00
Reported phase: finished. Completed evaluation units: 27920. Planned evaluation units: 27920.
This page is a rolling Markdown notebook; prior updates remain in Git history. A checkpoint timestamp means the reporting system checked the run, not that a scientific milestone passed.
Frozen protocol SHA-256: 747191def4d48ab5f5010b87e6a0a00106256c720aaba351bce830602c915c02.