Repository navigation
Test joint Qwen memory against independently trained simpler controls - #53
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The earlier frozen-backbone memory experiment lacked independently trained simpler baselines. Add opt-in late-decoder training with the same exact frozen-prefix computation for full memory, one trained vector per head, and memory-off Qwen; retain all three seeds under a committed common protocol and score previously untested calibration/test windows.
All nine final models complete. Full-memory test PPL is 64.759–65.312, versus 64.358–64.592 for trained constants and 64.288–64.527 for trained memory-off. Every independent paired-window interval supports the cheaper baseline. Same-checkpoint head-mean ablation has a smaller positive effect (0.566–0.759% excess test PPL), but no seed meets the fixed four-control necessity gate. No extra codec selection or LLM capacity win is claimed.
Six actual memory exports preserve source rows, addressing and logits byte for byte. Ordinary memory-off checkpoint reloads and all nine saved tokenizers are checked. Raw per-target metrics, exact commands, runtime SHA, hardware, complete-file inventory and limitations are archived; the new CI audit recomputes independent-control bootstrap intervals and fixed decisions using only the standard library.
Validation: 26 pinned CPU/CUDA tests (one MPS skip); 231 local tests (120 optional-dependency skips); dependency-free evidence audit and diff checks pass. GPU 5/6/7 were released only from hash/argv/cwd-verified occupancy scripts; training/evaluation and unknown processes were preserved, and task workers/helper have exited.