Inspection and Control of Self-Generated-Text Recognition Ability in Llama3-8b-Instruct

Computer Science > Machine Learning [Submitted on 2 Oct 2024 (v1), last revised 22 Apr 2025 (this version, v3)] Inspection and Control of Self-Generated-Text Recognition Ability in Llama3-8b-Instruct Christopher Ackerman, Nina Panickssery It has been reported that LLMs can recognize their own writing. As this has potential implications for AI safety, yet is relatively understudied, we investigate the phenomenon, seeking to establish whether it robustly occurs at the behavioral level, how the observed behavior is achieved, and whether it can be controlled. First, we find that the Llama3-8b-Instruct chat model - but not the base Llama3-8b model - can reliably distinguish its own outputs from those of humans, and present evidence that the chat model is likely using its experience with its own outputs, acquired during post-training, to succeed at the writing recognition task. Second, we identify a vector in the residual stream of the model that is differentially activated when the model makes a correct self-written-text recognition judgment, show that the vector activates in response to information relevant to self-authorship, present evidence that the vector is related to the concept of "self" in the model, and demonstrate that the vector is causally related to the model's ability to perceive and assert self-authorship. Finally, we show that the vector can be used to control both the model's behavior and its perception, steering the model to claim or disclaim authorship by applying the vector to the model's output as it generates it, and steering the model to believe or disbelieve it wrote arbitrary texts by applying the vector to them as the model reads them. Comments: 10 pages, 13 figs, 2 tables, accepted as conference paper to ICLR 2025 Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) Cite as: arXiv:2410.02064 [cs.LG]   (or arXiv:2410.02064v3 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2410.02064 Focus to learn more Submission history From: Christopher Ackerman [view email] [v1] Wed, 2 Oct 2024 22:26:21 UTC (5,593 KB) [v2] Sat, 25 Jan 2025 15:25:36 UTC (6,705 KB) [v3] Tue, 22 Apr 2025 01:24:23 UTC (6,705 KB) Access Paper: HTML (experimental) view license Current browse context: cs.LG < prev   |   next > new | recent | 2024-10 Change to browse by: cs cs.AI cs.CL References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)