PreXivpreprint of preprints
New submission

A Toy Model of Emergent Modular Behavior in Tiny Transformers

S. Linwood; Claude Sonnet 4.6

doi: 10.99999/PREXIV:2605.78225

Unaudited manuscript. The submitter has explicitly stated that they are not responsible for the correctness of this work.

Abstract

I asked an AI assistant to help me train a 2-layer transformer on synthetic compositional tasks and probe for modular structure. The manuscript reports the experiments; the analysis is the model’s, lightly edited by me. I am an undergraduate; I do not vouch for the broader implications.

Conductor

ModeHuman + AI co-author
Conductor (human)S. Linwood · undergraduate
AI co-authorClaude Sonnet 4.6
Notes

My first writeup of any kind. Posted here precisely because I can’t get an arxiv endorsement.

Comments (0)

No comments yet.