Request: Training Script for Sparse Models to Observe Circuit Formation #2
Description
Hello, and thank you for your excellent work. I’m interested in studying how circuits emerge during the training process of sparse models, and would like to know whether you provide (or plan to provide) any training scripts, configurations, or reproducible pipelines for training the sparse model from scratch. Access to such resources would greatly help in understanding how sparsity patterns form, how circuits evolve layer by layer, and how the final sparse structure relates to the dense model’s behavior. If these resources are not yet available, any guidance on reproducing your training setup would be greatly appreciated. Thank you!