Memories as attractors
A 25-pixel network stores two patterns, a T and an X, in one weight matrix. You corrupt $5$ of the T's $25$ pixels, then let the network run $x \leftarrow \text{sign}(Wx)$. Where does the state end up?
Five flipped pixels cost the overlap $0.40$, since each flip swings its term of the dot product from $+1$ to $-1$ — $2/25$ of overlap apiece. One sweep of $x \leftarrow \text{sign}(Wx)$ later, the state reads overlap $1.00$: the exact T, and the next sweep changes nothing — a fixed point. Not luck: every one of the $53{,}130$ possible 5-pixel corruptions of this T flows back to it in a single sweep. Corrupt past half the pattern, though, and the state falls elsewhere — often into the T's photographic negative, which the weights store for free.
Storage is Hebbian, one outer product per pattern, diagonal zeroed:
Multiplying by $W$ makes the memories vote: $Wx = T\,(T \cdot x) + X\,(X \cdot x)$ up to the removed diagonal, so each stored pattern pulls with strength equal to its overlap with the current state. A corrupted T overlaps the T at $0.60$ and the X near $0$, so the T's vote sets every pixel's sign. A memory here is not a file at an address — it is an attractor, a fixed point the dynamics fall into from anywhere in its basin.
PREREQUISITERepeatedly applying a matrix until a dominant direction wins — recall here is the same mathematics as power iteration, with a sign function snapping the state to the nearest corner.About this model
What it represents: a Hopfield network of 25 binary units with Hebbian outer-product weights — the standard minimal model of associative, content-addressable memory, run here with synchronous sweeps.
What it leaves out: Hopfield's own analysis updates one unit at a time (asynchronously), which guarantees an energy function that only decreases; synchronous sweeps like these can in principle fall into two-step cycles, which is one reason the instrument caps runs at 10 steps. Capacity is severe: outer-product storage holds only about $0.14N$ patterns — three or four for $N = 25$ — before memories interfere, and each pattern's negative is stored for free whether wanted or not. The biological mapping is loose throughout: real cortical circuits are not symmetric, not binary, and not updated in lockstep.
Source: Gerstner, Kistler, Naud and Paninski, Neuronal Dynamics, Ch. 17.2 (EPFL, free online) — the Hopfield model, its energy function, and its capacity limits.
You corrupt $5$ of the T's $25$ pixels. Before running the dynamics, what does the overlap-T stat read?
Why does the corrupted state fall back to the T rather than to the X?
Transfer: power iteration applies $A$ over and over and converges to the dominant eigenvector. Recall here applies $W$ (plus sign) over and over. What plays the role of the dominant direction?
A memory in this network is an attractor: a fixed point of $x \leftarrow \text{sign}(Wx)$ with a basin around it. Hebbian storage writes patterns into $W$ as outer products; recall is repeated application of $W$ — power iteration wearing a threshold — so any fragment close enough to a memory is a key that summons the whole of it.