Plasticity & Networks · Lesson 2 of 3 · ~6 min

The network is a matrix

Predict

Three inputs feed three outputs through a weight matrix $W$, and the layer computes $y = \text{step}(Wx)$. The Swap preset exchanges rows 1 and 3. What does the input pattern $(1,0,0)$ come out as?

Hold that thought. Load Swap, set the toggles to $(1,0,0)$, and read the output nodes.

Try

Try itWith Identity loaded, click the three x toggles into any pattern — the outputs copy it. Load Swap and set $(1,0,0)$. Then load Blend: switch on one input, watch the written-out sums, then switch on a second.

Notice

Under Identity, $y$ mirrors $x$ — each output reads one input with weight $1.0$. Under Swap, $(1,0,0)$ comes out as $(0,0,1)$: row 3 of $W$ carries a $1.0$ in column 1, so input 1 reports to output 3. Under Blend, one active input produces silence — the written-out sums show the two outputs it reaches collecting only $0.5$ each, below the threshold $\theta = 1$, and the third collecting $0$ — while two inputs that share an output push its sum to $1.0$ and it fires.

Explain

The whole circuit is one equation. Entry $w_{ij}$ is the synapse from input $j$ to output $i$; row $i$ is output $i$'s recipe for weighing the input vector — a dot product, exactly the summation from the last lesson written three at a time:

y_i = \text{step}\Big(\sum_j w_{ij}\,x_j\Big), \qquad \text{step}(s) = \begin{cases}1 & s \ge 1\\[2pt] 0 & s < 1\end{cases}

PREREQUISITEA matrix acting on a vector — the same machinery that moved space in the Matrices chapter now routes signals between neurons.

So a wiring diagram and a weight matrix are the same object: rewiring the circuit means editing entries. The step is what makes this more than linear algebra — it turns graded sums into all-or-none decisions, which is how Blend becomes a coincidence detector that no purely linear map could be.

About this model

What it represents: a single feedforward layer of three threshold units — synapses frozen into a fixed matrix $W$, activity as a 0/1 vector, one clean multiply-then-threshold per input pattern.

What it leaves out: real synapses are not an instantaneous matrix multiply. Membranes integrate inputs over tens of milliseconds and keep history, so a real layer has dynamics where this one has a single frozen snapshot. Biology also constrains the signs: by Dale's law a neuron's outgoing synapses are all excitatory or all inhibitory — one sign per column of $W$ — while a mathematical matrix mixes signs freely. Also missing: learning (here $W$ never changes), noise, and spikes themselves.

Source: Dayan and Abbott, Theoretical Neuroscience (MIT Press), Ch. 7 — network models built from firing rates, including the dynamics and sign constraints this snapshot omits.

Apply

Checkpoint — prove it to yourself

In $y = \text{step}(Wx)$, which entry wires input 3 to output 1?

Blend is loaded ($0.5$ per wire, two wires per output) and you switch on only $x_1$. What is $y$?

Transfer: Swap never loses a pattern — every input pattern comes out rearranged but intact. Under Blend, $(1,0,0)$, $(0,1,0)$, $(0,0,1)$ and $(0,0,0)$ all map to $(0,0,0)$. What did the threshold do?

Key idea

A layer of neurons is a matrix wearing a threshold: $w_{ij}$ is the synapse from input $j$ to output $i$, the matrix routes, and the step decides. Identity copies, a permutation rearranges, and fractional weights under a threshold detect coincidences — computation you choose by choosing entries.

← All lessons