One backward pass through a randomly initialised stack (width 48, batch 32),
no training: each bar is the size of the gradient reaching that layer, on a
log axis. The product argument of the prose is the whole story — watch it
with sigmoid + naive initialisation at depth 50, then repair it one toggle
at a time. He initialisation sets the weight scale, batch norm re-standardises
every layer's activations (modelled here by its scaling effect), and skip
connections give the gradient a term no weight can shrink. Which fixes
survive depth 200 alone, and which only together?