plain networks on CIFAR-10, no skip connections
Hover any epoch and read the two depths against each other. The claim worth checking is in the left panel, not the right: the 56-layer network is worse on the data it is being trained on, which is what rules out overfitting as the explanation. The step at epoch 30 is the learning-rate schedule dropping tenfold, and it helps both networks without closing the gap.