The focal-loss curve says what one example costs; this says what the whole
image costs. The population on the left is an illustration with its
distribution stated, not a measurement: a hundred thousand anchors, almost
all of them background the model already gets right at about
pt = 0.9. Before you move anything: at γ = 0 and the
published α = 0.25 the twelve positives pay 0.046% of the total loss. Push γ
up and watch which block loses
the budget — and notice that it is the easy background that gives
way, not the background as a whole. RetinaNet divides this total by the
number of positive anchors; that scales the bar and changes no share.