d_k
d_k = 64
Left: one draw. A query and four keys, each 512 standard-normal numbers, fixed until you press new draw; only the first d_k of them enter the dot product, so the slider changes how much of the same vectors the score sees. Right: the same measurement averaged over 160 independent draws, which is where the claim actually lives — a single draw is noisy, and pressing new draw a few times at d_k = 64 shows how much it varies. Predict what the red curve does as d_k grows, and whether the gold one follows it. The callout above proves Var(q·k) = d_k; this is what that variance does to the softmax, and what dividing it back out undoes.