One class, five detections, three real dogs. Work down the list in score order:
a detection whose best-overlapping ground-truth box clears IoU 0.5 and is
still unclaimed takes it and counts as a true positive; anything else, including
a detection whose best match is already taken, is a false positive.
After each one, precision and recall are recomputed over everything seen so
far, and a point joins the curve. Recall can only rise; precision falls
whenever a false positive lands. Average precision is the area under the
upward-interpolated curve, so where a detection sits in the ranking matters as
much as whether it is right — compare right answers first with
right answers last, which hold exactly the same five detections and
the same three hits, and differ only in the order the detector ranked them.
The three area conventions disagree on identical predictions, which is why an
AP quoted without its protocol is not a comparable number.