Object detection
Boxes instead of labels: how a model finds several objects at once, what IoU and mAP mean, and why NMS decides the count.
The idea
Classification says "there is a pallet in this image". Detection says "there is one here, here and here" and draws rectangles around them. That lets you count, locate and check for absence.
What it is good for
- Counting stock in a warehouse.
- Finding missing parts in an assembly.
- Checking protective equipment on a building site.
- Comparing shelf stock against a planogram.
Post-processing
A detector first emits thousands of candidate boxes. Two dials turn that into a count:
| Dial | If too low | If too high |
|---|---|---|
| Confidence threshold | Many false alarms | Objects missed |
| NMS IoU threshold | One object counted several times | Adjacent objects merged |
import numpy as np
def iou(a, b):
"""a, b as (x1, y1, x2, y2)."""
x1, y1 = max(a[0], b[0]), max(a[1], b[1])
x2, y2 = min(a[2], b[2]), min(a[3], b[3])
inter = max(0, x2 - x1) * max(0, y2 - y1)
area_a = (a[2] - a[0]) * (a[3] - a[1])
area_b = (b[2] - b[0]) * (b[3] - b[1])
return inter / (area_a + area_b - inter)
def nms(boxes, scores, thresh=0.5):
order = np.argsort(scores)[::-1]
keep = []
while len(order):
i = order[0]; keep.append(i)
# Drop every box overlapping the best one too strongly.
order = np.array([j for j in order[1:]
if iou(boxes[i], boxes[j]) < thresh])
return keep- Both thresholds belong in the operating documentation, because they change the result.
- With densely packed objects, evaluate soft NMS rather than hard NMS.
- Set the confidence threshold from the cost calculation, not from a default of 0.25.
The formulas
Worked through: A = (0,0,100,100), B = (50,50,150,150). The intersection is
50 × 50 = 2,500, each area 10,000, the union
10,000 + 10,000 − 2,500 = 17,500. So IoU = 2,500/17,500 = 0.143, well below
the usual 0.5 threshold despite obvious visible overlap. That shows how strict
the measure is.
The architecture families
| Family | Idea | Strength |
|---|---|---|
| Two-stage (Faster R-CNN) | Proposals first, then classification | Highest accuracy, slow |
| One-stage (YOLO, SSD) | Boxes straight from the grid | Real-time capable |
| Anchor-free (FCOS, CenterNet) | Centre point instead of anchors | Fewer hyperparameters |
| Transformer-based (DETR) | Set of queries, no NMS | Elegant, data-hungry |
Cost in practice
A mid-sized YOLO at 640 by 640 needs roughly 50 to 80 GFLOPs per image. On a card
delivering an effective 30 TFLOP/s in float16 that is theoretically over 300
images per second, practically 60 to 150 depending on pre- and post-processing.
For a 25 fps camera, one card therefore serves several streams at once. See
Speeding up inference.
Related courses and sources
Hugging Face computer vision course
From image preprocessing through convolutional networks to vision transformers, with runnable examples for detection and segmentation.
For development with image data; assumes Python and delivers runnable examples in exchange.
You Only Look Once
Detection in a single pass instead of proposals and checks. The reason real-time object detection became possible.
For real-time object detection; explains why it became possible at all.