Video and motion

The idea#

A video is a stack of images. What is new is the question of connection: is the person on the left in frame two the same as in frame one? Until that is answered, nothing can be counted.

What it is good for#

• Counting people at an entrance without identifying anyone.
• Measuring queue waiting times.
• Checking movement on a conveyor.
• Alerting on standstill or unusual motion.

Detecting motion without a model#

import cv2

cap = cv2.VideoCapture("hall.mp4")
# MOG2 learns the background continuously and flags what stands out.
bg = cv2.createBackgroundSubtractorMOG2(history=500, varThreshold=40,
                                        detectShadows=True)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))

while True:
    ok, frame = cap.read()
    if not ok:
        break
    mask = bg.apply(frame)
    mask[mask == 127] = 0                        # discard shadows
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)

    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL,
                                   cv2.CHAIN_APPROX_SIMPLE)
    moving = [c for c in contours if cv2.contourArea(c) > 1500]
    if moving:
        pass                                     # raise an event
cap.release()

This needs no GPU and no training data. For "is something moving here?" it is the right approach.

Alternating detection and tracking#

Optical flow#

One equation, two unknowns. This is the aperture problem: from a single point only the motion component perpendicular to the edge can be recovered. Lucas-Kanade resolves it by assuming the equation is constant over a small neighbourhood and solving the overdetermined system by least squares.

Evaluating tracking#

The standard metric is MOTA, and it is stricter than it looks:

One identity switch per person per minute sounds harmless but doubles the counted headcount at a one-minute dwell time. For counting applications IDF1 is therefore the more informative metric, because it judges identity consistency across the whole track.

Data protection for counting applications#

A plain count with no image storage can be lawful if processing happens in the device and only numbers leave it. Three conditions decide:

As soon as recognition across days is possible, this is biometric processing and the assessment changes entirely. See Face recognition.