Keypoints and descriptors
ORB, SIFT and matching: how two images of the same object are aligned even when rotated, scaled or differently lit.
The idea
To align two photos of the same shelf you look for distinctive places: corners, labels, box edges. A plain white wall does not help, because every point on it looks the same.
That is what a feature detector does: it finds unmistakable points and describes their surroundings as a sequence of numbers.
What it is good for
- Stitching panoramas.
- Finding a label again in a shelf photo.
- Straightening an angled document photo.
- Determining motion between two video frames.
Aligning two images
import cv2, numpy as np
a = cv2.imread("template.jpg", cv2.IMREAD_GRAYSCALE)
b = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
orb = cv2.ORB_create(nfeatures=3000)
ka, da = orb.detectAndCompute(a, None)
kb, db = orb.detectAndCompute(b, None)
# ORB gives binary descriptors -> Hamming distance, not Euclidean.
matcher = cv2.BFMatcher(cv2.NORM_HAMMING)
pairs = matcher.knnMatch(da, db, k=2)
# Lowe ratio: a match counts only if it is clearly better than the
# second best. That removes most wrong matches.
good = [m for m, n in pairs if m.distance < 0.75 * n.distance]
print(f"kept {len(good)} of {len(pairs)} matches")
if len(good) >= 8:
pa = np.float32([ka[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
pb = np.float32([kb[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)
H, mask = cv2.findHomography(pb, pa, cv2.RANSAC, 5.0)
print("inliers:", int(mask.sum()), "of", len(good))
aligned = cv2.warpPerspective(b, H, (a.shape[1], a.shape[0]))The detectors compared
| Detector | Descriptor | Speed | Scale robust |
|---|---|---|---|
| ORB | binary, 256 bits | very high | medium |
| SIFT | 128 floats | low | high |
| AKAZE | binary | high | high |
| Shi-Tomasi | none, points only | very high | none |
A corner as a mathematical criterion
Why the ratio test works
For a true match the best distance is clearly smaller than the second best, because only one correct correspondence exists. For a false match, best and second best are both accidental and therefore similarly far away. A ratio below 0.75 empirically filters around 90 percent of wrong matches at the cost of about 5 percent of the correct ones.
RANSAC, quantified
Worked through: at w = 0.5, s = 4 and p = 0.99 you need
N = log(0.01)/log(1 − 0.0625) = 71 attempts. At w = 0.2 that rises to
N = 2876. The cost therefore grows dramatically with the share of wrong matches:
which is why good prefiltering with the Lowe ratio matters more than a high
iteration count.
Application: shelf comparison
A practical flow for comparing a shelf photo against a planogram:
- Scale template and photo to a similar order of magnitude, otherwise too few points match.
- ORB with 3,000 to 5,000 points; more rarely helps.
- Lowe ratio at 0.7 to 0.8, stricter with repeating patterns.
- Accept the homography only with at least 20 inliers and a plausible determinant.
- Rectify first, then run the actual inspection.
Related courses and sources
OpenCV tutorials
The official guides to filters, edges, features and calibration, each with runnable Python code.
For anyone writing image processing themselves; every example runs.