An image as an array of numbers
Pixels, channels, colour spaces and histograms: what is actually in memory when an image is loaded, and why BGR is the most common source of bugs.
The idea
A greyscale image is a table: each cell holds a number between 0 (black) and 255 (white). A colour image is three such tables stacked, one each for red, green and blue. That is all it is.
What follows
Everything done to images is arithmetic on those numbers. Brightening is addition. Raising contrast is multiplication. Blurring is replacing each value with the average of its neighbours.
The colour spaces
| Space | Channels | Used for |
|---|---|---|
| RGB / BGR | Red, green, blue | Display, standard format |
| Greyscale | Brightness | Edges, shapes, speed |
| HSV | Hue, saturation, value | Colour filters, robust to brightness |
| LAB | Lightness, two colour axes | Colour distances matching perception |
| YCrCb | Luma, two differences | Compression, skin tones |
import cv2, numpy as np
img = cv2.imread("shelf.jpg") # CAREFUL: returns BGR, not RGB
print(img.shape, img.dtype) # (3000, 4000, 3) uint8
print(img.nbytes / 1e6, "MB") # 36.0
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
grey = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# Find all red areas. In HSV red is the special case:
# the hue wraps past 0, so two ranges are needed.
m1 = cv2.inRange(hsv, (0, 120, 70), (10, 255, 255))
m2 = cv2.inRange(hsv, (170, 120, 70), (180, 255, 255))
mask = cv2.bitwise_or(m1, m2)
print("red area fraction:", round(float(mask.mean()) / 255, 4))- After
imread, check forNone. A wrong path raises no exception. - Settle the channel order before any display or model call.
- Normalise to
float32and the 0 to 1 range for models, using the pre-training statistics. - Watch EXIF rotation: many phone photos are physically rotated in memory.
Greyscale conversion
The naive mean (R+G+B)/3 is therefore wrong: a saturated blue and a saturated
green with the same arithmetic mean appear equally bright in the result although
the eye perceives them very differently.
Histogram and exposure
Three diagnoses from one glance:
- Spike at 255, overexposed, information lost and unrecoverable.
- Everything between 60 and 120, low contrast, equalisation helps.
- Two separated humps: a natural threshold exists, see Otsu in Preprocessing with OpenCV.
Memory, worked through
A batch of 32 images at 1024 by 1024 with three channels takes
32 × 1024 × 1024 × 3 × 1 = 100.7 MB in uint8 and 402.7 MB in float32,
and that is only the input, before any network activations. This is exactly where
the memory pressure that forces a choice of image size comes from.