Time-frequency and images · Definition

2D convolution on images

signal processingtime-frequency and images

Image filters such as smoothing and edge detection are implemented with 2D convolution. Understanding its mathematical form lets you design them and predict their effect.

Definition

2D convolution is the operation that combines an image \(f(x,y)\) with a kernel \(h(x,y)\) by sliding the kernel and summing products at each position.

\[(f*h)[m,n]=\sum_{k}\sum_{l} f[k,l]\,h[m-k,n-l]\]

2D convolution is commutative and associative, and it is also linear and shift-invariant. If the kernel is separable, \(h[m,n]=h_1[m]h_2[n]\), the convolution can be done in two 1D passes, reducing the number of multiplications. In the Fourier domain, convolution becomes a pointwise product of the transforms.

Consider a \(3\times3\) image with a single bright pixel at the center, \(f[1,1]=9\), and the \(3\times3\) averaging kernel \(h\) with all entries equal to \(1/9\). When convolving, the bright pixel spreads over the nine pixels of its neighborhood, itself included, each receiving 1, which illustrates smoothing.

A typical mistake is forgetting to flip the kernel in the definition, using correlation instead of convolution. For symmetric kernels the result coincides, but with asymmetric kernels such as derivative filters the sign of the result changes.

Think about it

Why do convolution and correlation give opposite signs at an edge with a derivative kernel?

Because convolution flips the kernel, so its \(-1\) weight meets the brighter pixel, while correlation applies it to the darker one.

Exercises 10

  1. Which operation combines an image with a kernel by sliding it and summing products?
  2. Which property makes 2D convolution commutative?
  3. In which domain does 2D convolution become a pointwise product?
  4. Which type of kernel can be implemented in two 1D passes?
  5. What effect does the 3×3 averaging kernel have on an image?
  6. If \(f\) is a \(5\times5\) image with a 1 at the center and zeros elsewhere, and \(h\) is a \(3\times3\) kernel of ones, what is the total sum of the output, ignoring padding borders?
  7. A derivative kernel \(h\) in \(x\) has values \(-1\) on the left and \(+1\) on the right. Using the convolution defined in this concept (kernel flipped), what sign does the output have at an edge where intensity rises from left to right?
  8. What is the result of convolving the single-pixel image \(\delta[m,n]\) with any kernel \(h\)?
  9. An averaging \(3\times3\) kernel is applied twice in a row. What is the equivalent smoothing effect?
  10. A \(4\times4\) image \(f\) is convolved with the separable kernel \(h=h_1 h_2^T\), where \(h_1=[1,2,1]\) and \(h_2=[1,2,1]\). What is the output value at the central position if the window of \(f\) is all ones, without normalization?

References

  • Alan V. Oppenheim, Alan S. Willsky, Signals and Systems, 2nd edition, Pearson, 1996.
  • Alan V. Oppenheim, Ronald W. Schafer, Discrete-Time Signal Processing, 3rd edition, Pearson, 2010.