Image acquisition

Image Acquisition

Demosaic of Color Filter Array Image

A camera raw image file contains minimal processed data. It isn’t determined ‘white balancing’, ‘parameters’, and ‘color space’. It only has information of the intensity of the sensed-light from the image sensor.

So, if we want to obtain a colorful image, we should do a demosaicing by ‘Bilinear Filtering/Convolution(one of the example)’.

• What is a demosaicing? Demosaicing can be performed in different ways. Simple methods interpolate the color value of the pixels of the same color in the neighborhood.

Image Acquisition

• Method 1 : Compression first

  • On the ISP(Image Signal Processor), preprocess, demosaicing, ISP, compress are done : Digital Camera

• Method 2 : Demosaic first

  • On the ISP, only image compressing is performed, And on the powerful computer, decompress, preprocessing, demosaicing, and ISP are performed.

Example

Source

1. Original Scene (Original RGB Image)

2. Output of a 120×80-pixel sensor with a Bayer filter

3. Output color-coded with Bayer filter colors

4. Reconstructed image after interpolating missing color information

5. Full RGB version at 120×80-pixels for comparison (e.g. as a film scan, Foveon or pixel shift image might appear)

Image Filtering

It can be understood that the filter of the 1-D kernel is sequentially performed row and column. It also could be thought as 2-D kernel.

• Tip

  In the case of convolution of a filter, there is a lot of sampling in the case of an audio signal, and it is difficult to recognize even if there is distortion at the end. However, in the case of an image, the end becomes black after filtered. Therefore, it is essential to consider padding in images.

Frequency concepts on an image

The degree of change of the image grid can be understood as frequency. It could be interpreted as how many grid changes in the image corresponding to one degree. The analysis related to this can be divided into analog and digital domains as follows.

Basic of Signals

Continuous-time sinusoids

• $x(t) = acos(\Omega t+ \theta)$
  • Range of frequency : $0 \leq \Omega < \infty$
• Complex sinusoid $X(t) = Ae^{j\Omega t}$
  • Range of frequency : $-\infty < \Omega < \infty$
  • $Re(X(t)) = x(t)$

Discrete-time sinusoids

• $x(n) = acos(\omega n+ \theta)$
  • Range of frequency : $0 \leq \omega < \pi$
• Complex sinusoid $X(n) = Ae^{j\omega n}$
  • Range of frequency : $-\pi < \omega < \pi$
  • $Re(X(n)) = x(n)$
• Relation between \Omega and \omega
  • Sampling period : T, $\omega = \Omega T$
  • $\Omega = f_{s} \Leftrightarrow \omega = 2\pi$
  • Interprtable range : [0, $f_{s}/2$] (Considering Nyquist frequency)

Frequency Response of Filters

1-D filter

• Moving-average : In the M-tap moving average filter, the cutoff frequency of the LPF(moving average filter) decreases as M increases.

2-D filter (Source)

• Box-filter

  Hd = zeros(16,16);
  Hd(5:12,5:12) = 1;
  Hd(7:10,7:10) = 0;
    
  w = [0:2:16 16:-2:0]/16;
  h = fwind1(Hd,w);
    
  colormap(parula(64))
  freqz2(h,[32 32]);
  axis ([-1 1 -1 1 0 1])
  

  

• Bilinear Filter

Image enhancement

Filter-based approach

1. {Low pass filter of the signal(image)} = $S$

2. {Original signal} -{Low pass filtered of the signal(image)} = $S-S_{LPF} = S_{HPF}$

3. $S +\lambda* S_{HPF}$ –> Enhanced image

   (These days, enhancement or super-resolution techniques through deep learning are used.)

Interpretation of the Filter-based approach

• $S-S_{LPF} = S_{HPF}$ –> This operator serves as Edge Detector

• $S +\lambda* S_{HPF}$ –> Enhancement of the image

• Edge detector (HPF) example (Reference)

  h =[0.1667    0.6667    0.1667
      0.6667   -3.3333    0.6667
      0.1667    0.6667    0.1667];
  freqz2(h)