paper reading - A Fusion-based enhancing method for weakly illuminated images

Posted 2021-03-14 litun

目录

• A Fusion-based enhancing method for weakly illuminated images
  • approach
  • puzzle
  • reference

A Fusion-based enhancing method for weakly illuminated images

---

approach

• (Step) (1) illumination estimation

  • Derived from retinex theory, an input image could be decomposed into the illumination layer and reflectance layer, as (Eq.1) defined. A prior assumption that RGB channels have the common illumination is adopted to (Eq.1).

    [S^c(x, y) = I(x, y) odot R^c{x, y} quad cin{R, G. B} ag{1}]

    (Eq.1) where (S) is input weakly illuminated image need enhancement, (I) and (R) are illumination image and reflectance image respectively. (odot) means element-wise multiplication.

  • Due to the goal to enhance a weakly enhanced image, get the illumination layer is of vital importance. Inspired by the dark channel prior, which based on an observation that most local patches in haze-free outdoor images contain some pixels which have very low intensities in at least one color channel, initial illumination layer can be estimated by (Eq.2) defined as follow:

    [L(x, y) = max_{cin{R,G, B}}S^c(x, y) ag{2}]

    (Eq.2) is a transformed version of one dark channel prior theory, which is written as

    [1 - L = min_cfrac{1 - S^c}{alpha}quad cin{R, G, B}]

  • As the illumination is local smooth, a smoothing operator is required to adopt to refine (L). A simple but effective algorithm based on using a morphologically closing operator is adopted to smoothing processing, while the maximum filter which is used by the authors of one dark channel prior theory is abandoned for its time complexity. The morphologically closing operator smooths an image by fusing narrow breaks and filling gaps on the contours without over smoothing to produce nato effect.

    [I = frac{L · P}{255} ag{3}]

    (Eq.3) where P denotes the structuring element and (·) is the closing operation.

  • Guided filter is adopted to refine the estimated illumination to preserve the shape of contours.

    [I_i larr sum_jW_{ij}(g)I_j ag{4}]

    [W_{ij}(g) = frac{1}{|omega|^2}sum_{k:(i, j)inomega_k}(1 + frac{(g_i - mu_k)(g_j - mu_k)}{sigma_k^2 - epsilon}) ag{5}]

    (Eq.4,5) where (omega_k), (|omega|) are a window centered pixel (k) and pixel numbers of windows respectively. (mu) and (sigma^2) are the mean and variance of guided image (g) respectively. (epsilon) is a regularization parameter. (g) is the (V) layer of HAV color space of input image (S).

• (Step) (2) input derivation

  • To avoid distortion, we make original illumination estimation as the first input layer which contains the information of original structures of original estimated illumination.

    [I_1(x, y) = I(x, y) ag{6}]

  • To enhance global illumination situation, many approaches can be applied, such as gamma correction and sigmoid function. Here arc tangent transformation is adopted to this work.

    [I_2(x, y) = frac{2}{pi}arctan(lambda I(x, y)) ag{7}]

    (Eq.7) where (lambda) is a parameter that control the degree of luminance, defined as follow.

    [lambda = 10 + frac{1 - I_{mean}}{I_{mean}} ag{8}]

    As written in (Eq.8), a higher (lambda) will be obtained when smaller (I_{mean}) is derived which indicated the poor level of original estimated luminance.

  • Since dynamic range is compressed after improving global luminance, local contrast is reduced which is designed to be enhanced by using "contrast local adaptive histogram equation"(CLAHE¹) in the third input layer. CLAHE is adopted directly to original illuminance estimation, which is useful to expand local contrast between adjacent structures.

• (Step) (3) Weight definition

  • brightness weight Considering well-exposed property on pixel-level, a pixel-level weights system is designed for fusion, which guaranty a high value on well-exposed pixels and natural-looking brightness. After statistically evaluate 2000 pics on Google image, brightness weight system is defined based on a observation on the mean of meanstandard deviation and histogram of illumination of data set are 0.5.25 approximately respectively.

    [W_{B, k}(x, y) = exp{{-frac{(I_k(x, y) - 0.5)^2}{2(0.25)^2}}} ag{9}]

  • chromatic contrast weight Considering color contrast is of vital importance to image quality, contrast is evaluated by combining the estimated illumination with chromatic information.

    [W_{c, k}(x, y) = I_k(x, y)(1 + cos(alpha H(x, y) + phi)S(x, y)) ag{10}]

    (Eq.10) where (H) and (S) are hue and saturation vector in HSV color space respectively, (alpha) and (phi) are parameters to preserve the color opponency and represents the offset angle of the color wheel respectively. In this work, parameters are set to (alpha = 2) and (phi = 250degree) respectively according to².

    The impact of this weight ((alpha = 2) and (phi = 250degree)) is to highlight regions containing high contrast caused by both illumination and color.

  • final weight

    [W_k(x, y) = W_{B, k}(x, y)W_{C, k}(x, y) ag{11}]

    Normalization,

    [ar{W}_k(x, y) = frac{W_k(x, y)}{sum_kW_k(x, y)} ag{12}]

• (Step) (4) Multi-scale fusion

  • With a naive fusion method, we fusion derived input mentioned in subsection (Step) (3) by summing the multiplying results weight matrix and derived input respectively directly.

    [I_{fusion}(x, y) = sum_kar{W}_k(x, y)I_k(x, y) ag{13}]

  • However, disappointing artifacts are produced in generating results processing which is mainly caused by strong transitions of the weight maps. To overcome this, multi-scale linear and non-linear filters are adopted. While the fact that non-linear filters can not avoid these artifacts is borne out by extensive experiments, a multi-scale pyramid technique is adopted to solve this problems.

    The inputs are convolved by a Gaussian kernel to generate a low pass filtered versions.In our case, we decompose each derived input (I_k) into a Laplacian pyramid to extract image features, and each normalized weight (ar{W}_k) into a Gaussian pyramid to smooth the strong transitions. This method is effective since it blends image features instead of intensities.

    [F_l(x, y) = sum_kG_l{ar{W}_k(x, y)}L_l{I_k(x, y)} ag{14}]

    (Eq.14) where (l) is the number of pyramid level, (l = 6) is set in this work.

    [I_{final}(x, y) = sum_lU_d(F_l(x, y)) ag{15}]

    (Eq.15) where (U_d) is the up-sampling operator with factor (d = 2^{l - 1}).

    Finally,

    [S_{enhanced}^c(x, y) = R^c(x, y)I_final(x, y)quad cin{R, G, B} ag{16}]

puzzle

• morphologically closing operator

• chromatic filtering formula

reference

[1] E.D. Pisano,S.Zong,B.M.Hemminger,Contrast limited adaptive histogram image processing to improve the detection of simulated spiculations in dense mammograms, J.Digit.Imaging11(4)(1998)193–200.

[2] C.O.Ancuti,C.Ancuti,P.Bekaert,Enhancing by saliency-guided decolorization, in: Proceedings of IEEE Conferenceon Computer Vision and PatternRe-cognition, 2011.p.257–264.