As-Projective-As-Possible Bias Correction for Illumination Estimation Algorithms

Abstract:

Illumination estimation is the key routine in a camera's onboard auto-white-balance (AWB) function. Illumination estimation algorithms estimate the color of the scene's illumination from an image in the form of an R,G,B vector in the sensor's raw-RGB color space. While learning-based methods have demonstrated impressive performance for illumination estimation, cameras still rely on simple statistical-based algorithms that are less accurate but capable of executing quickly on the camera's hardware. An effective strategy to improve the accuracy of these fast statistical-based algorithms is to apply a post-estimate bias correction function to transform the estimated R,G,B vector such that it lies closer to the correct solution. Recent work by Finlayson, Interface Focus, 2018 showed that a bias correction function can be formulated as a projective transform because the magnitude of the R,G,B illumination vector does not matter to the AWB procedure. This paper builds on this finding and shows that further improvements can be obtained by using an as-projective-as-possible (APAP) projective transform that locally adapts the projective transform to the input R,G,B vector. We demonstrate the effectiveness of the proposed APAP bias correction on several well-known statistical illumination estimation methods. We also describe a fast lookup method that allows the APAP transform to be performed with only a few lookup operations.

Results:

Projective bias correction of the estimated illuminant of the gray world algorithm (GW). (A) A raw Canon EOS-1Ds image from the NUS dataset. (B) The corrected image using the GW algorithm. (C) The corrected image using the global projective transformation (GP). (D) The corrected image using the as-projective-as-possible with the lookup table (APAP-LUT). (E) The ground truth image.

We applied our proposed projective bias correction transformations to four different statistical-based methods: (i) gray wrold, (ii) Gray edges, (iii) shades of gray, and (iv) distribution PCA.

* For Shades of gray, the Minkowski norm (p) was set to 4.

* The first and second differentiations were used for gray edges with p=6 and σ=2.

* For the distribution PCA method, the selected percentage rate was 3.5%.

* All other methods source code collected from colorconstancy.com and NUS dataset webpage

For each illumination estimation method, we evaluate our proposed projective bias correction transformations, which are: (i) global projective transformation (GP), (ii) as-projective-as-possible bias correction (APAP), and (iii) APAP with the lookup table (APAP-LUT).

* We used σ_w=3.0 and γ=0.0625 for our APAP bias correction. The LUT consisted of 16×16 bins.

We evaluated our method on three different datasets: (i) the NUS dataset, (ii) the reprocessed version of Gehler's dataset (Gehler-Shi dataset), and (iii) the INTEL-TUT dataset.

As there are different ground-truth illuminants available for the Gehler-Shi dataset, we recalculated the ground-truth illuminants from the color rendition chart provided in each image of the dataset.

We used three-cross-fold validation for each camera in each dataset.

We reported the mean, median, best 25%, and worst 25% performance for each method before and after our correction, where the best 25% and worst 25% are the mean of the smallest 25% error values and the mean of the highest 25% error values, respectively.

The results of gray world, gray edges, shades of gray, and distribution PCA methods may differ slightly from other reported results in the literature, because we used fixed parameters in all experiments instead of fine-tuning the parameters per camera.

Download our results:

Note:

Download our results:

Download our results:

Paper

Citation:

Source Code:

Acknowledgements:

This study was funded in part by the Canada First Research Excellence Fund for the Vision: Science to Applications (VISTA) programme and an NSERC Discovery Grant.