When Color Constancy Goes Wrong:
Correcting Improperly White-Balanced Images

This paper focuses on correcting a camera image that has been improperly white balanced. This situation occurs when a camera's auto white balance fails or when the wrong manual white balance setting is used. Even after decades of computational color constancy research, there are no effective solutions to this problem. The challenge lies not in identifying what the correct white balance should have been, but in the fact that the in-camera white balance procedure is followed by several camera-specific nonlinear color manipulations that make it challenging to correct the image's colors in post-processing. This paper introduces the first method to explicitly address this problem. Our method is enabled by a dataset we generated with over 65,000 pairs of incorrectly white-balanced images and their corresponding correctly white-balanced image. Using this dataset, we introduce a k-nearest neighbor strategy that is able to compute a nonlinear color mapping function to correct the image's colors. We show our method is highly effective and generalizes well to camera models not in the training set.

When taking a photograph, we expect our images to be correctly white-balanced. Computer vision algorithms implicitly assume a correct white balance (WB) by expecting their input image colors to be correct. What happens when the WB is not correct? In such cases, the images have the familiar bluish/reddish color casts that not only are undesirable from a photography standpoint but also can adversely affect the performance of vision algorithms. Correcting improperly white-balanced images is poorly understood. Sources such as Matlab purport misleading solutions that suggest the problem is a matter of identifying what the correct WB should have been and then applying this as a post correction. However, this solution is not effective and does not consider white-balance within the full context of the in-camera processing pipeline.

The above figure shows a camera image rendered through a camera pipeline to an sRGB output with the incorrect WB in (A). (B) and (C) show traditional white balance correction applied to the image using different reference white points manually selected from the image (note: this would represent the best solution for an automatic illumination estimation algorithm). (D) shows the result of auto-color correction from Adobe Photoshop. (E) shows our result and (F) shows the ground truth camera image with the correct white balance applied.

Given an incorrectly WB input image, our goal is to compute a mapping that can transform the input colors to appear as if the WB was correctly applied. We search the training set to find images with similar color distributions. This image search is performed using compact features derived from input and training image histograms. Finally, we obtain a color correction matrix for our input image by blending the associated color correction matrices of the similar training image color distributions.

Quantitative and qualitative experiments, including a user study, were performed that demonstrated the effectiveness of this approach. More information can be found in the Paper and in the Results tab.