Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC Lights
Artificial lights, which are powered by alternating current (AC), are ubiquitous nowadays. The intensity of these lights fluctuates dynamically depending on the AC power. In contrast to previous color constancy methods that exploited the spatial color information, we propose a novel deep learning-ba...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9695461/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841533422213791744 |
|---|---|
| author | Jun-Sang Yoo Kang-Kyu Lee Chan-Ho Lee Ji-Min Seo Jong-Ok Kim |
| author_facet | Jun-Sang Yoo Kang-Kyu Lee Chan-Ho Lee Ji-Min Seo Jong-Ok Kim |
| author_sort | Jun-Sang Yoo |
| collection | DOAJ |
| description | Artificial lights, which are powered by alternating current (AC), are ubiquitous nowadays. The intensity of these lights fluctuates dynamically depending on the AC power. In contrast to previous color constancy methods that exploited the spatial color information, we propose a novel deep learning-based color constancy method that exploits the temporal variations exhibited by AC-powered lights. Using a high-speed camera, we capture the intensity variations of AC lights. Then, we use these variations as an important cue for illuminant learning. We propose a network composed of spatial and temporal branches to train the model with both spatial and temporal features. The spatial branch learns the conventional spatial features from a single image, whereas the temporal branch learns the temporal features of AC-induced light intensity variations in a high-speed video. The proposed method calculates the temporal correlation between the high-speed frames to extract the effective temporal features. The calculations are done at a low computational cost and the output is fed into the temporal branch to help the model concentrate on illuminant-attentive regions. By learning both spatial and temporal features, the proposed method performs remarkably under a complex illuminant environment in a real world scenario in which color constancy is difficult to investigate. The experimental results demonstrate that the proposed method produces lower angular error than the previous state-of-the-art by 30%, and works exceptionally well under various illuminants, including complex ambient light environments. |
| format | Article |
| id | doaj-art-8be90e1580484a13a2bc699bf066fcdd |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-8be90e1580484a13a2bc699bf066fcdd2025-01-16T00:01:09ZengIEEEIEEE Access2169-35362022-01-0110155281553810.1109/ACCESS.2022.31472529695461Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC LightsJun-Sang Yoo0https://orcid.org/0000-0001-9552-2546Kang-Kyu Lee1https://orcid.org/0000-0002-2773-7278Chan-Ho Lee2https://orcid.org/0000-0002-3450-4333Ji-Min Seo3Jong-Ok Kim4https://orcid.org/0000-0001-7022-2408Computer Vision Laboratory, Samsung Advanced Institute of Technology, Suwon-si, South KoreaSchool of Electrical Engineering, Korea University, Seoul, South KoreaSchool of Electrical Engineering, Korea University, Seoul, South KoreaDepartment of Electrical and Computer Engineering, Seoul National University, Seoul, South KoreaSchool of Electrical Engineering, Korea University, Seoul, South KoreaArtificial lights, which are powered by alternating current (AC), are ubiquitous nowadays. The intensity of these lights fluctuates dynamically depending on the AC power. In contrast to previous color constancy methods that exploited the spatial color information, we propose a novel deep learning-based color constancy method that exploits the temporal variations exhibited by AC-powered lights. Using a high-speed camera, we capture the intensity variations of AC lights. Then, we use these variations as an important cue for illuminant learning. We propose a network composed of spatial and temporal branches to train the model with both spatial and temporal features. The spatial branch learns the conventional spatial features from a single image, whereas the temporal branch learns the temporal features of AC-induced light intensity variations in a high-speed video. The proposed method calculates the temporal correlation between the high-speed frames to extract the effective temporal features. The calculations are done at a low computational cost and the output is fed into the temporal branch to help the model concentrate on illuminant-attentive regions. By learning both spatial and temporal features, the proposed method performs remarkably under a complex illuminant environment in a real world scenario in which color constancy is difficult to investigate. The experimental results demonstrate that the proposed method produces lower angular error than the previous state-of-the-art by 30%, and works exceptionally well under various illuminants, including complex ambient light environments.https://ieeexplore.ieee.org/document/9695461/Temporal color constancytemporal correlationAC lighthigh-speed video |
| spellingShingle | Jun-Sang Yoo Kang-Kyu Lee Chan-Ho Lee Ji-Min Seo Jong-Ok Kim Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC Lights IEEE Access Temporal color constancy temporal correlation AC light high-speed video |
| title | Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC Lights |
| title_full | Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC Lights |
| title_fullStr | Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC Lights |
| title_full_unstemmed | Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC Lights |
| title_short | Deep Spatio-Temporal Illuminant Estimation Under Time-Varying AC Lights |
| title_sort | deep spatio temporal illuminant estimation under time varying ac lights |
| topic | Temporal color constancy temporal correlation AC light high-speed video |
| url | https://ieeexplore.ieee.org/document/9695461/ |
| work_keys_str_mv | AT junsangyoo deepspatiotemporalilluminantestimationundertimevaryingaclights AT kangkyulee deepspatiotemporalilluminantestimationundertimevaryingaclights AT chanholee deepspatiotemporalilluminantestimationundertimevaryingaclights AT jiminseo deepspatiotemporalilluminantestimationundertimevaryingaclights AT jongokkim deepspatiotemporalilluminantestimationundertimevaryingaclights |