Seon Joo Kim

Seon Joo Kim

■ Radiometric Calibration

    I have been working on radiometric calibration which is a problem of how brightness (radiance) in the real
    world is mapped in images. Factors involved in this mapping are radiometric response function, vignetting, and
    exposures. Here are some of the contributions of my research.

    ● Robust radiometric calibration and vignetting correction from correspondence

       I introduced an algorithm that robustly estimates the radiometric response function, exposures, and the vignetting
       effect given multiple images taken with freely moving camera. This method advances the state of the art in
       radiometric calibration by allowing general image sets to be used, while previous methods were limited to images
       taken with a static camera or a rotating camera. The algorithm can be applied to radiometrically align images for
       seamless mosaics and 3D model textures as well as to generate high dynamic range (HDR) mosaics.



       ▪ Seon Joo Kim and Marc Pollefeys, "Robust Radiometric Calibration and Vignetting Correction",
       IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 30, No. 4, April, 2008 [pdf]

      ▪ Seon Joo Kim and Marc Pollefeys, "Radiometric Alignment of Image Sequences", Proc. IEEE Conference
       on Computer Vision and Pattern Recognition (CVPR), 2004 [pdf]







                                                              High Dynamic Range (HDR) Mosaic

● Joint Feature Tracking and Radiometric Calibration

I presented an algorithm suited for video data taken with auto-exposure where the correspondence (feature tracks)

and the radiometric response function along with the exposure values are computed simultaneously.

                     The method advances the conventional feature tracking algorithm (KLT tracker) which requires the brightness of
                     features to stay constant by unifying the problems of feature tracking and radiometric calibration into a common
                     framework.

▪ Seon Joo Kim, Jan-Michael Frahm and Marc Pollefeys, "Joint Feature Tracking and Radiometric Calibration
from Auto-Exposure Video", Proc. Int. Conf. on Computer Vision (ICCV), 2007 [pdf]

▪ Seon Joo Kim, D. Gallup, J.-M. Frahm, A. Akbarzadeh, Q. Yang, R. Yang, D. Nister, M. Pollefeys,
"Gain Adaptive Real-Time Stereo Streaming", Proc. Int. Conf. on Computer Vision Systems (ICVS), 2007 [pdf]

- Video1 (Feature Tracking Comparison)

- Video2 (Tracking & Radiometric Calibration)

- Video3 (Tracking & Exposure Computation)

( The system is suited for videos taken in a high dynamic range scene. The graph above
compares the exposure estimates to the ground truth )

Depth map comparison

without/with the alignment

Input sequence

Radiometrically aligned

3D model & texture

                                            ( The joint method can be applied to build an adaptive stereo system.
                                              Our method provides robust feature tracking, simpler stereo cost function,
                                              and texture alignment)

● Radiometric calibration with illumination change for outdoor scene analysis

I introduced a new algorithm to compute the radiometric response function and the exposure of images given a

sequence of images of a static outdoor scene taken over time where the illumination is changing. This is the first

radiometric calibration method to work with regular images with illumination changing where the lighting cannot

be controlled.

                   ▪ Seon Joo Kim, Jan-Michael Frahm, and Marc Pollefeys, "Radiometric Calibration with Illumination Change
                     for Outdoor Scene Analysis, Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
                     2008 [pdf]

(Before and after the calibration. Notice that the brightness change in the original sequence is affected by the camera
exposure. It does not show how the scene is really changing. The calibrated sequence shows the actual brightness change

in the scene)

Video

■ Tracking

     I am also interested in tracking. I worked on projects developing tracking system for automatic surveillance as an
     intern at GE Research (2005) and at Cortex-US(2004).

     Here are some videos of the work done at GE. The goal is to detect salient motion from video (rotating PTZ
     camera) that contains other motions such as waving trees and flags as well as the camera motion.

     - Video1 (Blue regions indicate salient motion)

     - Video2 (Blue regions indicate salient motion / green regions indicate non-salient motion)

     - Video3

     I have also worked on tracking as class projects including face tracking and 3D active tracking.

     ▪ 3D active tracking demos

        video1, video2, video3

     ▪ Face tracking demos

        video1, video2, video3

■ Biometrics (Fingerprint recognition)

      I worked on fingerprint recognition for my Masters degree in Yonsei University, Seoul, Korea. I worked
      on every phase of fingerprint recognition since I was the first one to start the project. Visit the following
      site for more details on biometrics.

     Biometrics Engineering Research Center at Yonsei University