My current / past research topics include :

           - Radiometric calibration (radiometry in computer vision)

           - Hyperspectral imaging of historical documents

           - Computational photography / image editing

           - Tracking

           - Biometrics
 

    I have been working on radiometric calibration which is a problem of how brightness (radiance) in the real world is mapped in images. Factors involved in this mapping are   
    radiometric response function, vignetting, and exposures. Here are some of the contributions of my research.

    ● Robust radiometric calibration and vignetting correction from correspondence

       I introduced an algorithm that robustly estimates the radiometric response function, exposures, and the vignetting effect given multiple images taken with
       freely moving camera. This method advances the state of the art in radiometric calibration by allowing general image sets to be used, while previous
       methods were limited to images taken with a static camera or a rotating camera. The algorithm can be applied to radiometrically align images for
       seamless mosaics and 3D model textures as well as to generate high dynamic range (HDR) mosaics.

       ▪ Seon Joo Kim and Marc Pollefeys, "Robust Radiometric Calibration and Vignetting Correction",  IEEE Transactions on Pattern Analysis and Machine Intelligence 
         (PAMI), Vol. 30, No. 4, April, 2008 [pdf]

       ▪ Seon Joo Kim and Marc Pollefeys, "Radiometric Alignment of Image Sequences", Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 
          2004 [pdf]

                                                         
                                                            (Input)                                          (Aligned Output)

                 (Input)

               (Output : High Dynamic Range Mosaic)

      

    ● Joint Feature Tracking and Radiometric Calibration

        I presented an algorithm suited for video data taken with auto-exposure where the correspondence (feature tracks) and the radiometric response function along with the
        exposure values are computed simultaneously. The method advances the conventional feature tracking algorithm (KLT tracker) which requires the brightness of
        features to stay constant by unifying the problems of feature tracking and radiometric calibration into a common framework.

        ▪ Seon Joo Kim, David Gallup, Jan-Michael Frahm and Marc Pollefeys, "Joint radiometric calibration and feature tracking system with an application to stereo" ,        
          Computer Vision and Image Understanding (CVIU), Vol. 114, No. 5, May, 2010, 574-582   [pdf]

        ▪ Seon Joo Kim, Jan-Michael Frahm and Marc Pollefeys, "Joint Feature Tracking and Radiometric Calibration from Auto-Exposure Video",
          Proc. Int. Conf. on Computer Vision (ICCV), 2007 [pdf]

           - Video1 (Feature Tracking Comparison)

           - Video2 (Tracking & Radiometric Calibration)

           - Video3 (Tracking & Exposure Computation)


                 

             The system is suited for videos taken in a high dynamic range scene.  The graph above compares the exposure estimates to the ground truth.

                     
              
                                       (Top) Inputs, (Middle) Aligned Textures, (Bottom) Depth map before and after the alignment.

          The joint method can be applied to build an adaptive stereo system.   Our method provides robust feature tracking, simpler stereo cost function,
          and texture alignment.

      ● Radiometric calibration with illumination change for outdoor scene analysis

          I introduced a new algorithm to compute the radiometric response function and the exposure of images given a sequence of images of a static outdoor scene taken over
          time where the illumination is changing. This is the first radiometric calibration method to work with regular images with illumination changing where the lighting cannot
          be controlled. 

            ▪ Seon Joo Kim, Jan-Michael Frahm, and Marc Pollefeys, "Radiometric Calibration with Illumination Change for Outdoor Scene Analysis,
               Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR),  2008 [pdf]

              
             (Before and after the calibration. Notice that the brightness change in the original sequence is affected by the camera
             exposure. It does not show how the scene is really changing. The calibrated sequence shows the actual brightness change in the scene.)

          Video

     This work is accepted for the presentation at IEEE Visualization 2010:

       ▪ Seon Joo Kim, Shaojie Zhou, Fanbo Deng, Chi-Wing Fu, Michael S. Brown,
      "Interactive Visualization of Hyperspectral Images of Historical Documents", IEEE Transactions on Visualization and Computer Graphics (TVCG)
      (Proc. of IEEE Visualization Conference), 2010, To appear.

                (A) Contrast of some parts in the original RGB image (left) is enhanced with our gradient fusion method (middle). Side by side comparison of
            enhanced regions are shown on right. (B),(C) Enhancement using each band is shown locally through the hyperspectral lens. User can interactively
            choose the band that suits the application (highlighted with dashed line), where in this case is the contrast enhancement.

     I am also interested in tracking. I worked on projects developing tracking system for automatic surveillance as an intern at GE Research (2005) and at Cortex-US(2004).

     Here are some videos of the work done at GE. The goal is to detect salient motion from video (rotating PTZ camera) that contains other motions such as waving trees
     and flags as well as the camera motion.

     - Video1 (Blue regions indicate salient motion)

     - Video2 (Blue regions indicate salient motion / green regions indicate non-salient motion)

     - Video3

     The work at GE was presented in:
        S. J. Kim, G. Doretto,  J. Rittscher, P. Tu, N. Krahnstoever, M. Pollefeys, “ A model change detection approach to dynamic scene modeling”,  
        Proc. IEEE Conference   on Advanced Video and Signal Based Surveillance (AVSS), 2009   [pdf]

     I have also worked on tracking as class projects including face tracking and 3D active tracking.

     ▪ 3D active tracking demos

        video1,  video2,  video3

     ▪ Face tracking demos

        video1, video2, video3