Panoramic Mosaics using PTZ Cameras

Panoramic Mosaics : using Pan Tilt Zoom Cameras.

Introduction

Results

Outdoor Panoramas

Indoor Panoramas

Office Room

Lobby

Observations.

Introduction

As part of the research on using active camera networks for 3D reconstruction of events in wide-area environments, we implemented a completely automatic, feature-based
method for computing calibrated panoramic mosaics from pan-tilt-zoom cameras (Sony SNC-RZ30, Canon VB-C10). Results from some of the datasets are shown here alongwith relevant details

Results

[camera]
We built panoramas in an outdoor environment, and two indoor environments. Details are shown below :

Outdoor Panoramas :

These mosaics were created from two cameras placed on window-sills of two adjacent windows (4-5 meters apart ) of Sitterson Hall, UNC Chapel Hill.
The observed construction site is about 80-120 meters in length and width. The cameras were setup to automatically capture images at pre-destined times of
the day independent of each other (no synchronization). Currently we only handle static scenes and do not segment moving objects.

(1) Dataset 179, Camera B : 16 Mpixels Mosaic

Details :

Mosaic cube-map face : 4K x 4K pixels (uncropped)
# Images captured = 12 x 5 = 60, at roughly 3.5X zoom.
Root mean square Residual (RMS)

registration Error = 0.37 pixels

Timings :

Image Capture = l min.
Computing Homographies = 10 - 12 mins.
Bundle Adjustment = 10 - 15 mins.
Rendering Cube Map = 1 - 2 mins

Full-res image (1.2 MB)

(2) Dataset 184, Camera A : 36 Mpixel Mosaic

Details :

Mosaic cube-map face : 6K x 6K pixels (uncropped)
# Images captured = 17 x 7 = 119 , at roughly 5.5X zoom.
RMS Registration Error = 0.43 pixels.

Timings :

Image Capture = 2 mins.
Computing Homographies = 25 - 30 mins.
Bundle Adjustment = 50 mins.
Rendering Cube Map = 5 mins.

Full-res image (2 MB)

One of the 119 images

(3) Dataset 177, Camera A & B : 9 Mpixel Mosaic Pair, Rectified Image Pair.

		Details : Pair of mosaics computed using camera pair, A and B. Each mosaic cube-map face : 3K x 3K pixels (uncropped) # Images captured = 12x5 = 60, at roughly 2.8X zoom. Full-res Left Image, Camera A. Full-res Right Image, Camera B.
		The above pair was rectified after computing the Fundamental Matrix robustly from the a pair of cubemap panoramic mosaics. Low-res, Original Left Image, Camera A. Low-res, Original Right Image, Camera B. Low-res, Rectified Left Image, Camera A. Low-res, Rectified Right Image, Camera B.

Indoor Panoramas :
We also tried our method on datasets captured in indoor settings. First we tried scanning an office room with a similar camera pair. Only one of the
panoramas is displayed below. Our indoor scenes had fewer reliable features compared to the outdoor scene and this made it difficult to create
mosaics at higher resolutions.

Office Room : 9 Mpixel Mosaic

Images of the office scene were noisier and we had to over-expose the camera during image capture. The auto-focus feature of the camera sometimes
produced blurred images as objects in the scene were often a lot nearer.

Details :

Mosaic cube-map face : 3K x 3K pixels (uncropped)
# Images captured = 12 x 5 = 60 , at roughly 2.7X zoom.
RMS Registration Error = 0.41 pixels.

Timings :

Image Capture = 2 mins.
Computing Homographies = 30-35 mins.
Bundle Adjustment = 12-13 mins.
Rendering Cube Map = 1-2 mins.

Full-res image
One of the 60 images

Lobby : 9 Mpixel Mosaic

In the lobby of Sitterson Hall, there was enough illumination, however this gave rise to blending artifacts as images were captured with auto-exposure settings.
However this did not give rise to any registration errors. Higher-resolution panoramas of such scenes can only be created by looking for details wherever it exists instead of capturing detail everywhere.

Details :

Mosaic cube-map face : 3K x 3K pixels (uncropped)
# Images captured = 12 x 5 = 60 , at roughly 2.7X zoom.
RMS Registration Error : 0.54 pixels.

Full-res left image

Full-res right image

Observations :

Work in Progress :

Fix blending artifacts caused by camera's auto-exposure by estimating the camera response function and build HDR images.
Handle moving objects in scene using background modeling and create mosaics where they are absent completely.
Build multi-resolution panoramic pyramids.
Create an out-of-core viewer for viewing large images at multiple resolutions.
Create a real-time pan-tilt-zoom camera simulator which would generate novel views from the mosaics in an out-of-core fashion, for any PTZ configuration.

Sudipta Sinha : ( ssinha@cs.unc.edu ) March 4, 2004.