Zoom Panoramas
Ranjitha Kumar
When we zoom in on a particular object in our environment, we often lose much of the environment surrounding the object. Panoramas offer a wider angle of view, solving the problem to a certain degree; however, since the camera is still at a very high magnification, only the immediate neighborhood surrounding the object is presented along with the object at very high magnification. Instead, a wide angle shot of the same region would capture a greater sense of context for the object at the cost of having the object of interest fixed at a lower resolution and magnification. Now suppose we have a series of frames that capture the transition from a wide angle shot to a telephoto shot. If we could seamlessly combine all the frames together into one large panorama such that the object of interest is presented at its highest magnification within the wide angle shot setting, we could place the magnified object within a broader context.
Therefore, I wrote a program that attempts to create such panoramas using frames taken from horizontal panning and zooming video input. The user gives a set of video frames as input to program and specifies what part of the last frame contains the magnified object of interest. Then, the program finds the least artifact path through the space-time volume, which is created by stacking the video frames one behind the other through time; flattening this path through the volume produces the desired panorama. This method ensures that every pixel neighborhood in the resulting panorama resembles some pixel neighborhood in one of the frames, creating an image that is pleasing to the appearance. The algorithm for my program was based heavily on the “Space-Time Scene Manifolds” paper, which describes an efficient method for creating panoramas from panning video [1]. I added the zoom component to this general framework. Below, I describe the various experiments I tried to test the performance of my program.
Figure 1. Flower-Bug Experiment
Initially, I saw great potential for “zoom panoramas” in the field of macro photography: suppose we wanted to capture a bug on edge of a petal at high magnification, but also wanted to capture the whole flower and the surrounding garden. Then we could potentially start with a wide angle shot of the flower and then zoom in on the bug to create this effect. (A chap stick was used as a makeshift bug in Fig. 1.) This experiment was moderately successful: it was a very windy day so the petals kept moving, creating shaky artifacts in the resulting panorama. To reverse some of the effect of the wind, I ran a post-processing algorithm to accentuate strong edges such as the edges of the petals.
Figure 2. Coach Store Front
In my second experiment, I panned first, and then held the video camera stationary while zooming in on the object of interest. To produce Fig. 2, I first zoomed across the front of the store until I reached the window pane with the purses, which were going to be the focus of the panorama; then, I zoomed on these purses. Notice how this technique projects the objects of interest out of the scene and into the viewer’s focus. I was still not completely satisfied with these results because transitions between the increasing degrees of magnification are not gradual.
Figure 3. Love
Figure 4. Faith
Figure 5. Hope
Figure 6. Charity
In my final set of experiments, I tried two new things: first, I used a tripod to remove vertical jitter, and second, I panned and zoomed simultaneously. I hypothesized that by panning and zooming at the same time I could a produce a more pronounced and gradual zoom effect in my final panorama. This hypothesis turned out to be correct; notice how the zooming effect in Figs. 3-6 is more apparent than in Fig. 2. In Figs. 3-6, I chose to focus on the angels displayed on the façade of Memorial Church. The creation of Figs. 3-6, has given me an idea for future work: I would like to create panoramas in which there were multiple points of high magnification and low magnification. For example, I would ultimately like to condense Figs. 3-6 into one panorama, in which all the angels would be at high magnification and the spaces in between them would be at low magnification. Therefore, the video input would consist of any ordering of panning, zooming in and zooming out. This technique would enable us to create an image which captured many objects that were spatially separated by distances exceeding their individual dimensions at high magnification.
References
[1] Y. Wexler D. Simakov, Space-Time Scene Manifolds, Proc. ICCV 2005.
