Title: "Using Shape Priors to Regularize Intermediate Views in Wide-Baseline Image-Based Rendering"
Speaker: CÚdric Verleysen
Location: "Shannon" Seminar Room (a105) Place du Levant 3, Maxwell Building, 1st floor
Date / Time (duration): Wednesday 19/3/2014, 10h00 (~ 45')
Abstract: Nowadays, when a viewer watches a video content, his/her viewpoint is fixed to one of the cameras that have recorded the scene. In order to increase the viewer's immersivity, the next generation of video content will allow him/her to interactively define his/her viewpoint. This domain is known as free-viewpoint rendering, and consists of the interpolation of views from images captured by some real cameras. However, the state-of-the-art solutions require that the real cameras share very similar viewpoints, meaning that a close scene can be rendered only with a dense camera network, and that far scenes can be rendered only with very high-resolution cameras. These requirements make free-viewpoint rendering an expensive technology, slowing down its entry on the market.
During this presentation, we will see that it is possible to reconstruct intermediate views of a specific object in-between two arbitrary calibrated cameras which have very different viewpoints (relative angle of approximatively 45░). Such reconstruction is ill-posed, because of the multiple occlusions and the strong foreshortening effect present in such camera networks (wide-baseline stereo pairs). We propose to regularize this reconstruction, in such a way that the silhouette of the reconstructed object, such as observed in the intermediate views, tends to belong to a manifold representing the possible deformations of the shape of the object.
Precisely, we will first introduce a new kind of image element, called the epipolar line segment, and we will see that its transformation, when moving from one real view to another, is limited to a 1D translation and 1D scaling. Then, we will regularize the determination of the parameters of these transformations based on a prior knowlede about possible shapes of the reconstructed object. This prior knowledge is acquired by learning a nonlinear shape manifold representing the possible silhouettes of the object, such as described by Elliptic Fourier Descriptors.
Finally, we will see that that the proposed method preserves the topological structure of the objects during the intermediate view synthesis, while effectively dealing with the occluded regions and with the severe foreshortening effect associated to wide-baseline configurations.
Last updated May 01, 2014, at 11:35 AM