The aim of this project is to study the geometry and algebra of multiple camera systems. During the last decade there has been many attempts at making fully automatic structure and motion systems for ordinary camera systems. Much is known about minimal cases, feature detection, tracking and structure and motion recovery for ordinary cameras. Many automatic systems rely on small image motions in order to solve the correspondence problem. In combination with most cameras' small fields of view, this limits the way the camera can be moved in order to make good 3D reconstruction. The problem is significantly more stable with a large field of view. This has spurred research in so called omnidirectional or non-central cameras. A difficulty with ordinary cameras with or without large field of view is the inherent ambiguities that exists for structure and motion problem for ordinary cameras. There are ambiguous configurations for which structure and motion recovery is impossible.

In this project we will investigate the geometry behind an alternative approach to vision based structure and motion systems. We consider a rig equipped with multiple cameras. This gives a large combined field of view with simple and cheap cameras. If the cameras can be placed so that the focal points coincide, then the geometrical problems is identical to that of a single camera. Also if the cameras are positioned so that they have a large field of view in common, i.e. a stereo setup, there are known techniques for calculating structure and motion. In this project we consider cameras where neither of these constraints have to be satisfied.

More specifically we will (i) identify and model structure and motion problems geometrically as illustrated above, (ii) solve such problems, (iii) generalise problems and solutions, e.g. with respect to higher dimension, other feature types (lines, conics, curves, quivers), to infinitesimal motion, missing data, ambiguous configurations, (iv) study applications of the theory to autonomous guided vehicles (interest point detection, descriptors, tracking, recognition, structure and motion estimation, reasoning), (v) study applications of the theory to 3D reconstruction using multiple moving cameras. Many of these problems involve study of families of polynomial equations in several variables. Thus we would also like to (vi) develop techniques for solving such equations in a fast and numerically stable manner, e.g. using methods from algebraic geometry and numerical linear algebra.

Funded by the Swedish Research Council.

Principal Investigator: Kalle Åström.

Period: 2009-2011.