Real-time object tracking without feature extraction

The motion of the object is estimated by minimizing the difference between real and rendered images. The 3-D shape of the object is measured in prior, and we can generate image from any viewpoint using computer-graphics technique. The texture of the object is also modeled, therefore, the appearance of the generated image must be coincide if the estimated posture of the object is equal to the real object. Numerous minimization techniques heve been proposed by computational mathematics researches, therefore we can track the full rigid motion of the object in real-time.

Real-time 6-DOF tracking of curved object using 4 cameras Movie of 6-DOF tracking (4 times fast forwarded) We propose a model-based method using intensity images taken with multiple viewpoint cameras connected to PC cluster system. At first, the whole 3-D shape and reflectance model of the ob ject are prepared using rangefinders. Each rangefiner is constructed with a camera, pro jector and PC, and all PCs are connected via network each other. For tracking the ob ject, several CG images with varied ob ject pose and position are generated in each PC using the ob ject model, and then compared to the input intensity image in parallel. The result of the comparison is transferred to a master PC, and the pose and position of the ob ject are estimated by minimizing the residual of the CG and input images. We made a special CG generator which is a precise simulator of the real camera to generate a CG image identical to the input image. We confirmed the ability of our method and achieved a 6DOF real-time ob ject tracking system using four cameras in the rate of 5 frames/sec.

Real-time 6-DOF tracking and modeling (SLAM) using stereo camera Movie of 6-DOF tracking We propose a method to determine the 3-D structure of the scene and camera motion simultaneously using stereo camera. The shape of the scene is measured by usual stereo method, and it is used to track the motion of the camera in real-time. The position and posture of the camera are calculated by minimizing difference between input image and CG image which is rendered from measured 3-D shape of the scene. If the field of view is changed enough, the 3-D measurement of the scene is activated. The computational load of stereo matching is so heavy that we must avoid the performance degradation of motion tracking while stereo measurement to make the tracking enough precise and sta- ble. Therefore, we parallelized two processes, and propose some ingenious methods to refine the estimated camera motion. In experiments, we confirmed that stable and precise tracking and modeling are realized without interference of intermittent modeling process activation.

Real-time tracking of mirrored surface High-light of specular objects causes errors in video tracking. Especially, very bright area caused by the reflection of light source may erase interest points in feature-based methods. In order to overcome the problem, we propose an appearance-based method which estimates the position and posture of a target object by minimizing the difference between an input image and CG images rendered from the model of object and environment. Lighting condition of environment is modeled by the sphere mapping technique, and used for rendering images with precisely measured object shape on graphics accelerator hardware. In experiments, we confirmed that mirror-finished object is tracked accurately in half of video-rate. Spherical mirror for environment (lighting) modeling, white object for shape measurement, mirrored object as tracking target

Performed with :

See also :

• Takayuki Moritani, Shinsaku Hiura and Kosuke Sato: Object Tracking by Comparing Multiple Viewpoint Images to CG Images," Systems and Computers in Japan, Vol.37, No.13, pp. 28-39 (Oct. 2006)

• Shinsaku Hiura, Takayuki Moritani and Seiji Inokuchi: Object Tracking by Comparing Generated CG Images, Proc. of SICE2002(CD-ROM) (2002)
  
• Takayuki Moritani, Shinsaku Hiura and Seiji Inokuchi : Object Tracking by Comparing CG Images with Multiple Viewpoint Images, IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI2003), pp. 241-246 (Jul. 2003 at Tokyo)
  
• Takayuki Moritani, Shinsaku Hiura, Kosuke Sato, Visual Object Tracking Based on Multi-Viewpoint 3-D Gradient Method, 1st International Symposium on Systems & Human Science (SSR2003), pp. 257-274 (Nov. 2003 at Osaka)
  
• Takayuki Moritani, Satoshi Kawabata, Reiko Miyagi, *Shinsaku Hiura, Kosuke Sato : Autonomous Scene Monitoring System based on Multi Viewpoint Images, Proc. of SICE2004 CD-ROM (Oct. 2004 at Osaka)
  
• Takayuki Moritani, Shinsaku Hiura, Kosuke Sato : Tracking Multiple Objects Occluding each other Without Feature Extraction, SSR2005 CD-ROM (Mar. 2005 at San Francisco)
  
• Shinsaku Hiura, Takayuki Moritani, Kosuke Sato : Scene Analysis based on an Optimization of CG Images, SIGGRAPH2005 (Jul. 2005 at Las Vegas)
  
• Takayuki Moritani, Shinsaku Hiura, Kosuke Sato : Simultaneous object modeling and tracking using stereo cameras without feature extraction, SSR2006 CD-ROM(Mar. 2006 at Vienna)
  
• Takayuki Moritani, Shinsaku Hiura and Kosuke Sato : Real-Time Object Tracking without Feature Extraction, Proc. of 18th ICPR, pp.747-750 (Aug. 2006 at Hong Kong)