University of Oulu

Camera based motion estimation and recognition for human-computer interaction

Saved in:
Author: Hannuksela, Jari1,2
Organizations: 1University of Oulu, Faculty of Technology, Department of Electrical and Information Engineering
2University of Oulu, Infotech Oulu
Format: ebook
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 1.3 MB)
Persistent link:
Language: English
Published: 2008
Publish Date: 2008-12-09
Thesis type: Doctoral Dissertation
Defence Note: Academic dissertation to be presented, with the assent of the Faculty of Technology of the University of Oulu, for public defence in Auditorium TS101, Linnanmaa, on December 19th, 2008, at 12 noon
Reviewer: Doctor Ville Kyrki
Professor Roope Raisamo


Communicating with mobile devices has become an unavoidable part of our daily life. Unfortunately, the current user interface designs are mostly taken directly from desktop computers. This has resulted in devices that are sometimes hard to use. Since more processing power and new sensing technologies are already available, there is a possibility to develop systems to communicate through different modalities. This thesis proposes some novel computer vision approaches, including head tracking, object motion analysis and device ego-motion estimation, to allow efficient interaction with mobile devices.

For head tracking, two new methods have been developed. The first method detects a face region and facial features by employing skin detection, morphology, and a geometrical face model. The second method, designed especially for mobile use, detects the face and eyes using local texture features. In both cases, Kalman filtering is applied to estimate the 3-D pose of the head. Experiments indicate that the methods introduced can be applied on platforms with limited computational resources.

A novel object tracking method is also presented. The idea is to combine Kalman filtering and EM-algorithms to track an object, such as a finger, using motion features. This technique is also applicable when some conventional methods such as colour segmentation and background subtraction cannot be used. In addition, a new feature based camera ego-motion estimation framework is proposed. The method introduced exploits gradient measures for feature selection and feature displacement uncertainty analysis. Experiments with a fixed point implementation testify to the effectiveness of the approach on a camera-equipped mobile phone.

The feasibility of the methods developed is demonstrated in three new mobile interface solutions. One of them estimates the ego-motion of the device with respect to the user’s face and utilises that information for browsing large documents or bitmaps on small displays. The second solution is to use device or finger motion to recognize simple gestures. In addition to these applications, a novel interactive system to build document panorama images is presented.

The motion estimation and recognition techniques presented in this thesis have clear potential to become practical means for interacting with mobile devices. In fact, cameras in future mobile devices may, for the most of time, be used as sensors for self intuitive user interfaces rather than using them for digital photography.

see all

Series: Acta Universitatis Ouluensis. C, Technica
ISSN-E: 1796-2226
ISBN: 978-951-42-8978-1
ISBN Print: 978-951-42-8977-4
Issue: 313
Copyright information: © University of Oulu, 2008. This publication is copyrighted. You may download, display and print it for your own personal use. Commercial use is prohibited.