Interacting with information and robotics systems
The aim of this paper is to present the results obtained during the project 131-CEEX/2006, INTEROB: Interacting by Gestures with Information and Robotics Systems for the time frame 2006-2008. The project focused on natural interaction using gesture recognition technology applied in various applicati...
Gespeichert in:
| Datum: | 2008 |
|---|---|
| Hauptverfasser: | , , , , , , , |
| Format: | Artikel |
| Sprache: | English |
| Veröffentlicht: |
Інститут фізики напівпровідників імені В.Є. Лашкарьова НАН України
2008
|
| Schriftenreihe: | Оптико-електронні інформаційно-енергетичні технології |
| Schlagworte: | |
| Online Zugang: | http://dspace.nbuv.gov.ua/handle/123456789/32188 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Назва журналу: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Zitieren: | Interacting with information and robotics systems / S.G. Pentiuc, R.D. Vatavu, O.C. Ungurean, G. Laurent, C. Christophe, L. De Backer, L. De Strycker, Il. Hamburg // Оптико-електронні інформаційно-енергетичні технології. — 2008. — № 2 (16). — С. 105-108. — Бібліогр.: 6 назв. — англ. |
Institution
Digital Library of Periodicals of National Academy of Sciences of Ukraine| id |
irk-123456789-32188 |
|---|---|
| record_format |
dspace |
| spelling |
irk-123456789-321882012-04-15T00:19:59Z Interacting with information and robotics systems Pentiuc, S.G. Vatavu, R.D. Ungurean, O.C. Laurent, G. Christophe, C. De Backer, L. De Strycker, L. Hamburg, Il. Системи технічного зору і штучного інтелекту з обробкою та розпізнаванням зображень The aim of this paper is to present the results obtained during the project 131-CEEX/2006, INTEROB: Interacting by Gestures with Information and Robotics Systems for the time frame 2006-2008. The project focused on natural interaction using gesture recognition technology applied in various applications such as: virtual environments, augmented reality, collaborative working and interaction with robotics systems. 2008 Article Interacting with information and robotics systems / S.G. Pentiuc, R.D. Vatavu, O.C. Ungurean, G. Laurent, C. Christophe, L. De Backer, L. De Strycker, Il. Hamburg // Оптико-електронні інформаційно-енергетичні технології. — 2008. — № 2 (16). — С. 105-108. — Бібліогр.: 6 назв. — англ. 1681-7893 http://dspace.nbuv.gov.ua/handle/123456789/32188 en Оптико-електронні інформаційно-енергетичні технології Інститут фізики напівпровідників імені В.Є. Лашкарьова НАН України |
| institution |
Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| collection |
DSpace DC |
| language |
English |
| topic |
Системи технічного зору і штучного інтелекту з обробкою та розпізнаванням зображень Системи технічного зору і штучного інтелекту з обробкою та розпізнаванням зображень |
| spellingShingle |
Системи технічного зору і штучного інтелекту з обробкою та розпізнаванням зображень Системи технічного зору і штучного інтелекту з обробкою та розпізнаванням зображень Pentiuc, S.G. Vatavu, R.D. Ungurean, O.C. Laurent, G. Christophe, C. De Backer, L. De Strycker, L. Hamburg, Il. Interacting with information and robotics systems Оптико-електронні інформаційно-енергетичні технології |
| description |
The aim of this paper is to present the results obtained during the project 131-CEEX/2006, INTEROB: Interacting by Gestures with Information and Robotics Systems for the time frame 2006-2008. The project focused on natural interaction using gesture recognition technology applied in various applications such as: virtual environments, augmented reality, collaborative working and interaction with robotics systems. |
| format |
Article |
| author |
Pentiuc, S.G. Vatavu, R.D. Ungurean, O.C. Laurent, G. Christophe, C. De Backer, L. De Strycker, L. Hamburg, Il. |
| author_facet |
Pentiuc, S.G. Vatavu, R.D. Ungurean, O.C. Laurent, G. Christophe, C. De Backer, L. De Strycker, L. Hamburg, Il. |
| author_sort |
Pentiuc, S.G. |
| title |
Interacting with information and robotics systems |
| title_short |
Interacting with information and robotics systems |
| title_full |
Interacting with information and robotics systems |
| title_fullStr |
Interacting with information and robotics systems |
| title_full_unstemmed |
Interacting with information and robotics systems |
| title_sort |
interacting with information and robotics systems |
| publisher |
Інститут фізики напівпровідників імені В.Є. Лашкарьова НАН України |
| publishDate |
2008 |
| topic_facet |
Системи технічного зору і штучного інтелекту з обробкою та розпізнаванням зображень |
| url |
http://dspace.nbuv.gov.ua/handle/123456789/32188 |
| citation_txt |
Interacting with information and robotics systems / S.G. Pentiuc, R.D. Vatavu, O.C. Ungurean, G. Laurent, C. Christophe, L. De Backer, L. De Strycker, Il. Hamburg // Оптико-електронні інформаційно-енергетичні технології. — 2008. — № 2 (16). — С. 105-108. — Бібліогр.: 6 назв. — англ. |
| series |
Оптико-електронні інформаційно-енергетичні технології |
| work_keys_str_mv |
AT pentiucsg interactingwithinformationandroboticssystems AT vatavurd interactingwithinformationandroboticssystems AT ungureanoc interactingwithinformationandroboticssystems AT laurentg interactingwithinformationandroboticssystems AT christophec interactingwithinformationandroboticssystems AT debackerl interactingwithinformationandroboticssystems AT destryckerl interactingwithinformationandroboticssystems AT hamburgil interactingwithinformationandroboticssystems |
| first_indexed |
2025-07-03T12:43:04Z |
| last_indexed |
2025-07-03T12:43:04Z |
| _version_ |
1836629711693807616 |
| fulltext |
5
STEFAN-GHEORGHE PENTIUC
1
, RADU-DANIEL VATAVU
1
, OVIDIU CIPRIAN UNGUREAN
1
,
LAURENT GRISONI
2
, CHRISTOPHE CHAILLOU
2
, LUC DE BACKER
3
, LIEVEN DE STRYCKER
3
,
ILEANA HAMBURG
4
INTERACTING WITH INFORMATION AND ROBOTICS SYSTEMS
1
University Stefan cel Mare of Suceava,
str.Universitatii nr.13, RO-720229 Suceava, Romania
ungurean.ovidiu@gmail.com
2
Laboratoire d'Informatique de Lille,
Villeneuve d’Ascq 59650, France,
Christophe.Chaillou@lifl.fr
3
University of Gent, KAHO Gent,
Sint-Pietersnieuwstraat 25 B - 9000 Ghent, Belgium
<luc.debacker@kahosl.be>;
4
Institute for Work and Technology Gelsenkirchen, Germany
<hamburg@iat.eu>
Abstract.. The aim of this paper is to present the results obtained during the project 131-CEEX/2006,
INTEROB: Interacting by Gestures with Information and Robotics Systems for the time frame 2006-2008. The
project focused on natural interaction using gesture recognition technology applied in various applications such
as: virtual environments, augmented reality, collaborative working and interaction with robotics systems.
Key words: INTEROB project, virtual objects.
INTRODUCTION
The INTEROB project (Interacting by Gestures with Information and Robotics
Systems) was launched in 2006 with the main objective to develop natural
gesture-based interaction for interfacing computers and robots. Natural interaction
is an aspect that became more and more important in the dialog human –
informatics / robotics systems while gestures represent such an intuitive and
natural way of interaction and communication. Fundamental and applicative
research was targeted for this project in what concerns the achievement of high
technological models for the recognition and interpretation of human gestures
having as final aim the natural interaction with:
- information systems such as virtual reality applications (interaction and
navigation inside a virtual environment)
- information systems such as augmented reality applications (interaction with
virtual and real objects)
- robotics systems (natural gesture-based commands for mobile and static robots)
The focus was mainly on video processing using techniques and methods that are
specific to image processing, pattern recognition and computer vision with the
following attainable objectives:
STEFAN-GHEORGHE PENTIUC, RADU-DANIEL VATAVU, OVIDIU CIPRIAN UNGUREAN,
LAURENT GRISONI, CHRISTOPHE CHAILLOU, LUC DE BACKER, LIEVEN DE STRYCKER, ILEANA
HAMBURG, 2008
ПРИНЦИПОВІ КОНЦЕПЦІЇ ТА СТРУКТУРУВАННЯ РІЗНИХ РІВНІВ ОСВІТИ З ОПТИКО-ЕЛЕКТРОННИХ ІНФОРМАЦІЙНО-
ЕНЕРГЕТИЧНИХ ТЕХНОЛОГІЙ
6
real time acquisition and interpretation of hand and head gestures using a
stereoscopic view
embedded system for the recognition and interpretation of human gestures
interaction system for a 3D virtual reality application as well as for an augmented
reality system for a smart desk application
experiment the proposed technology with robotics systems: Hercules, a static arm
type robot and a mobile robot prototype developed in collaboration with
Polytech’Lille.
The project is part of the thematic area 3.2.3 on natural interactions using human
gestures and under the technological platform PT3 – embedded system by
developing a smart device using the System on Chip technology of the Research
of Excellence Romanian national funding programme. The tasks were carried out
by 6 Romanian partners (University "Ştefan cel Mare" of Suceava, Technical
University “Gheorghe Ascachi” of Iasi, University “Alexandru Ioan Cuza” Iasi,
AGIR – Association of Engineers from Romania and 2 SMEs). The project also
included 4 international partners (Laboratoire d'Informatique de Lille, France,
Ecole Polytechnique de Lille, France, University of Gent, KAHO Gent, Belgium
and Institute for Work and Technology Gelsenkirchen, Germany).
OBJECTIVES
Gestures represent a natural and intuitive way of interaction and communication:
people gesture in order to express ideas, concepts; we gesture in order to interact
with objects. The projected considered the theme of human gesture recognition
for interaction with information systems (virtual reality and augmented reality
applications) and robotics systems. The accent was on video processing using
techniques specific to image processing, pattern recognition and computer
vision.
The main objective was to obtain a mathematical model, algorithms, programs
and hardware architectures in order to achieve a natural interaction mechanism
including flexible, efficient and ergonomic gestures. The following measurable
objectives have been identified:
• design of a gesture dictionary composed of both hand postures as well
as dynamic gestures (hand postures with motion trajectories; head
movements), taking into account ergonomic aspects
• real time acquisition and interpretation of hand and head gestures (using
the stereoscopic view, 2 video cameras)
• development of an embedded system for preprocessing the video
information with a System on Chip architecture
• implementation of the proposed interaction system inside a 3D virtual
reality application
• implementation of the proposed system inside an augmented reality
application for a smart desk
ПРИНЦИПОВІ КОНЦЕПЦІЇ ТА СТРУКТУРУВАННЯ РІЗНИХ РІВНІВ ОСВІТИ З ОПТИКО-ЕЛЕКТРОННИХ ІНФОРМАЦІЙНО-
ЕНЕРГЕТИЧНИХ ТЕХНОЛОГІЙ
7
• experimenting the interaction technology with robotics systems (the
static arm type robot Hercules developed at the University of Suceava
and a mobile prototype robot developed in collaboration with
Polytech’Lille, France)
Considering the CEEX Research of Excellence programme’s objectives, the
project considered:
• development of the Romanian RD system in order to accumulate new
knowledge, results and high edge experience in a leading scientific and
technological domain (natural interaction with information and robotics
systems) as well as assuring the transfer of the results towards the inner
social environment for the development of its competitively
• supporting formation, development, integration and consolidation of
the HCI domain with respect to the international standards and
research
PROJECT JUSTIFICATION AND BENEFITS
By approaching the very modern and up to date domain of gesture based
interaction with information and robotic systems, the proposal unified in
common bounds the specific and specialization of the research previously
conducted by the national partners as well as the European collaborators for the
domains of video processing, pattern recognition, virtual and augmented reality
applications and robotic systems.
The benefits of the project can be identified in the domain of basic fundamental
research in the psycholinguistic field (by the means of studies on gestures and
on the user’s capacity of communication with information and robotics systems)
as well as in the applicative research by developing a gesture based interaction
system and validating it by 4 implementations in 4 different scenarios (virtual
reality, augmented reality, static robot controlling, mobile robot control). We
consider the project as being multi-disciplinary, at the frontier of scientific and
technical knowledge. The proposal allowed identification of human and material
resources of high specialization in RD domains with consequences in promotion
of the compatibility of the national research with regards to the European Union
and PC7 tendencies.
RESULTS
The proposal aimed both fundamental research in what concerns human gestures
and the user’s capacity of communicating by means of gesture with information
and robotics systems as well as applicative research by developing an interaction
system using only video information and its validation by implementing the
system in 4 different scenarios. The following results have been obtained during
the project:
1) Design of formalism for gestures and gesture dictionaries from the point of view of the human computer
interaction field as well as a gesture dictionary consisting in postures and trajectories, validated by
ПРИНЦИПОВІ КОНЦЕПЦІЇ ТА СТРУКТУРУВАННЯ РІЗНИХ РІВНІВ ОСВІТИ З ОПТИКО-ЕЛЕКТРОННИХ ІНФОРМАЦІЙНО-
ЕНЕРГЕТИЧНИХ ТЕХНОЛОГІЙ
8
ergonomic studies and experimentations using volunteer users. (Figure 1). We addressed the multiple
forms of representation that human gesture takes at different levels in human computer interaction. The
various representations range from raw data in gesture acquisition, mathematical model for analysis,
pattern for recognition, record for database up to event for end-level applications. We introduce a
hierarchical view for gesture processing as it appears in HCI.
2) Recognition of hands and head gestures (including hand postures, gesture trajectories and head pitch
and yaw movements) (Figures 2 and 3). Recognition of the hand postures was performed using a multi-
layered perceptron for which a large database samples was constructed. The reported performance of
the network accuracy was over 98%.
3) Interaction with virtual reality and performing virtual object manipulation using one or two hands
(selection, translation, rotation and change of scale) (Figure 2).
4) Interaction with robotics systems (Figure 4).
Fig. 1. Gesture dictionary for working with virtual objects
Fig. 2. Interacting with virtual objects: manipulation inside virtual environments (selection, translation, rotation,
change of scale)
ПРИНЦИПОВІ КОНЦЕПЦІЇ ТА СТРУКТУРУВАННЯ РІЗНИХ РІВНІВ ОСВІТИ З ОПТИКО-ЕЛЕКТРОННИХ ІНФОРМАЦІЙНО-
ЕНЕРГЕТИЧНИХ ТЕХНОЛОГІЙ
9
Fig. 3. Detecting head movements for simulating mouse events
Fig. 4. Interacting with robotics systems
ACKNOWLEDGEMENTS
This work was supported from the national funding grant “Research of
Excellence”, Ref. No. 131/2006, acronym INTEROB.
REFERENCES
1. Ştefan Gheorghe Pentiuc, Radu-Daniel Vatavu, Ciprian-Ovidiu Ungurean, Tudor Ioan Cerlinca,
Techniques for Interacting by Gestures with Information Systems, European Conference on the Use of
Modern Information and Communication Technologies, ECUMICT 2008, Gent, Blegium, 13-14 March
2008.
2. Pentiuc, S.G., Vatavu, R., Cerlinca, T.I., and Ungureanu, O. “Methods and Algoritms for Gestures
Recognition and Understanding”. The Eighth All-Ukrainian International Conference,
UkrOBRAZ’2006, pp. 15-18, Ukraine, August 2006.
3. Radu-Daniel Vatavu, Ştefan-Gheorghe Pentiuc, Christophe Chaillou, On Natural Gestures for Interacting
with Virtual Environments, Advances in Electrical and Computer Engineering, 24(5), 2005, pp. 72-29.
4. Radu-Daniel Vatavu, Ştefan-Gheorghe Pentiuc, Interacting with Gestures: An Intelligent Virtual
Environment, 1st International Conference on Virtual Learning, ICVL, 2006, pp. 293-299.
5. Radu-Daniel Vatavu, Ştefan-Gheorghe Pentiuc, Christophe Chaillou, Laurent Grisoni, Samuel
Degrande, Visual Recognition of Hand Postures for Interacting with Virtual Environments, Advances in
Electrical and Computer Engineering, Volume 6 (13), Number 2(26), 2006, University „Stefan cel
Mare” of Suceava, ISSN 1582-7445, pp. 55-58 (republication in special issue from Development and
Application Systems 2006).
6. Radu-Daniel Vatavu, Ştefan-Gheorghe Pentiuc, Multi-Level Representation of Gesture as Command for
Human Computer Interaction, to appear in the Computing and Informatics, 2008.
Надійшла до редакції: 29.11.2008р.
ПРИНЦИПОВІ КОНЦЕПЦІЇ ТА СТРУКТУРУВАННЯ РІЗНИХ РІВНІВ ОСВІТИ З ОПТИКО-ЕЛЕКТРОННИХ ІНФОРМАЦІЙНО-
ЕНЕРГЕТИЧНИХ ТЕХНОЛОГІЙ
10
STEFAN-GHEORGHE PENTIUC - professor on the Faculty of Electrical Engineering and
Computer Science University “Stefan cel Mare” of Suceava, Romania, phone: +40.230.524.801,
http://www.eed.usv.ro/~pentiuc, Email: pentiuc@usv.ro
|