A Gaze-Speech System in Mixed Reality for Human-Robot Interaction

John David Prieto Prada, Myung Ho Lee, Cheol Song

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Human-robot interaction (HRI) demands efficient time performance along the tasks. However, some interaction approaches may extend the time to complete such tasks. Thus, the time performance in HRI must be enhanced. This work presents an effective way to enhance the time performance in HRI tasks with a mixed reality (MR) method based on a gaze-speech system. In this paper, we design an MR world for pick-and-place tasks. The hardware system includes an MR headset, the Baxter robot, a table, and six cubes. In addition, the holographic MR scenario offers two modes of interaction: gesture mode (GM) and gaze-speech mode (GSM). The input actions during the GM and GSM methods are based on the pinch gesture and gaze with speech commands, respectively. The proposed GSM approach can improve the time performance in pick-and-place scenarios. The GSM system is 21.33 % faster than the traditional system, GM. Also, we evaluated the target- to-target time performance against a reference based on Fitts' law. Our findings show a promising method for time reduction in HRI tasks through MR environments.

Original languageEnglish
Title of host publicationProceedings - ICRA 2023
Subtitle of host publicationIEEE International Conference on Robotics and Automation
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages7547-7553
Number of pages7
ISBN (Electronic)9798350323658
DOIs
StatePublished - 2023
Event2023 IEEE International Conference on Robotics and Automation, ICRA 2023 - London, United Kingdom
Duration: 29 May 20232 Jun 2023

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
Volume2023-May
ISSN (Print)1050-4729

Conference

Conference2023 IEEE International Conference on Robotics and Automation, ICRA 2023
Country/TerritoryUnited Kingdom
CityLondon
Period29/05/232/06/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Fingerprint

Dive into the research topics of 'A Gaze-Speech System in Mixed Reality for Human-Robot Interaction'. Together they form a unique fingerprint.

Cite this