Telepresence technology
Background
Remote work is not always an option because a job may primarily involve on-site work which would be less efficient when performed remotely, or require face-to-face communication. To facilitate remote work in jobs that require on-site work, it is necessary to have some way of performing the work on-site by remote control (e.g., by machines or robots/artificial bodies). However, the objectives and timing of operations performed at the remote site can differ from what was intended by the remote operator due to a lack of repeatability or delays in the response of the operation interface or artificial body. This can significantly decrease efficiency and productivity. It should also be possible to accurately ascertain the local situation from a remote location, but problems may arise such as lack of information for recognizing changes in the current environment or behavior of people on site. Furthermore, there is a tendency for people who are present on site to feel uncomfortable and anxious about working with someone who is in a remote location.
With the telepresence technology we are developing, we aim to make it possible for a person to use an artificial body to perceive the situation at a remote location and interact with other people and the environment as if they are working on site. Thus, they can carry out their required tasks and engage in face-to-face communication without any hindrance. We are currently researching and developing cognitive assistive robotics and lifelike communications technologies that will be needed to make this telepresence a reality. Cognitive assistive robotics aims to improve the operability at remote locations by predicting motions and reactions to compensate for latency and lack of operational information, while lifelike communication technology provides methods of delivering information with the same quality of experience as being present on site.
Approach
Cognitive assistive roboticsa
Cognitive assistive robotics is used to create a system with which human operators can conduct the majority of their work from a remote location by supporting the remote operation of robots. The main factors that hinder people's ability to work by remote control are limitations of the operating system or robot and differences between a human and robot body structures. Cognitive assistive robotics aims to solve these problems through techniques that support and extend human movement and cognitive abilities.
Lifelike communication
With the spread of remote work, communication using video conferencing tools such as Zoom, MS teams, and Webex has become commonplace. However, in contrast to face-to-face communication, it is difficult to comprehend the gaze, facial expressions, and gestures of others through a screen, which can lead to misunderstandings.
Considering such limitations of remote communication, we are studying lifelike communication technology to reproduce the experience of face-to-face communication remotely. In addition, we are exploring information presentation methods and user interfaces that bring the quality of remote communication as close as possible to that of face-to-face communication.
To achieve lifelike communication? our research focuses on the subjective sensations of the speaker, which are important in the communication experience, rather than attempting to faithfully reproduce the elements of face-to-face communication by using highly realistic and highly detailed information representation, which has been used in conventional telecommunication studies.
To sufficiently acquire subjective sensations, we have been studying methods of presenting information that are suitable for remote communication. For example, we proposed a method for representing the sense of presence and distance that visitors feel from the body of an exhibitor at a technical exhibition held online. This method utilized stereophonic sound to represent the voice of the exhibitor's explanation, as if the visitor were receiving an explanation face-to-face. We determined that the proposed method can induce the experience of receiving an explanation from the exhibitor.
Additionally, we aim to improve communication between athletes and coaches in sports training by expressing sensations that are difficult to convey directly in verbal through other modalities. For example, focusing on windsurfing, we are enhancing the expression of the sensation of sprinting and grasping equipment on sailing to make communication more effective during coaching. We are also studying remote assessment, in which a remote nurse observes a patient via video communication. we propose method that can express a sense of focus and peering into the patient by the nurse, making it easier to convey the nurse’s intentions and realize remote assessment of the lifelike face-to-face assessment. In the future, we aim to express such bodily sensations remotely, devise presentation methods tailored to individual users, and utilize prosthetic bodies.
Future works
We will proceed with technical development and verification in specific fields such as nursing and equipment maintenance in which it is currently difficult to work remotely. Our goal is to expand the concept of telepresence technology and promote its research and development to broaden the possibilities of remote work and individuals’ capabilities.
Activities
* A list of publications can be found on this page.