For an essential worker to perform their tasks by means of telepresence, it is necessary to reproduce the worker's actions using an artificial body or robot. Robots have been used for relatively simple manual tasks such as restocking shelves in stores and replacing local area network cables in datacenters. However, more complex jobs such as nursing and caregiving involve many skills with embodied-tacit knowledge that must be acquired in the field and are difficult to perform with robots. These skills, which can be considered embodied knowledge, cannot be reproduced by artificial bodies or robots because there are no clear requirements for handling them digitally, making it difficult to use them remotely. To reproduce the skills required for jobs such as nursing and caregiving, which are complicated even for people, the challenge is how to express these skills digitally with artificial bodies and robots.
To digitally represent the skills demonstrated in high-level professional work, it is necessary to clarify and systematize the physical behaviors that are based on the expertise required of these skills. For example, when treating a patient with asthma in the hospital, the healthcare worker has the situational knowledge that the patient has asthma and the occupational knowledge that the patient’s breathing should be checked with a stethoscope. Thus, the healthcare worker performs the physical action of using a stethoscope to check the patient’s breathing. By systematically organizing the embodied knowledge that is needed in different environments and situations and what physical behavior needs to be expressed based on this knowledge, it is possible to define how artificial bodies and robots should function in each environment and situation.
In our effort to systematize physical behavior, we are conducting research and development on how to systematically define the physical behavior information needed for the movement of artificial bodies and robots and how to acquire and express this information. Some of the physical behaviors that are targeted by physical-behavior information can be expressed as words in the form of instruction manuals, while others can only exist as embodied-tacit knowledge within a particular field. There may also be discrepancies between the actions described in a manual and those that take place in the field. Our goal is to reproduce the skills of physical knowledge in artificial bodies and robots by clarifying how these physical behaviors are performed in practice and combining them with expert knowledge so that they can be handled digitally. We have developed a technique for understanding the physical behavior of customers and employees in a convenience store with finer granularity than could be achieved from manuals. In addition, we constructed a dataset for recognizing the behavior of employees working in high places in the construction of telecommunications infrastructure from a first-person perspective and evaluating whether the workers were adhering to the safety rules defined in manuals.
Our next steps are to apply the techniques for understanding the physical behavior of customers and employees in a convenience store and construction workers working in high places, investigate methods of physical behavior recognition for understanding the actual situation in the field, and establish methods for linking specialized knowledge and physical behavior.
* A list of publications can be found on this page.