Cybernetics

Technology

Human Information Laboratories aim to create user experiences that take advantage of the unique characteristics of remote work. We are researching technologies that extend human capabilities to enhance the remote work experience. We aim to connect real space with cyberspace and create new user experiences through the fusion of real and cyber. We are focusing on remote education for playing musical instruments and remote education for sports. Instruction has generally been conducted in the real world, where skills and physical techniques are transmitted directly from person to person. We extend human capabilities by making it possible to transfer skills independent of time and place.

We are researching two technologies in this area. First, we are collecting objectively observable information such as surface electromyography and electroencephalography to digitize the skills of professional athletes and craftspeople. Based on this information, we are studying "Motor-skill-transfer technology," in which electrical stimulation is applied to muscles and transmitted directly to the person; in the second, we are studying "Embodied-knowledge technology," which extracts characteristic physical movements to digitize the person's senses.

Our Human Information Laboratories aim to create user experiences that take advantage of the unique characteristics of remote work. We are researching technologies that extend human capabilities to enhance the remote work experience. In addition, we aim to connect real space with cyberspace to create new user experiences, focusing on remote education for musical instruments and sports. Instruction is generally conducted in the real world, where skills and physical techniques are taught directly from person to person. Our goal is to extend human capabilities by making it possible to transfer skills regardless of time and place.

Toward this end, we are researching methods of transferring motor skills and embodied knowledge. For the former, we are collecting objectively observable information using techniques such as surface electromyography and electroencephalography to digitize the skills of professional athletes and experts. On the basis of this information, we are investigating the application of electrical stimulation to the muscles as a technology for motor skill transfer. For embodied-knowledge technology, we are studying ways of extracting characteristics of physical movements to digitize a person's sensations.



Research

  1. Motor-skill-transfer technology
  2. Embodied-knowledge technology

Publications

2025

2024

2023

2022

2021

2025

Journal Papers

-Embodied-knowledge technology-

  1. Mitsuhiro Goto, Kenji Esaki, Teruhisa Inoue, Hitoshi Seshimo, and Takaya Hirano (Research Institute for Lifelong Sports and Social Creation), "An evaluation of a remote training system with bodily reliving experience for windsurfing", Sports Engineering, vol. 28, no.1, pp.1-10, Apr. 2025

Conference Papers

-Motor-skill-transfer technology-

  1. Arinobu Niijima, Masato Shindo, and Ryosuke Aoki. 2025. Invisible Light Touch: Standing Balance Improvement by Mid-Air Haptic Feedback. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25). Association for Computing Machinery, New York, NY, USA, Article 502, 1–11. https://doi.org/10.1145/3706598.3713396
  2. Arinobu Niijima and Shoichiro Takeda. 2025. Improving Putting Accuracy with Electrical Muscle Stimulation Feedback Guided by Muscle Synergy Analysis. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25). Association for Computing Machinery, New York, NY, USA, Article 362, 1–11. https://doi.org/10.1145/3706598.3713605
  3. Teodora Mitrevska, Benjamin Tag, Monica Perusquia-Hernandez, Arinobu Niijima, Ludwig Sidenmark, Erin Solovey, Abdallah El Ali, Sven Mayer, and Francesco Chiossi. 2025. SIG PhysioCHI: Human-Centered Physiological Computing in Practice. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '25). Association for Computing Machinery, New York, NY, USA, Article 862, 1–5. https://doi.org/10.1145/3706599.3716289
  4. Masato Shindo and Ryosuke Aoki, “Preliminary Investigation of Clustering Analysis for Understanding Individual Sensory Contributions in Human Postural Control,” 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Copenhagen, Denmark, Jan. 2025
  5. Ryosuke Aoki and Masato Shindo, “Preliminary Investigation on Vertical Ankle Sway in Static Standing Under Eyes-Closed and Foam Pad Conditions,” 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Copenhagen, Denmark, Jan. 2025

-Embodied-knowledge technology-

  1. Shoichiro Takeda and Yasunori Akagi, "Gromov–Wasserstein Problem with Cyclic Symmetry", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

2024

Journal Papers

-Embodied-knowledge technology-

  1. Takasuke Nagai, Shoichiro Takeda, Satoshi Suzuki, Hitoshi Seshimo, ”MMW-AQA: Multimodal In-the-Wild Dataset for Action Quality Assessment”, IEEE Access, Vol.12, pp.92062-92072, 2024.

Conference Papers

-Motor-skill-transfer technology-

  1. M. Shindo, Y. Kitamura (Kanagawa Prefectural Yokosuka High School), K. Wada (Kanagawa Prefectural Yokosuka High School), H. Aoki (Kanagawa Prefectural Yokosuka High School), and R. Aoki, "Investigation on Effects of Load Distribution of Upper and Lower Limbs on the “Set” Position Posture in the Sprint Start," 15th International Conference on the Engineering of Sport (ISEA2024), UK, 2024.
  2. Takashi Isezaki, Michiaki Suzuki (Tokyo Metropolitan Institute of Medical Science), Kengo Okitsu, Yukio Koike, Yukio Nishimura (Tokyo Metropolitan Institute of Medical Science). 2024. Muscle Synergy Analysis Under the Constraint of Connectivity Between Brain and Muscle Activity. (Accepted) 2024 International Joint Conference on Neural Networks (IJCNN), Yokohama, Japan.
  3. Masato Shindo and Ryosuke Aoki. 2024. Exploring Potential of Electromyography-based Avatar Operation Using Residual Muscles of ALS Individuals: Case Study on Avatar DJ Performance. In Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems (CHI EA '24). Association for Computing Machinery, New York, NY, USA, Article 146, 1–7. https://doi.org/10.1145/3613905.3650857
  4. Kohei Umezawa, Takashi Isezaki, Kengo Okitsu, Osamu Yokoyama (Tokyo Metropolitan Institute of Medical Science), Michiaki Suzuki (Tokyo Metropolitan Institute of Medical Science), Yukio Nishimura (Tokyo Metropolitan Institute of Medical Science). Refined Force Estimation in Monkey's Pinching Tasks Through Integrated EMG and ECoG Data: A Kalman Filter Method. In: 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2024
  5. Takuya Kanda, Takashi Isezaki and Kengo Okitsu. 2024. A Study on Changes in Estimation Accuracy for EEG Data During Calibration and Operation in MI-BCI. In: 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2024; Orlando, Florida, USA.
  6. Kengo Okitsu, Takashi Isezaki, Kei Obara, Yukio Nishimura, Enhancing Brain Machine Interface Decoding Accuracy through Domain Knowledge Integration. In: 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2024
  7. Takashi Isezaki, Takuya Kanda, Kengo Okitsu, ”Possibility of Motion Estimation Using Dry Electroencephalography”, NEURO2024 LBA, Jul. 2024
  8. M. Shindo and R. Aoki, "Immediate After-Effects of Maintaining Light Touch using Electrical Muscle Stimulation to Index Finger Extensor Muscles on Postural Control *," 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Kuching, Malaysia, 2024, pp. 3467-3472, doi: 10.1109/SMC54092.2024.10831074.
  9. Arinobu Niijima, Yukio Koike, and Hironori Ishikawa. 2024. Motor-Skill-Download System Using Electrical Muscle Stimulation for Enhancing Piano Playing. In Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '24). Association for Computing Machinery, New York, NY, USA, 313–317. https://doi.org/10.1145/3675094.3681942
  10. Arinobu Niijima, Masato Shindo, and Ryosuke Aoki. 2024. Improving Standing Balance with Mid-Air Haptic Feedback. In SIGGRAPH Asia 2024 Emerging Technologies (SA '24). Association for Computing Machinery, New York, NY, USA, Article 8, 1–2. https://doi.org/10.1145/3681755.3688947
  11. Yota Takahara, Arinobu Niijima, Chanho Park, and Takefumi Ogawa. 2024. Minimizing Errors in Eyes-Free Target Acquisition in Virtual Reality through Auditory Feedback. In Proceedings of the 2024 ACM Symposium on Spatial User Interaction (SUI '24). Association for Computing Machinery, New York, NY, USA, Article 47, 1–2. https://doi.org/10.1145/3677386.3688884
  12. Yuki Kubo and Buntarou Shizuki. 2024. Signal2Hand: Sensor Modality Translation from Body-Worn Sensor Signals to Hand-Depth Images. In SIGGRAPH Asia 2024 Posters (SA '24). Association for Computing Machinery, New York, NY, USA, Article 16, 1–2. https://doi.org/10.1145/3681756.3697943
  13. Hana Mizukami, Arinobu Niijima, Chanho Park, and Takefumi Ogawa. 2024. Exploring the Impact of Size and Position on Visual Feedback Efficacy for Ballet Dancers. In Companion Proceedings of the 2024 Conference on Interactive Surfaces and Spaces (ISS Companion '24). Association for Computing Machinery, New York, NY, USA, 54–57. https://doi.org/10.1145/3696762.3698052

-Embodied-knowledge technology-

  1. Shoichiro Takeda, Yasunori Akagi, Naoki Marumo, Kenta Niwa, "Optimal Transport with Cyclic Symmetry", The Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24), 2024, pp.15211-15221.
  2. Mitsuhiro Goto, Kenji Esaki, Teruhisa Inoue, Hitoshi Seshimo, Takaya Hirano, "Bodily Sensory Reproduction Player for Windsurfing Remote Training", 15th International Conference on the Engineering of Sport (ISEA2024), UK, 2024.

2023

Journal Papers

-Motor-skill-transfer technology-

  1. Arinobu Niijima, Mana Sasagawa, Texture Profile Analysis of Pseudo Food Texture Display by Jamming, Transactions of the Virtual Reality Society of Japan, 2023, Vol. 28, Issue 1, p. 3-10
  2. K. Eguchi and R. Aoki, "Practical R-R Interval Editing for Heart Rate Variability Analysis Using Single-Channel Wearable ECG Devices," in IEEE Access, vol. 11, pp. 25543-25582, 2023, doi: 10.1109/ACCESS.2023.3253933.
  3. Shindo M, Isezaki T, Koike Y, Aoki R (2023) Induced effects of electrical muscle stimulation and visual stimulation on visual sensory reweighting dynamics during standing on a balance board. PLoS ONE 18(5): e0285831. https://doi.org/10.1371/journal.pone.0285831

Conference Papers

-Motor-skill-transfer technology-

  1. M. Shindo, T. Isezaki and R. Aoki, "Electrical Stimulation Intensity to Induce Sensory Reweighting Dynamics While Standing on Balance Board," 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Sydney, Australia, 2023, pp. 1-4, doi: 10.1109/EMBC40787.2023.10340950.
  2. Y. Ogata (Kanagawa Prefectural Yokosuka High School), M. Kobayashi　(Kanagawa Prefectural Yokosuka High School), T. Kageyama　(Kanagawa Prefectural Yokosuka High School), M. Shindo, and R. Aoki, "Study on Front Crawl Form Analysis Using a Dry-Land Training Tool and a Motion Capture System," 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Sydney, Australia, 2023 (one-page paper).
  3. Shiina Takano and Arinobu Niijima. 2023. Game with Breathing Control System for Learning Circular Breathing. In Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY Companion '23). Association for Computing Machinery, New York, NY, USA, 254–258. https://doi.org/10.1145/3573382.3616042
  4. Arinobu Niijima, Ryosuke Aoki, Yukio Koike, and Shinji Miyahara. 2023. Motor-Skill-Transfer Technology for Piano Playing with Electrical Muscle Stimulation. In SIGGRAPH Asia 2023 Emerging Technologies (SA '23). Association for Computing Machinery, New York, NY, USA, Article 14, 1–2. https://doi.org/10.1145/3610541.3614570
  5. Arinobu Niijima. 2023. Estimating Attention Allocation by Electrodermal Activity. In Proceedings of the 2023 ACM International Symposium on Wearable Computers (ISWC '23). Association for Computing Machinery, New York, NY, USA, 87–91. https://doi.org/10.1145/3594738.3611370
  6. Arinobu Niijima. 2023. Learning Effects and Retention of Electrical Muscle Stimulation in Piano Playing. In Proceedings of the 2023 ACM International Symposium on Wearable Computers (ISWC '23). Association for Computing Machinery, New York, NY, USA, 104–108. https://doi.org/10.1145/3594738.3611373
  7. Arinobu Niijima and Yuki Kubo. 2023. Assisting with Fingertip Force Control by Active Bio-Acoustic Sensing and Electrical Muscle Stimulation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 743, 1–13. https://doi.org/10.1145/3544548.3581192
  8. Masato Shindo and Arinobu Niijima. 2023. Standing Balance Improved by Electrical Muscle Stimulation to Popliteus Muscles. In Proceedings of the Augmented Humans International Conference 2023 (AHs '23). Association for Computing Machinery, New York, NY, USA, 27–34. https://doi.org/10.1145/3582700.3582711

2022

Journal Papers

-Motor-skill-transfer technology-

  1. Arinobu Niijima, Ryosuke Aoki, Diminished Grip Force from Superimposed Electrical Stimulation on Muscle Coordination, Transactions of the Virtual Reality Society of Japan, 2022, Vol. 27, Issue 3, p. 167-172

Conference Papers

-Motor-skill-transfer technology-

  1. Arinobu Niijima, Toki Takeda, Ryosuke Aoki, and Shinji Miyahara. 2022. Muscle Synergies Learning with Electrical Muscle Stimulation for Playing the Piano. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 54, 1-10. https://doi.org/10.1145/3526113.3545666
  2. Shiina Takano and Arinobu Niijima. 2022. Involuntary Exhalation Control by Facial Vibration. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST ’22 Adjunct). Association for Computing Machinery, New York, NY, USA, Article 7, 1-3.https://doi.org/10.1145/3526114.3558693
  3. Yuki Kubo. 2022. Ring-type Indirect Pointing Device for Large Displays using Three-axis Pressure Sensor. In Proceedings of the 2022 ACM Symposium on Spatial User Interaction (SUI ’22). Association for Computing Machinery, New York, NY, USA, Article 33, 1-2. https://doi.org/10.1145/3565970.3568185

-Embodied-knowledge technology-

  1. Shoichiro Takeda, Kenta Niwa, Mariko Isogawa, Shinya Shimizu, Kazuki Okami, Yushi Aono, Bilateral Video Magnification Filter, The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 17369-17378

Awards

  1. Best Poster Award, Yuki Kubo. 2022. Ring-type Indirect Pointing Device for Large Displays using Three-axis Pressure Sensor. In Proceedings of the 2022 ACM Symposium on Spatial User Interaction (SUI ’22). Association for Computing Machinery, New York, NY, USA, Article 33, 1–2. https://doi.org/10.1145/3565970.3568185

2021

Journal Papers

-Motor-skill-transfer technology-

  1. Arinobu Niijima, Toki Takeda, Kentaro Tanaka, Ryosuke Aoki, and Yukio Koike. 2021. Reducing Muscle Activity when Playing Tremolo by Using Electrical Muscle Stimulation to Learn Efficient Motor Skills. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 3, Article 123 (Sept 2021), 17 pages. https://doi.org/10.1145/3478110

Conference Papers

-Motor-skill-transfer technology-

  1. A. Niijima, “Posture Feedback System with Wearable Speaker,” 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico, 2021, pp. 7007-7010, doi: 10.1109/EMBC46164.2021.9630687.
  2. T. Isezaki, R. Aoki and Y. Koike, “Correction of Electrodes ID Configuration based on the Distribution of Surface EMG Features,” 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico, 2021, pp. 7083-7086, doi: 10.1109/EMBC46164.2021.9629817.
  3. M. Shindo, T. Isezaki, R. Aoki and Y. Koike, “Force Control on Fingertip Using EMS to Maintain Light Touch,” 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico, 2021, pp. 4641-4644, doi: 10.1109/EMBC46164.2021.9630237.
  4. Arinobu Niijima and Yuki Kubo. 2021. Assisting with Voluntary Pinch Force Control by Using Electrical Muscle Stimulation and Active Bio-Acoustic Sensing. In Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology (UIST ’21 Adjunct). Association for Computing Machinery, New York, NY, USA, 11-13. https://doi.org/10.1145/3474349.3480214