Research AreaⅣ Promoting the health and pursuit of Ikigai (a reason for living) for the elderlySupport for the Maintenance of Cognitive Communication Skills of the Elderly

Integrating cognitive science and information science to open up next-generation research on cognitive communication support

Leaders

• 
  Project leader

  Yugo Hayashi

  Professor, Department of Comprehensive Psychology, College of Comprehensive Psychology

  Profile
• 
  Group leader

  Asako Kimura

  Professor, Department of Information Science and Engineering, College of Information Science and Engineering

  Profile
• 
  Group leader

  Tomoko Izumi

  Professor, Department of Information Science and Engineering, College of Information Science and Engineering

  Profile

A new research area for realizing sustained cognitive support for the elderly through attentive listening

As population aging and birth rate decline continue in Japan, the elderly have increasingly fewer opportunities to interact with younger people. The COVID-19 pandemic has made it even more difficult for the elderly to have interpersonal contact and receive assistance. In Japan, it is said that approximately 4.62 million people have dementia, most of whom are elderly, and another 4 million people suffer from mild cognitive impairment. To prevent dementia, several therapeutic methods are being studied and proposed, including reminiscence therapy, exercise therapy, and cognitive stimulation therapy, in all of which a human presence is essential in the role of an advisor or nursing care provider monitoring an aged person’s activities or supporting the adequate administration of necessary treatment.

Among various ways of assisting the elderly, we are focusing our attention on support through attentive listening, which is found to be particularly effective in the form of reminiscence therapy. It involves having an elderly person talk about happy memories from his or her past and pleasant episodes in his or her current daily life. It has been learned that the verbalization of ideas and feelings by the elderly can be greatly enhanced if they are provided with visual information, including information emitted by smartphone, and other types of information that stimulate their olfactory and tactile senses, that is, providing multimodal information as stimuli, activating memory in high-order cerebral and cognitive activities.

However, there is considerable difficulty today in implementing projects for assisting the elderly in maintaining their cognitive functions and mental health, given the ongoing situation wherein direct interpersonal communication is restricted or difficult, coupled with the shortage of personnel capable of providing such care and opportunities for training in attentive listening for elderly care recipients. Therefore, in this research project, we attempt to open up a new area of research focusing on realizing durable cognitive support through active listening to the elderly with the use of information and communications technologies (ICTs) toward the ultimate goal of maintenance of the mental health of the elderly.

• 

  Results envisaged in the project
  Development of systems that induce memory recall and verification of their effectiveness on cognitive functions

Developing a system for dialogue support and health enhancement equipped with a cognitive model based on human analysis

To provide the elderly with appropriate and stable support in a manner that follows the human mental mechanism, it is necessary to construct a model to understand the state of health of the elderly based on empirical evidence and, based on this model, an information system. To do so, we take an interdisciplinary approach in this project, combining a model-based approach with a system-based approach that develops assistive technologies based on the knowledge provided by the model.

Firstly, the Hayashi Group focuses its research on the exploration of the human mental mechanism for the purpose of developing assistive technologies. The group pursues this research from a cognitive scientific approach in two teams. One team focusing on modeling conducts analysis to make it possible to understand the mental and physical health mechanisms of the elderly via attentive listening-centered dialogue. The other team focusing on assistance examines facilitation methods to get the elderly to feel better and activate their recollection and verbal description during active listening.

The modeling team conducting analysis for understanding mental and physical health mechanisms has been further divided into two units. In the psychological analysis unit, researchers in psychology and educational psychology are constructing a psychological model based on verbal information. The sensing unit, mainly comprising information engineers, is working on analysis and model development related to the detection of states based on non-verbal information. The group is working toward realizing accurate, system-based prediction of health condition through machine learning backed by engineering-based sensing technologies, in addition to psychological experiment-based analysis.

The assistance team, aiming to propose technologies that support human cognitive activities and interpersonal communication by attentive listening, has also been divided into two units. The communication support unit, mainly comprising cognitive scientists, postdoctoral research fellows, and doctoral students, conducts analysis and model development of activities related to communication support, such as memory recall inducement and verbal description during attentive listening. For example, the unit has already experimented with a method it designed, involving the use of an eye movement measuring device and a conversation agent that indicate where one’s interlocutor is looking during conversation. The results revealed that this system, when used by a pair of students working on a problem together, facilitated their communication and enhanced the ease of problem solving. The other unit, mainly comprising researchers specializing in virtual reality, works on system support. This unit is charged with the development of an experimental system that emits stimuli to assist communication with the use of virtual reality (VR) and mixed reality (MR).

The Kimura Group is working toward the goal of cognitive function activation by introducing body movements and a variety of sensations into interactive activities that elderly persons perform in VR and MR environments.

Within the Kimura Group, Team 1 is developing interactions that require body movements, such as hand/arm, foot/leg, and eye movements, while Team 2 is working to identify, among various interaction methods involving foot movements for moving around in a VR space, ones that are easier for the elderly to use and are more effective in cognitive activation. Team 3’s research is focused on interaction methods that fuse multiple senses so that they can complement one another. In the MR environment, CG images can be superimposed on physical objects to make them appear different in virtual reality. This technology can make, for example, a small box you are carrying in your hand look much larger when viewed through HMDs. This difference in appearance between real and virtual spaces is known to provoke a tactile sensation not corresponding to physical reality, such as the perceived weight of the object. Such inter-sensory illusion can be used to design and examine interaction methods in which certain sensations that the elderly tend to have difficulty feeling are amplified or rendered more perceptible through other senses. Team 4 will then verify the effects of the interaction methods developed and examined by Teams 1-3 on the cognitive functions of the elderly from a cognitive psychological perspective.

The Izumi Group focuses on people’s memories. Divided into two units, the group studies the characteristics of digital data such as photographs, video images, and sounds to examine which ones are effective in inducing recollection of memories. The group also intends to design methods for storing memories correctly as data, propose interactive information systems that provide such data, and verify the effectiveness of memory recall aided by the proposed systems on cognitive function activation.

Assoc. Prof. Izumi has already conducted research for the purpose of realizing engineering-based application of data related to past memories, proposing thus far a communication support system using memory information for interaction between dementia patients and their family care providers. In the current project, she is investigating whether or not personal memories can be recalled when presented with photographic or textual data of other people, such as those uploaded in large quantities on social media. Analysis is also conducted to see if there is any difference between photographic and textual data in the content they recall and to discern what elements of each data set are useful for memory recall. With regard to past memories, including those from times when digital recording devices were not available, the group will conduct a study to see whether or not currently available data of images, music and so on can help them recall past memories.

Becoming one of the world’s best research centers specializing in cognitive communication support

In the long term, we hope to develop a robot capable of intelligent behavior and dialogue by applying information and communication systems based on cognitive scientific models, and have such a product used at welfare facilities for the elderly and generally in society.

Toward this goal, we are also focusing our efforts on training researchers with originality and strengthening the capabilities of the Research Center for Cognitive Sciences, Ritsumeikan University, so as to establish it as a research center for cognitive communication support capable of disseminating cutting-edge research findings. We also hope that the results of this project will allow us to strengthen our partnerships with external industrial, educational, and welfare organizations interested in practical application.

• 

  A scene from a basic psychological experiment that analyzes conversation of a pair of students; here, the speakers’ eye movements and facial expressions and data on utterances are collected to analyze their pertinence to the research themes. In the future, similar experiments will be conducted by using VR or including elderly persons as participants.