September 25, 2023 TOPICS

[Ambitious Graduate Students] Attempting to build Doraemon:Using "common-sense knowledge" to improve the efficiency of robot learning costs

Shoichi Hasegawa, Graduate School of Information Science and Technology

From ChatGPT which has quickly penetrated every corner of society including business to cat-shaped robots that carry food in family restaurants... The ever-evolving expansion of AI and robot technology and research is evident even in casual, everyday settings. The growing field is the supported by the new ideas and accumulated research outcomes of individual researchers.

One such researcher, Shoichi Hasegawa, a second-year doctoral student in the Graduate School of Information Science and Technology at Ritsumeikan University, is striving to reduce the learning cost of household robots with a unique approach. Hasegawa pursues his research based on a childhood dream of wanting to create a robot that is like a friend. We asked him about his research, which has won awards at international symposiums and attracted attention from around the world, and what he finds appealing about the life of a researchers.

Using probabilistic logical inference to create a robot with common-sense knowledge

When we humans are placed in a new place or environment, we are able to understand the situation and respond to things to some extent based on our previous experiences. When recognizing a kitchen, even in a house that you are visiting for the first time, you can determine that "this is a kitchen" by looking at the objects there, such as the refrigerator, stove, frying pans, and so on. This kind of "common sense" use of knowledge is only possible because we are human, but it is a completely different story with robot learning.
Existing service robots that help with household chores can carry objects to a predefined position only after receiving a detailed description of “what is where”. Therefore, they must learn “what is where,” namely, the relationship between objects and places based on the definitions of what words—for example, “cup” or “plate”—refer to, using a large amount of data. This high learning cost is a big issue. A robot cannot understand a vague command like "bring me a cup” unless the location and other details are specified first; it must complete a series of tasks to be able to operate smoothly in a living space.
This led Hasegawa to focus on the concept of probabilistic logical inference, which is similar to human common sense. In his study, the results of a questionnaire regarding the location of objects were used to create knowledge similar to human common-sense knowledge. By combining this knowledge with the robot's own knowledge of visual and other information obtained in the experimental environment, Hasegawa created a system that could improve the efficiency of data learning and enable the prediction of the location of undefined objects.

“Suppose you tell a robot, ‘Bring me some tea.’ Using probabilistic logical inference, the robot determines that the word ‘tea’ is included in the category of ‘drinks,’ so there is a high probability that it will be in the same location as other drinks like ‘milk’ and ‘water’. The robot is able to think that it is highly likely that ‘tea’ will be in the ‘kitchen,’ the place where there is a high probability that other ‘drinks’ learned from the training data will also be present. With this study, we were able to process information based on word relationships and link it to probabilities to achieve advanced predictions.”

Experiments conducted in a simulated environment based on a virtual living space showed that learning cost could be reduced by a factor of 1.6 or more compared to existing methods. These groundbreaking research results were duly recognized with the Best Paper Award and the SICE International Young Authors Award at the 2023 IEEE/SICE International Symposium on System Integrations (SII2023). Hasegawa has since presented the results of an experiment conducted in a real environment, and he is in the process of conducting research for the practical application of this technique.

Using large language models to compensate for a small vocabulary

While producing outstanding results, Hasegawa is constantly reflecting on the problems with his own research, and he is always on the lookout for new research approaches. The issue with using probabilistic logical inference was that the vocabulary that the robot could use was limited. To solve this problem, Hasegawa focused on large language models, which use large amounts of text data to perform language processing tasks such as sentence generation, machine translation, and answering questions.

“Large language models are gaining attention as a tool for planning a robot's behavior. To execute an action plan to ‘carry an object,’ for example, flowcharts used to be designed by hand, following steps such as ‘move,’ ‘touch,’ and ‘grab,’ but this process limits the scope of what can be done in response to verbal commands. Therefore, I am attempting to have the robot write out actions it should take into a large language model by inputting information it automatically acquires in text format, like the names of objects and places, the locations of objects, the skills the robot possesses, and language commands. I have completed building this system and am now looking to move into the experimental phase.”

The robot in this study, which employs a large language model, has performed well and received high praise, including a second place finish in the @Home Domestic Standard Platform League Open Challenge at the RoboCup Japan Open 2023, and this study is expected to lead to the creation of more versatile service robots.

A life dedicated to robot research that started with a receptionist android!?

Hasegawa already has a wealth of awards and research achievements at the doctoral stage. He pours himself into his research every day, but what led him to become fascinated with robots was a human-like receptionist android that he saw at the Aichi Expo when he was in elementary school.

“I still remember how amazed I was. My anticipation for the future and interest in service robots increased dramatically after that. Then, I fell in love with Doraemon. As I read more and more of the series, I began to seriously think that I would like to create a robot that could understand human conversations, think for itself, and speak with emotion, just like a friend.”

With an overflowing desire to conduct research, Hasegawa entered the Department of Robotics in the College of Science and Engineering at Ritsumeikan University. While learning about controls and technology, he also gained experience designing robots, including cutting his own aluminum plates as a member of the Robotics Club. In order to get closer to realizing his dream, he entered the Graduate School of Information Science and Engineering where he could specialize in research on intelligent service robots.

“I applied to Professor Tadahiro Taniguchi's laboratory because I believed that utilizing higher-level knowledge, such as concepts that arise from composite data, in robots would lead to the development of systems that can process unknown information. The more I pursue my research, the more I feel that robots will become commonplace.”

Gaining knowledge and inspiration from the colleagues he met at conferences

Unlike their hectic undergraduate years, graduate students who set their own research topics have more time to carefully study prior research and related literature. Hasegawa says that because he is able to sit back, accumulate knowledge, and contemplate deeply, he is able to engage in intensive research. Another appealing feature of graduate school life is the opportunity to interact with and be stimulated by a wide variety of researchers because there are more opportunities to participate in academic conferences.

“When I attended an annual conference of the Japanese society for artificial intelligence, I met Dr. Masahiko Osawa, a well-known Doraemon researcher, and when I told him I was interested in his research, he invited me to a reception afterward. We exchanged ideas on the spot, and I am now participating in Dr. Osawa's research activity.”

Hasegawa was also chosen for the Ritsumeikan University NEXT Student Fellowship Program. This is a program that helps Fellows deepen their expertise and acquire a broad range of research perspectives while working with talented researchers who are conducting cutting-edge research in a diverse array of fields. According to Hasegawa, the program also offers many opportunities to interact with other Fellows in both the humanities and the sciences.

“When you advance to the doctoral level, it is easy to become immersed in your own research field and end up struggling on your own. By interacting with other fellowship students, I was exposed to research trends and perspectives in a variety of fields and was able to gain valuable experience. I felt reassured that I had made some research colleagues. The program is also appealing because I can get a bird's eye view of my own research by explaining it to others in a way that is easy to understand.”

Hasegawa seems to be leading a rewarding graduate school life, where he spends his time in friendly competition with researchers who are active at the vanguard of their respective fields.

The path of a researcher striving to fulfill a dream

In order to realize a dream he has had since elementary school, Hasegawa says he would like to pursue a career as a researcher.

“I would like to steadily accumulate outcomes so I can demonstrate the usefulness of the research I am currently working on. My immediate goal is to attract attention at international conferences and promote the widespread adoption of service robots. Above all, I still want to create Doraemon. I will continue my research in my current field, but I also plan to expand into areas such as human-robot interaction, which aims to realize human-like qualities in robots, so that I can take one step closer to realizing my dream.”

Although he is busy writing his dissertation, Hasegawa says with a smile that it is not unpleasant. Expectations are high for the yet-to-be-seen future he will create. Who knows? It may not be long before we see cat-shaped robots around the world.

Related information


September 20, 2023 NEWS

Understanding How Choice Overload in ChatGPT Recommendations Impacts Decision-Making