What kind of research are you currently working on?
Morita : As an expert in perceptual psychology, until now, I have been researching topics such as depth perception, in which two-dimensional objects such as paintings and photographs are perceived as three-dimensional. My interests have expanded to perceptual information processing other than vision, and I am currently researching “co-embodiment” in virtual spaces.
Virtual co-embodiment refers to a system in which multiple people collaboratively manipulate a single avatar in a virtual space. When two people perform a body movement at the same time, the average of both movements is reflected in the avatar.
Prior research has reported that when two people share a single-body avatar in a virtual space, they can move the avatar’s body more efficiently than when they move the body by themselves. However, it was unclear what the exact factors were that affected movement performance when users embodied an avatar. Therefore, my research partners and I focused on the interactions between two users manipulating an avatar together for verification of their effectiveness.
What kind of experiments did you conduct?
Morita : In our experiment, we asked two participants to perform a reaching task in which they extended their arms toward a target displayed in a virtual space and touched it with the avatar’s fingers. We set the following three conditions for this experiment and then compared them: (1) Solo condition, where only the participant’s own movements were 100% reflected in the avatar’s movements without any co-embodiment with other participants; (2) Real-time condition, where participants controlled an avatar that combined their hand movements with another person’s hand movements in real-time at 50% each; and (3) Pre-recording (pre-rec) condition, where participants controlled an avatar that combined their own hand movements with pre-recorded hand movements from another person at 50% each. The difference is that under the real-time condition, the two participants can adjust their respective movements in response to the avatar’s movements, whereas under the pre-rec condition, the other participant does not make any adjustments.
Experiments conducted under these three conditions showed that the straightness of the reaching trajectory was significantly better under the real-time condition than under the pre-rec condition, and that the time required for reaching was also shorter. These findings suggest that the participants manipulating the avatars are adjusting their own movements to match those of their counterparts, resulting in more efficient movement when combined.
When the participants were asked about their sense of agency and sense of body ownership under each condition, the results were lowest for the real-time condition. In other words, even though participants did not subjectively feel that they were controlling the avatars by themselves, they were still able to move efficiently when using the co-embodied avatar.
Subsequent studies have provided stronger support for our previous findings. As before, we conducted experiments with co-embodied avatars under the same three conditions. This time, however, we added a twist when notifying participants about the conditions in advance. One pair of participants was notified of the correct information, while another pair was provided with a condition that was different from the actual condition—namely, they were given the pre-rec condition when participating under the real-time condition and the real-time condition when participating under the pre-rec condition.
The findings showed that for both sets of participants, the efficiency of movement was higher when tasks were performed under the real-time condition. In other words, we showed that regardless of prior knowledge, participants were adjusting their own movements in response to the other person’s movements reflected in the avatar’s arm.
Can you tell us about your latest research?
Morita : Although I found that movement efficiency improves when co-embodied avatars are used, I was left with a new question: Is this because the body (avatar) is being shared with someone else, or because the movement is being shared? Therefore, in a more recent study, my research partners and I compared the respective effects of body sharing and movement sharing in an experiment using an avatar.
In a virtual space, we had two participants manipulate the left and right arms of an avatar to perform a lifting task where they lifted a stick with a ball on it. When they were given this task under the body sharing and movement sharing conditions, we found that achievement performance improved only under the movement sharing condition. From this, we speculate that movement sharing may be a factor in the improved movement performance that stems from co-embodiment.
Tell us about the research you plan to do going forward.
Morita : I would like to connect the findings from my research using the new technology of co-embodiment in virtual spaces to the quest for understanding others. In psychology, there is a wealth of research on self-understanding, which defines what the “self” is physically, behaviorally, socially, and psychologically, but defining what constitutes the “other” is a very difficult task.
In the real world, the body is a clear boundary that physically separates the self from others, and it plays an extremely important role in the recognition of others who are not oneself. By conducting research using virtual spaces, which enable co-embodiment, I would like to attempt to capture the essence of “otherness” through the lens of movement.