[Ambitious Graduate Students] Seeking Depth in a Semi-transparent World:Developing Techniques to Improve Perceived Depth in 3D Fluoroscopic Visualization
Tangible cultural heritage is never permanent, and this is something that many likely realized when they saw the fires at Notre Dame Cathedral and Shuri Castle
Amid this backdrop, digital archiving, or using digital technologies to preserve tangible cultural heritage, is becoming increasingly important. Creating a detailed archive of a structure is essential for its restoration, and even if it cannot be restored, the archive will play an invaluable role in keeping it etched in people's memories and keeping a record for history.
At Ritsumeikan University, a researcher has succeeded in semi-transparently visualizing internal structures. He developed the world's first 3D fluoroscopic visualization technology that allows you to see both the interior as and exterior structures at a glance.
Daimon Aoi (2nd year doctoral student, Graduate School of Information Science and Technology) is further advancing this innovative technology to develop a technique that enables people to recognize semi-transparent three-dimensional images more accurately. We spoke with Aoi to learn more about his cutting-edge research that aims to preserve tangible cultural heritage for the future.
Table of Contents
What is 3D fluoroscopic visualization?
In the past, computer graphics (CG) of 3D measurements often resulted in opaque visualizations that did not simultaneously reveal the internal and external structures of an object.
The 3D fluoroscopic visualization technology that Aoi is working on is a cutting-edge technology that allows the viewer to see both the external and internal structures and intuitively grasp all three dimensions because the entire structure is depicted semi-transparently.
In addition, while the smallest unit of conventional computer graphics technology is a plane, such as a triangle or a rectangle, 3D fluoroscopic visualization technology enables delicate expressions using points that can depict more precise images.
* For a detailed report on 3D fluoroscopic visualization technology, please read the shiRUto article “The World’s First 3D Fluoroscopic Technology: Taking the Digital Archiving of Cultural Heritage to the Next Level.”. The video below shows the Hachiman-yama Gion festival float that was reproduced using 3D fluoroscopic visualization.
Aoi is trying to further develop the world's first 3D fluoroscopic visualization technology. With this research, he aims to create "visual guides" that enable people to accurately perceive depth, something that has been difficult to express in semi-transparent 3D images until now.
Accurate depth reproduction
When internal structures are visualized semi-transparently and three-dimensionally using 3D visualization, accurate depth can be lost. Three-dimensional visualization sometimes results in the viewer perceiving depth that is not as deep as the actual depth (“Conventional method” in the photo).
“While I expect that this 3D fluoroscopic visualization technology will be used for fine manipulation and safety-critical sites, there are still issues that need to be solved with regard to expressing complex internal three-dimensional structures as semi-transparent, stereoscopic images. If depth and other distances can be accurately perceived, we can expect to see applications in a variety of fields," says Aoi.
Currently, Aoi is working on creating visual guides that enable people to accurately perceive depth by repeatedly applying his technology to the digital archiving of tangible cultural heritage.
“I am developing a technique to automatically extract the elements that make up the edges, such as corners and lines, from the original 3D data and draw them as solid lines in order to correct perception, thereby enabling a more accurate understanding of depth (‘Proposed method’ in the photo below). Using techniques from psychophysical experiments, we are now starting to demonstrate that depth can be perceived more accurately with the addition of visual guides," explains Aoi, who feels encouraged by his research findings.
Going forward, Aoi's goal is to develop a technology that draws and emphasizes edges with dashed lines to ensure that viewers can accurately and easily perceive depth. In some cases, when the edges of a semi-transparent image are emphasized with solid lines, the lines in the foreground hide those in the background, so Aoi is motivated to figure out how to draw dashed lines that enable viewers to accurately and easily perceive depth.
The strength of Aoi’s research lies in the process of how he creates these visual guides. He is attempting to shed light on the mechanisms of human depth perception by conducting psychophysical experiments.
“Human” perception is the key
Research in the field of CG and visualization is dominated by computer-aided CG and visualization techniques and methods, with few attempts to utilize psychophysical experiments. Aoi describes the significance of conducting psychophysical experiments as follows:
“In order to yield semi-transparent 3D images that can be practically applied in real-world situations, it is very important to understand if humans are perceiving the images accurately. Therefore, I believe that by comparing and verifying the depth distance expressed by semi-transparent 3D images and the distance perceived by humans, we can achieve more accurate and practical semi-transparent 3D images."
For Aoi, however, there seems to be a constant struggle not only to create computer-generated images and simulations, but also to gather subjects for experiments.
“While you can proceed with image data creation at your own pace, you cannot do this with psychophysical experiments on human subjects. You have to coordinate many things with the subjects, and there is quite a lot of work because the experiments must be conducted safely while also taking infection control measures. Nevertheless, I believe that by conducting psychophysical experiments, we can produce semi-transparent 3D images with a high degree of accuracy and utility, which allows us to have more in-depth discussions and more granular research findings than was previously possible," explained Aoi.
Don't expect perfection and value feedback from others
The extremely busy Aoi says that what is important to him is not to spend too much time on his work.
“You have to work efficiently and within a limited time frame when you are doing so many things, including creating data for experiments, coordinating with the subjects, conducting analyses, and generating output. When I was doing my master's, I was thoroughly engaged in adjusting experimental data until I was satisfied with the results. However, now I try not to spend too much time on this and instead aim to perform as well as I can within the given time frame. Also, when you are working by yourself, it is easy to lose flexibility in your thinking, so I frequently seek advice from my professors to keep my research on track,” says Aoi.
Applying research findings to society
Ritsumeikan University selects promising, highly motivated doctoral program students to serve as Ritsumeikan University NEXT Student Fellows, and it is implementing a program to help these fellows deepen their expertise and acquire a broad range of research perspectives while working with talented researchers who are conducting cutting-edge research in a diverse array of fields. Aoi is one of those doctoral students who was chosen for the Ritsumeikan University NEXT Student Fellowship Program. After completing his doctoral program, he says he would like to work as a researcher in a company, making use of the knowledge of computer graphics, visualization techniques, and virtual reality that he has learned in graduate school. He wants to apply the technologies he developed through his research to actual products and send them out into the world.
“Since I plan to work in the private sector, it is very important that I find a company that matches my skillset. I am very grateful to have been selected for the fellowship program and have received so much support from the university. Just the other day, I was referred to a company, so I plan to go to Niigata soon for a matching event with that company," says Aoi.
Three-dimensional fluoroscopic visualization technology enables the visualization of internal structures that are invisible to the naked eye. By using psychophysics to accurately reproduce the depth that humans can perceive, Aoi seeks to further enhance the practical utility of the technology. When he puts this innovative technology into practice in the real world, he will surely go beyond the digital archiving of tangible cultural heritage and find ways to enrich society in various fields, including medicine and architecture. While we wait for such a future, let’s keep an eye on what he does next.
Related information
- [Graduate Students Stepping up to the Challenge] The Science of Acupuncture:Building up Evidence and Aiming for New Horizons in Acupuncture Treatment
- Shedding Light on the Unknown Reality of Mid-Edo Tenmei Kabuki—Using Digital Archives to Conduct an Integrated Analysis of Ukiyo-e Prints from All Over the World