JST CREST Materialized Graphics Project (2018-2024)
Materialized Graphics refers to 3D graphics that can physically interact with humans and their environment. While sensing human movement to update the visuals, the system simultaneously modulates force and temperature distributions on the skin to create a physical perception of the object’s presence.Such physical interactions can be generated contact-free using Midair haptics, specifically through ultrasound technology. In the JST CREST project “Materialized Graphics for Multidimensional Interaction” in the Symbiotic Interaction area, we aim to realize this type of computer graphics and propose novel interaction systems based on it.
A key piece of materialized graphics is high-quality 3D CG visible by naked eyes. Kakeya group at Univ. of Tsukuba develops a device to create a high-resolution 3D image without vergence-accommodation conflict. The method enables a large and practical 3D display causing less eyestrain. We integrate the device with haptics.
Machine learning is a useful tool to understand interaction and simultaneously, to generate haptic signals in realtime interaction. The top researchers develops the interaction algorithm of materialized graphics and provides mathematical tools to accelerate the interaction research.
Materialized Graphics enriches Symbiotic Interaction
The core of the project is to achieve literal materialized graphics by integrating haptics, 3D CG, and machine learning. It opens up a new field of VR and computer graphics towards various applications. Our final goal is to understand the human interaction and develop human-support systems using such a powerful tool. Materialized graphics provides a new channel of computer-human interface and human-human communication. Advanced haptic stimulation can guide human motion and control the biological and mental state to improve the human behavior.
