Balandino Di Donato: Using the XTH Sense to Create Virtual Water Games

The amazing thing about virtual environments is that with a bit of imagination and creativity, you are able to create your own world. For the past few months, I’ve been playing with the concept of emulating gestural interactions with objects from the real world and perceiving the same auditory feedback in the virtual realm, yet without the object itself. In a previous work, I used Integra Live and the Myo armband, to simulate the interaction of virtual piece of paper, which when crumbled gave the same auditory feedback of a real crumbled piece of paper. Inspired by this work, I continued exploring this direction during the XTH Sense Creative Lab. For this occasion, my goal was to emulate the auditory feedback generated by the interaction of a tea cup filled with water. In about two days I put together this demo below, pretty neat imho! A few words about my creation process. It was my first time using the XTH Sense, so I first had to experiment with the XTH Sense device to get familiar with its capabilities. After a couple of hours spent testing and observing the data variation in relationship to the hand and arm’s movements, I made few observations. First, the XTH Sense, which captures motion, direction and orientation sensors (integrated in a 9-DoF IMU) and muscle sound (also known as mechanomyogram or MMG), engages the user differently than other seemingly similar technologies such has EMG- and camera-based devices. This is due to the nature of the MMG signal itself. As we can see from the picture below, during a static contraction, i.e., when muscles are...

Pin It on Pinterest