Blog

behind the scenes, lessons learned, places to be and cool hacks

Balandino Di Donato: Using the XTH Sense to Create Virtual Water Games

The amazing thing about virtual environments is that with a bit of imagination and creativity, you are able to create your own world. For the past few months, I’ve been playing with the concept of emulating gestural interactions with objects from the real world and perceiving the same auditory feedback in the virtual realm, yet without the object itself. In a previous work, I used Integra Live and the Myo armband, to simulate the interaction of virtual piece of paper, which when crumbled gave the same auditory feedback of a real crumbled piece of paper. Inspired by this work, I continued exploring this direction during the XTH Sense Creative Lab. For this occasion, my goal was to emulate the auditory feedback generated by the interaction of a tea cup filled with water. In about two days I put together this demo below, pretty neat imho! A few words about my creation process. It was my first time using the XTH Sense, so I first had to experiment with the XTH Sense device to get familiar with its capabilities. After a couple of hours spent testing and observing the data variation in relationship to the hand and arm’s movements, I made few observations. First, the XTH Sense, which captures motion, direction and orientation sensors (integrated in a 9-DoF IMU) and muscle sound (also known as mechanomyogram or MMG), engages the user differently than other seemingly similar technologies such has EMG- and camera-based devices. This is due to the nature of the MMG signal itself. As we can see from the picture below, during a static contraction, i.e., when muscles are...

Gender equity will shape the future of our bodies and technology

The statistics of women in technology as well as digital media related industries are shamefully low. The game industry alone only employs about 2.9 percent woman in leadership roles, and other sectors are not terribly dissimilar. Hardware start ups are by far the most notorious for being skewed in terms of gender ratios and outright hostility to women. To bring attention to gender inequity, numerous articles have surfaced to highlight that the percentage of computing jobs held by women in tech has actually fallen over the past 23 years as the technology sector has grown and pervades our daily lives. One study showed that in 2013, just 26% of computing jobs were held by women in the U.S. alone, which was down from 35% in 1990. Since the 1980s, we have also been witnessing a steady decline of women in computer science even though the percentage of engineering jobs for women are supposedly on the rise. While well-meaning conferences and pioneering organizations focused on fostering gender equity and training for women in technology are rapidly emerging to rectify this increasing gap, what continues to fascinate me is the absence of a robust discussion of alternative genres and product diversification. Rather than envision how we might design technologies differently from a women-centered point of view, much of the mainstream attention emphasizes fitting into a pre-existing system and approach to technological development. We continue to dress in drag when we could instead pro-actively shape the discourse and re-humanize our relationship to future technologies. Why? My hunch is that if we go all the way back to the forking moment post World...

Be Biocreative: How will you use the XTH Sense?

Wow! March has been like a time warp. We’ve been having lots of fun driving around Berlin, London and New York gathering case studies of musicians, researchers, animators, dancers, virtual reality creators, even kids, using the XTH Sense to enable us to better understand the various ways our community might interface with our new biocreative instrument. We’ve come away pleasantly surprised and genuinely heart-warmed by all the ingenious ways people envision integrating the XTH Sense into their creative and research practice. One thing all the interviews had in common was a sense of awe and wonder when they heard their raw muscles sounds with headphones for the first time as they began to slowly move and re-attune themselves with their bodies. For composer & cellist Illay Chester, experimenting with XTH Sense was “very curious, like entering a different dimension, a whole new world, and its right there inside of you.”  Dancer, Susanne Eder, was amazed that the instrument could make the tiniest movement of her muscles perceptible, which restored “an open and natural child-like sense of play” to her often over-disciplined practice. And researcher, Pedro Lopes, observed that in contrast to the barriers he typically encounters with other tracking or biomedical devices, it was refreshing that “everything is completely transparent, and you can dig into any layer of the raw data and code.”  Here’s what media & sound artist, Andrew Demerjian had to say: http://www.xth.io/wp-content/uploads/2016/03/andrew_clip.m4v Over the next few weeks, leading up to Kickstarter, we’ll be announcing two great opportunities for our community to shape the future of the XTH Sense. Be on the look out for a short survey next...

Mixed Reality with the XTH Sense

Last week we organized our first creative lab exclusively focused on using the new XTH Sense™. We gathered at the Integra Lab at the Birmingham Conservatory in UK with researchers in sound and technology to explore some creative applications of the XTH Sense™. Integra Lab is a music interaction design and research group. Their mission is very well aligned with our vision at XTH: to empower people to explore sound and music through easy-to-use technologies and tools. The lab team is an experienced group of artists, designers and developers with expertise in interaction design, digital musical instrument design, haptic interaction, gestural control, immersion and mixed reality, music composition and performance. Together with Federico Visi (ICCMR, Plymouth University) and Balandino di Donato (Integra Lab), we set out for three intense days of programming, movement and sound explorations in mixed reality. To create mixed reality environments means to enable virtual and physical worlds to interact with each other. An object in the physical world, like a cup for instance, can be linked to sounds in the virtual realms, so that the cup itself becomes augmented without losing its physical properties, but rather exploring them creatively with the body. It is a truly interesting approach to body and technology, mixing methods from VR (Virtual Reality) and HCI (Human-Computer Interaction) to create experiences where the real and the virtual co-exist, rather than disrupt or undermine each other. At XTH, we want to explore innovative ways of expression that merge our bodies with technology, and mixed reality is a fantastic test bed for this. Federico and Balandino used the XTH Sense™ to create two...

Biomedia, a new medium that restores human connection

In our last post, we claimed we were driving the creation of a new medium of expression: biomedia.  This week, we thought we’d share where it came from and how it connects to our larger vision. Eugene Thacker coined the term biomedia to show how biology is becoming the new media. Thacker says that the body is the medium and its biology is the message. He uses this idea to examines how bioinformatics uses DNA to perform computations. Echoing the cybernetic easy slippage, he sees a confluence between genetic and computer codes, between informatics and biology. Thacker views the scope of biomedia as a way to restore the materiality of our bodies. He suggests that we do not use computer technology in the service of biology, but rather that we understand the expression of our body more fully by using computers look at, and use, the body biology. At XTH, we are extending Thacker’s notion of biomedia. With our approach to body technology, we are creating technologies that create real-time, dynamic interaction with computers using the physiological processes of our own bodies. We are creating a direct connection between biology and media. In contrast to the current direction of the wearable market, which views the body as purely data to track, measure, and analyze, our XTH Sense™ is a “bio-expressive technology”. It reveals the human body as more than data. The XTH Sense amplifies inaudible sounds that characterize each of our own bodies. It re-establishes our intimate connection with ourselves, and with one another. It lets us share human experience through our bodies. By redefining Thacker’s idea of biomedia,...

Forging a New Medium Together through Open Knowledge

Most of us are familiar with the concept of open source and open hardware. An open project implies that source code and hardware schematics are released under a license which allows modification, re-use and re-distribution for free. But the value of making fully open software and hardware goes beyond the simple use of a particular technology. Makers of open software or open hardware do so in order to guarantee the freedom of other creators to participate in technological innovation. When you buy software or hardware which are not open ( i.e., closed source) you’re not actually buying the technology. You’re only buying the right to use it. We have seen many technological firms release open SDK (software developer kits) or API (application programming interface), but that is not the same as producing an open technology. Again, what an SDK or an API provides you is the right to use “part” of a technology. The actual technology is kept hidden under the hood, in ways that constrain your creativity, your ideas and your work. This is where the philosophy of open culture comes into play. Projects like Arduino, LittleBits, Adafruit and Makerbot are great examples of technology makers enabling people to create freely, contribute to one another’s projects and build on a vast and constantly growing network of knowledge. What we are doing at XTH is an expansion upon the open knowledge and methodologies created through those communities. Our contribution is to drive and shape a new field in biocreative expression, experiences, interaction, and technologies. We believe the open source/open hardware community will enable the emergence of a new medium,...

Why we have chosen to invest in our community

Here we are, it’s 2016. In the past months you have not heard from us, and we want to thank you for all the email you sent us, either to ask us when the XTH Sense will be ready to buy (very soon! more on that later next week), or simply to show your support. It is so much appreciated, and we need it! We have been a bit absent online because it has been an intense period, not only of work, but of reflection too. We have been actively seeking funding from investors and tech incubators. While we keep receiving enthusiastic feedback and very useful insights from insiders, eventually, our conversations always end up with a clash of values: “Your product is great, but you’re not going to earn increasing profits by making a product only for creatives.” We can see the point and agree. To our eyes, however, the fundamental assumption is wrong: we are not here to make increasingly high profit margins, and we want don’t want to make only a single product to sell out to a major corporation. Of course, we want to build a sustainable company, yet our vision is more expansive. We want to build a movement and a new medium around a deeper understanding of our relationship to the body and technology. Investors seem not to trust this aspirational goal. So we have been reflecting upon this, a lot.  We had a kind of existential crisis, and we finally took a decision. We are going back to the roots of the XTH Sense project with renewed energy and more mature ideas....

XTH Sense’s expanded functionality reaching new audiences

The past weeks have been quite packed for our XTH Team. We have been working on two major presentations: a large-scale participatory concert by co-founder Marco Donnarumma in collaboration with spatial sound innovators 4DSOUND; and a presentation/pitch by co-founder Heidi Boisvert at THNK FSTVL, a think-tank of creative leaders focused on social change and forward-thinking solutions to the most pressing issues of our century. Marco’s new work, entitled 0-Infinity, was commissioned by 4DSOUND in occasion of the launch of their 24h program Circadian, which premiered at TodaysArt Festival in The Hague. The ambitious program curated by 4DSOUND brought together world-renown artists to create new works that prompted reflection on the relation of sound, light, physiology and sound affect. Marco created an unstable system that, by amplifying the heartbeat and blood flow of visitors and their movements in space, created an intense and entraining communal experience of sound and light. The work was realized during a residency at 4DSOUND’s new HQ in Budapest. There, Marco together with the 4DSOUND team, interfaced the new wireless XTH Sense with the original spatial sound algorithms created by 4DSOUND. Salient features of muscle sounds and blood flow sounds were mapped to the dimensions of multiple sound sources, while the 9-degrees-of-freedom from the XTH Sense’s accelerometer, gyroscope and magnetometer were mapped to the movement of those sound sources in space. It was particularly interesting to experience how the infrasound and low frequencies produced by the human body and amplified by the XTH Sense could be spatialized across the studio, roughly 300sqm, in a way that was clearly perceivable to the listener. This is rather uncommon,...

First Batch of Custom XTH Sense Almost Ready

It’s been awhile since our last update. Marco & I’ve been busy finishing up our PhDs, traveling around to demo the first iteration of the new XTH sense at various conferences and festivals, including SOLID by O’Reilly, NIME (New Interfaces for Musical Expression), NYEAF (New York Electronic Art Festival), and cultivating some exciting partnerships we hope to share with you very soon! In the meanwhile, our team (whom you met in our last post) has made some astonishing progress. Marije has finalized the firmware and Marco’s been working with Will to parse the two new data sets (body temperature & 9-degrees-of-freedom). They’ve also been getting the PD patch into C for the API.  Once these remaining items are complete all the energy will be focused on the software side: feature extraction, musical algorithms, and interfacing with web-based applications, mobile devices, Unity, Isadora, Ableton and anything else the community can think of. In addition, Simon’s been preparing a small, custom batch of boards and Marina’s sent out the design for both the transmitter and receiver encasing to a 3D printer . She has also been tirelessly hand-sewing some adjustable straps for all parts of the body. We’ll use this first batch for demo-ing, market discovery and a new commission of Marco’s work. On the branding front, Paola has completed our beautiful brand book, which we are beginning to implement to increase our visibility, and she and I are moving into Kickstarter mode in preparation for our launch in the Fall. Things are moving swiftly!  Sign up for our newsletter for more details or follow us on Facebook or Twitter to...

The New XTH Sense is Coming: Meet the Team

In our first post, we announced the launch of our new website and told you we were going to use this space to share our development process. So to wet your appetite, what you see above is our current prototype of the brand new XTH Sense! Today, we will introduce you to the amazing global team that is making it possible, so you can put a face to a name, and know who’s been up to what for context as we share more over the coming weeks. From the very moment Marco & I first received the Creativity + Technology = Enterprise funding from Harvestworks through Rockefeller Foundation’s Innovation Award to commercialize the new stand alone, wireless version of the Xth Sense, we knew it was essential to bring on board hardware engineer, sound artist and performer Marije Baalman. Marije was the mastermind behind Sense/Stage and a key member of the legendary STEIM, a pioneering lab for physical approaches to new musical instruments and interactive music. Marije invited fellow engineer and maker of impossible things, Simon Claessen to join us in developing the transmitter and receiver. Together, Marije has been spearheading the firmware and Simon has been designing the PCB boards. As we started talking through the functional and usability design for the components, we soon realized it was important to identify a very specific type of industrial designer. We reached out to our friends at the Waag Society for some recommendations. We were immediately inspired by wearable and innovative materials designer, Marina Toeters, and felt her fashion sensibility and work with Philips was a perfect fit. Once the...

Welcome to the new XTH blog

Last year at this time, Marco and I publicly launched XTH at the Sonar Music Festival in Barcelona where we were invited to be a sponsor for the Music Hack Day. Both the Sonar Prize and the FICOD Innovation Award were given to the hack by Corné Driesprong, Jelle Akkerman, and Yuya Kikukawa, who used the original wired version of the Xth Sense. We’ve come along way since then. We’ve assembled an amazing team. Together, we’ve been tirelessly designing, developing and testing our first market-ready product, a new 2-inch wireless version of the Xth Sense with added functionality and a robust API to interface with mobile phones, web-based applications and game engines. And we’re almost there!  So, get ready for the first open-source biowearable musical instrument to rock your world. Just last Friday, we heard our first wireless sounds! A major milestone. All the elements were communicating: the transmitter, receiver, firmware and API. Marco was so excited he was bouncing around his flat in London using his heartbeat to run some tests. He, Will, Marije and Marina are working on the final tweaks: getting rid of the noise, improving the signal spectrum and adjusting the mic to be more secure against the skin to amplify the sound. Marco wanted to share his heartbeat with you, so here you go! You might need headphones to hear the low frequencies. http://www.xth.io/wp-content/uploads/2015/06/heartbeat-test-ngr.wav To celebrate our year anniversary and to honor our open-source ethos, we decided to start this blog as space to share with you our development process and the lessons learned along the way. Over the next few weeks, starting in...

Pin It on Pinterest