The Smile Powered Synthesizer
This is a product that using your web camera and motion-tracking technology, instantly bestows your movements with new capabilities and powers!
Using the magic of augmented reality and machine learning anyone can now play a full musical synthesizer simply using their face =P
By piggybacking onto a person's own natural movements, we reduce the complexity and time required to learn a new skill - in this case a musical instrument - whilst still retaining fully immersive expression and control. This allows practically everybody to create music in a purely fun way in only a matter of seconds.
You can play with the fully-functional prototype at https://interface.place
This project uses existing cutting edge technologies and applies them in exotic new ways to create a potentially disruptive breakthrough into how interfaces can be controlled in the future.
It is a creative tool but also proof of the power of the concept, showing that an expressive and simple interface can control more than simple household appliances and allows many people to have fun making music who were previously unable to do so.
The ability to express, communicate and collaborate satisfies very real human needs. This allows everyone with a computer, the internet and a face to make music, easily, for free. It has dreams of being the most accessible musical-instrument and hopes to prove the approach of gesture piggy-backing to be a valuable input mechanism for the future - where no physical touch is required and no new skills are necessary.
Developed in the UK and continuously improved by Drake Music charity, specialists in accessible music technology, who have been testing the product comprehensively to ensure it works with the broadest variety of disabled users as part of their monthly #DMLab meetups which bring together disabled artists, inventors and makers.
This project was a product of free-time afforded by COVID and driven by the continued encouragement and motivation of #DMLab and their patrons, but it has captured the imagination of many others and has been featured at various events and exhibitions such as SXSW21 and Music Maker Festival. This exposure brought many new users and increased the audience base to include various schools and institutions, music / art therapists as well as hundreds of weekly new international home users. It is clear that there is demand for the project and people find it valuable.
Accessibility is a primary characteristic of the project so it is vital that the technology works for all diversities and physical appearances. Having access to #DMLab and an online audience allows the project to be tested and any idiosyncrasies or discriminatory behaviour can be isolated and improved so that ultimately the project becomes fully inclusive and one of the very few of it's kind that can claim so.
As a musical-instrument it has wider value as part of the music making ecosystem and has MIDI integration directly built in which allows it to communicate with any other external musical gear, be it a drum machine or keyboard, so that it can be used as part of professional quality studio setup.
As a showcase for the potential of gestural expression technology - using natural movements as new control mechanisms - creating a musical-instrument represents the perfect fit for a proof of concept as it needs to combine many levels of control into one simple interface whilst maintaining a fine level of expression. If a compelling instrument can be successfully made with no physical interface it will validate the concept for use controlling everything else, paving the way for integration with the Internet-of-things (iOT) as well as interfacing with other electronic devices (eg. radio controlled toy car) and services.
Ultimately the true potential lies in promoting this type of gestural expression so it becomes a recognised form of input and another mainstream way to interface with technology. This research and data about human input uncovered during the development is very valuable and useful in many other markets and applications.
In the future, if the technology is broadly adopted in products and other contexts it has the opportunity to bring back abilities and skills lost to disability and age. The repercussions could extend the work-life age, create new jobs, promote independence as well as bringing the job market to new demographics.
The unique circumstances brought about by lockdown are ending and project progress will not be certain unless the opportunity arises to fully commit and dedicate time to the project, something I am hoping CreaTech will help highlight!
