“In 1842, when [Franz] Liszt took over the post of Kapellmeister in Weimar, he astonished the orchestra by saying, ‘Oh please, gentleman, a little bluer, if you please! This tone type requires it!’ Or, ‘That is a deep violet, please, depend on it. Not so rose!’ The orchestra eventually got used to the maestro seeing colors where they only saw notes.” -Wednesday Is Indigo Blue by Richard E. Cytowic and David M. Eagleman
Synesymph is a sound visualizing application I created for my senior thesis degree project. It uses Augumented Reality to simulate Synesthesia, a neurological condition in which one sense (in this case, hearing) involuntarily evokes another sense (in this case, sight). This is called Chromesthesia and is sometimes referred to as “colored hearing.”
Using 2D image targets, I took static visuals and brought them to life, syncing movement and color with audio. This was accomplished by working with the gaming software, Unity3D in tandem with the software development kit, Vuforia.
Synesymph has proven to be an effective interactive piece. Below are a few reactions from the app's test run in the Polykhroma Gallery Series with Tufts University students.
I created Gmail and Instagram campaigns to generate interest and increase attendance at Synesymph’s launch during my Thesis Symposium.
I presented my research, explaining what I learned about how pitch controls color while timbre controls movement and scintillation. To hear more about the science behind synesthesia as well as my design process, watch one of my talks below!
Still curious? Take a look through the process book "Synesymph, Bring Sound to Sight".