A synesthesia installation : Zeek ALL

A synesthesia installation : Zeek ALL
February 18, 2016 Romain Cavagna

During winter 2014, with the platform developed during the M.A.E.L.T. research project that involved Artefacto, Spectaculaires and EESAB, the Ubeek association with the help of Artefacto, Spectaculaires and ATM have proposed a synesthesia installation during the Transmusicales Festival.

People were invited to simultaneously manipulate sonorous and visual objects in Augmented Reality. While also being immersed inside one of the 6 environments available, people were able to compose the music together.

A 5.1 sonorous environment was deployed (3 HPs in the front + 1 bass and 2 HPs in the back). The software, developped by Artefacto, allowed the user to play up do 1024 tracks simultaneously with applying in real-time a dedicated spatialized compression.

With a blue carpet on the ground, a ChromaKey was configured so as to insert people into one of the 6 environments proposed. A dedicated interactive webpage available on a ad-hoc Wifi, allowed users to manipulate sounds and 2D/3D metaphors in real-time. The sounds and the metaphors were present in real-time in the same location in Augmented Reality which allowed people to experience a unique synesthesia installation : people saw sounds and heard 2D/3D metaphors.

Video made by Le Mag Numerique

Top video made by Ubeek