A synesthesia installation : Zeek ALL

A synesthesia installation : Zeek ALL

During winter 2014, with the platform developed during the M.A.E.L.T. research project that involved Artefacto, Spectaculaires and EESAB, the Ubeek association with the help of Artefacto, Spectaculaires and ATM have proposed a synesthesia installation during the Transmusicales Festival.

People were invited to simultaneously manipulate sonorous and visual objects in Augmented Reality. While also being immersed inside one of the 6 environments available, people were able to compose the music together.

A 5.1 sonorous environment was deployed (3 HPs in the front + 1 bass and 2 HPs in the back). The software, developped by Artefacto, allowed the user to play up do 1024 tracks simultaneously with applying in real-time a dedicated spatialized compression.

With a blue carpet on the ground, a ChromaKey was configured so as to insert people into one of the 6 environments proposed. A dedicated interactive webpage available on a ad-hoc Wifi, allowed users to manipulate sounds and 2D/3D metaphors in real-time. The sounds and the metaphors were present in real-time in the same location in Augmented Reality which allowed people to experience a unique synesthesia installation : people saw sounds and heard 2D/3D metaphors.

Video made by Le Mag Numerique

Top video made by Ubeek

Romain Cavagna is mainly working on Augmented Reality and Interactive Projections as a research project leader at Artefacto. He got his PhD in computer science from the Rennes 1 University on content management and quality optimization for 3D contents distribution and during it's postdoctoral research fellow at LIP6 he worked mainly on p2p networks for games. He is also the president of the Ubeek association whose first objective is to gather people and skills around artistic and digital projects.