miRatio – a concert in virtual rooms
miRatio is a production by the Mediendom of Fachhochschule Kiel and guitar player / composer Jens Fischer.
miRatio premiered in 2017 at Mediendom, Kiel.
In an impressive three-dimensional sound space Jens Fischer plays live together with musicians, who can be seen in videos within a fulldome projection.
The virtual setting was created by multimedia producer Bob Weber. “miRatio” means “amazement”. Abstract spaces with extraordinary perspectives and optical illusions are guiding to the inner nature of music.
In a unique way, “miRatio” uses the 64-channel SpatialSound Wave System by the Fraunhofer Institute for Digital Media Technology. With the aid of this sound system, the instruments played by the musicians, who can be seen in the dome, are acoustically perceived exactly at their positions.
- Music and artistic direction by Jens Fischer
- Visual concept, 3D-animation, compositing by Bob Weber
- Video production by Linn Marx
- Compositing, 3D-audio technology by Marko Herrmann
- 3D-audio technology, software development by Markus Schack
- Executive producer: Eduard Thomas
- A production of Mediendom / Kiel University of Applied Sciences, Germany
- Made possible by „Stiftungen der Förde Sparkasse“ and the „Kieler Planetarium e. V.“
- Jens Fischer, guitars
- Jens Schliecker, piano
- Katie Zahn, flute
- Pirkko Langer, cello
- Friedrich Paravicini, cello, bass, Ondes Martenot
- Nils Rohwer, marimba
- Lucas Kockbeck, drums
miRatio – making of
For each musical piece, to begin with, a separate room was conceived and set up in a 3D program. Spaces and objects were modelled and their surfaces designed to look as if they were made of various materials such as wood, metal or fabric.
Empty 3D surfaces served as placeholders, where at a later stage the musicians were to be seen within the 360-degree setting.
As soon as all elements were positioned in the 3D space, the animation of moving objects and a virtual camera followed. The scenes were suitably illuminated and calculated as if they had been filmed through a fisheye lens. This is how the round picture format for the dome was finally created.
In the score each tone was defined, because the musicians not only had to record their part in the sound studio, but also had to repeat it exactly in sync with the playback later on in the video shoot.
These videos were shot one after the other on the stage of a music club and later inserted into the virtual setting at their designated positions. Eventually, the final fulldome video was generated in 4k resolution.
The live guitar and the pre-produced tracks by the musicians are mixed together in a DAW (digital audio workstation) and transmitted on 16 channels to the SpatialSound-Wave System of the Mediendom. The DAW and a Digistar5-System, which generates the projection in the dome, are precisely synchronized by a time code.
The SpatialSound-Wave software “distributes” the audio signals of all instruments to 64 speakers and 4 subwoofers of the Mediendom. Special software calculates the exact position of the virtual sound source in the dome from the coordinates of the musicians’ videos (tracking points), so that the instruments can be heard coming exactly from where they are seen in the dome.