text: Mixed Reality Lab- Classical Meets Media « Adafruit Industries – Makers, hackers, artists, designers and engineers!
  Via Mixed Reality Lab
  The audience will be able to listen to classical music and at the same time appreciating the meaning of the musical scenes in visual art. The dynamic visual art projected behind the orchestra adds a new dimension to classical music appreciation and it helps the audience to apprehend the intention of the composer and the expression of the performer.
  As one of the most common aims of fine Arts and specially in the traditional thinking of Music, one of the earliest uses was in the accompaniment of theatrical dance or story-telling, so it is very usual that composers should from time to time produce what we know as "program music" – music written to portray events, activities or moods such as pastoral scenes or storms.
  Thinking on an specific score, what defines a composer to another are a lot of little details that, along the time have been elaborated and transforming in one or other way; so there are some factor in a Mozart score that could never been attributed to Haydn. Those several items are taken carefully by the analysis process in order to distinguish the real parameters for a composer.
  Pre-Concert Artistic Process
  As an application tool for designing in real time there are different types of artistic parameters we have to consider previously.
  Before designing what kind of image we are going to obtain by processing the music as an input signal we have to consider the artistic parameters are embodied as an inherent part of the score, and witch is the most important part of the process.
  Violin Bow Tracking:
  In-Concert Live Visual Art Creation
  We will put two color strips on the violin bow (Figure 1) to track the bowing actions. Expressions like staccato, accent., pizz., can be captured by tracking the bow actions as well as other violin issues: Sautille, alla punta, tallone, sul tasto, sul ponticello, col legno, etc…
  Dynamics of the music piece like cresc., decresc., p, f, sfp, etc can be tracked using the microphone output and be represented into visual effects.
  We will also track and process the pitch and beat of the violin sound in real time. Pitchddle~ patch is designed by Carlos D. Perales on Max MSP, based on fiddle patch design by M. Puckette and R. Climent. This pitch estimator and sinusoidal peak finder the pitch and amplitude of an incoming sound, both continuously and as a stream of discrete "note" events. Fiddle~ optionally outputs a list of detected sinusoidal peaks used to make the pitch determination.