Douglas Holmes (Doctorate of Music in composition with a specialty computer music and minor in multimedia)
Virtualized sound space: engineering, realization, and delivery concerns
Current Projects: Binaural Mixer, and head position tracking
|
|
Influences: my obsession with sound
- The differences between Monophonic, Stereo, and Surround recording formats
- The Beatles
- Please Please Me (1963, Hard left/right stereo)
- Meet the Beatles (1964 Mono and Stereo Release)
- Abby Road (1969, Studio 8 Track Tape)
- Pink Floyd
- Dark Side of the Moon (1973, Stereo, Quadraphonic LP, Quadraphonic 8 track tape) (SACD 2007)
- A Momentary Laps of Reason (1987, Binaural recordings) -spatial headphone mix
- SoundGarden
- SuperUnknown (1994, stereo): “Head Down,” (construction sounds, clearly audible in headphones)
- Movies
- Star Wars IV - A New Hope (1977, Lucas Sound, Ben Burt)
- Apocalypse Now (1977, Coppola, Walter Murch, and early 5.1 Dolby Labs)
- Blade Runner (1982, Ridley Scott, Music by Vangelis)
- Live spaces
- Sound of Antiphonal Voices, Chorus (church service and concerts)
- Acoustically tuned performance space
- Clearly hear timbre and directional cues
- Natural environments
- City, Caves, Beach, Mountains, streams......
- Diffusion for live performance
Engineered sound and space:
- Expansion of the story space (the stage)
- Mono (all speakers receive same signal)
- Flat, difficult to define sonic individuality in a mix (every sound in coming from the same point is space)
- Stereo (two distant speakers driven by two separate tracks)
- Left and right material mixed to blend in the room
- Phantom center is created by panning crossover
- Small sweet spot
- Binaural (headphones deliver two separated signals)
- Isolated track, the transducer is in, or close to your ear canal (summed in your brain)
- Personal and isolated
- Surround sound speaker arrangements
- Designed to expand the stereo field and immerse the listener
- Expansion of the sweet spot allowing multiple listeners
- Problems with compliance: mix and differing listening formats
- Stereo to mono (fold-down)
- A good mix translates from stereo to mono (engineered decisions)
- Problems with phase (constructive and destructive processes) realized in the summing of the electronic or digital signal. Summing is done in the electronics, not in your head
- Stereo to headphones
- Good Idea to make sure your stereo mix is headphone compliant
- Headphone (binaural) to stereo
- A good mix translates from stereo to headphones (engineered decisions)
- Difficult to go from binaural to stereo
- Problems with phase (constructive and destructive processes) realized in the summing of the acoustic signal. Done in the room, not your head
- Surround sound to stereo
- Fold-down of surround and stereo channels
- Phantom Center (how to represent the center channel)
- Outcome is overpowering music and or sound effect tracks
- Solutions for audio playback compliance
- Multiple mixes packaged together
- Example: film stock with optical, magnetic, and digital information encoded on the sprocket side
- Super Audio CD (digital rights management)
- Sony/Phillips (early 2000s)
- Multi-layered optical disk
- Delta sigma modulation. Up to 50Khz frequency and 120dB dynamic range
- 5.1 layer (4 GB) (only used by SACD player)
- Stereo CD layer can be played on any CD capable device
- Cmusic: synthesis software developed by Richard More (1980s)
- Virtual space is described by using inner and outer virtual rooms
- Inner room: The listening space
- Outer room: The sounds virtual space
- Designed openings (holes) in the inner room walls represent the speaker placement
- Psychoacoustic principals are based at sound localization of sound source
- It is impossible to have a sound source closer than the loudspeaker itself (the speaker is the vibration source)
- A concept I'll explore in programming
Immersive sound:
- What is it?
- Higher fidelity (processing artifacts distract)
- From your perspective
- Totally enveloping (isolation and focus)
- Personal and private
- Different aesthetics for different media (*absolute and programmatic, music terminology)
- Music, an absolute space
- Film and audio, a programmatic space
- Gaming, a variable programmatic space
- Capturing space
- Stereo, encoded spatial recording
- Blumlein pair (two bidirectional microphones off set by 90 deg., Alan Blumlein early 1930s)
- Multi-channel recording
- Decca Tree (orchestral recording)
- Sound Field, encoded spatial recording
- Microphones designed to mimic the human head
- Ambisonics (math used for direct focal point in the sound field, 1970s UK)
Encoding the listener perspective (engineered focal point): current approaches to immersive sound encoding
- Channel-Based
- Binaural, Stereo, and surround formats in which placement of speakers determine engineering choices (Encoded)
- Scene-Based
- Ambisonics, sound field is navigated (Decoded)
- Object-Based
- Individual sound objects - ie. a musical instrument stored with location data (Renderd)
- Dolby Atmos (uses Channel-Based and Object-Based)
Head-Related Transfer Function and binaural encoding (rendered sound, relative to head position)
- Describes how we hear spatially(graphic)
- Localization of sound
- Based on physical characteristics
- Psychoacoustic
- Open Source C++ Libraries for programming
- Early research: Filters and delays
- Used in gaming (OpenAL)
- HRIR head-related impulse response (IRCAM, MIT, and UC Davis)
- Convolution of source with impulse
- Institute for hearing Technology and Acoustics, RWTH Aechen University
Encoding for delivery:
- Audio Descriptive Model Broadcast WAV format (ADM-BWF)
- Large Header is able to store sound placement data
- Used for Immersive Music along with a streaming format
- HDMI2.1 –eARC- (Analog Devices support)
- Two way communication
- Insertion and extraction of audio stream
- Up to 32 channels of uncompressed audio
- DSD support since HDMI1.2
- Dolby Atmos Master File (DAMF)
- Priority format
- Multiple file (3, XML data and Audio) used to describe Atmos mix
Demonstrations:
- Tracking head model (MAX/MSP Object-Oriented Programming Language and GyrOSC (Iphone application sending sensor data over WiFi)
- Binaural Mixer demonstration (Python3 Object-Oriented Programming Language utilizing PyGame, and OpenAl Libraries) The Code
Footnotes:
Absolute Music: is music for music’s sake. Examples Mozart’s Symphony No. 40 in G minor K550 (1877).
Programmatic Music: instrumental music with extra-musical narrative. Examples Vivaldi's “The Four Seasons” (1725) and Berlioz’s “Symphonie fantastique” (1830).
