Notes on Digital Audio

Chapter 6. Meeting 6, Recording: Digital Audio 6.1. Announcements • Next quiz will be next week 6.2. Quiz Review • ? 6.3. Listening: League of Automatic Composers • Listening: League of Automatic Music Composers, “Oakland One,” League of Automatic Music Composers 1978-1983 • The League of Automatic Music Composers: founded in the 1970s by Jim Horton and including John Bischoff, Tim Perkins, and Rich Gold (Holmes 2008, p. 276) • Made use of the KIM-1, created by MOS Technologies in 1975 (Holmes 2008, p. 275) Source: Wikipedia, by user en:Wtshymanski. Public domain image. Source: Wikipedia, by user Swtpc6800. Public domain image. 6.4. Basics of Digital Encoding • Digital is discrete, analog is continuous • Take discrete time samples of a smooth analog signal • Each sample measures amplitude at a point in time • Time interval (spacing) is constant; often given as a rate in samples per second • Amplitude steps are positive or negative values within a fixed range of values • Encoding (analog to digital conversion) is always lossy • Decoding (digital to analog conversion) my repair some of the loss • PCM: Pulse Code Modulation 6.5. Two Parameters of Digital Encoding: Sampling Rate • Sampling rate • Bit depth 6.6. Two Parameters of Digital Encoding: Sampling Rate • Sampling rate • How quickly amplitudes are measured, or the time resolution Source: Ballora, Mark. Essentials of Music Technology. © Prentice Hall, 2002. All rights reserved. • Determines what frequencies can be recorded: higher sampling rates can record higher frequencies • Doubling the sampling rate doubles the amount of data stored • Measured in Hertz (samples per second) • Examples: 44100 Hertz (CD Audio), 48000 Hertz, 88.2k, 96k 6.7. Two Parameters of Digital Encoding: Bit Depth • Bit depth • How accurate are amplitude measurements when sampled Source: Ballora, Mark. Essentials of Music Technology. © Prentice Hall, 2002. All rights reserved. • Determines what range of amplitudes can be recorded, or dynamic range: higher bit depths can record more dynamic range • Doubling the bit depth doubles the amount of data stored • Measured in bits • Examples: 16 bit (CD Audio), 24 bit, 32 bit 6.8. Encoding and Decoding • Encoding: smooth analog wave forms are measured in discrete samples Figure 2 in Tenney, James C. "Sound-Generation by Means of a Digital Computer." Journal of Music Theory 7, no. 1, 24-70. Copyright 1963, Yale University. Reprinted by permission of the present publisher, Duke University Press. • Decoding: digital information converted to pulses that are filtered into smooth analog wave forms Figures 3 and 4 in Tenney, James C. "Sound-Generation by Means of a Digital Computer." Journal of Music Theory 7, no. 1, 24-70. Copyright 1963, Yale University. Reprinted by permission of the present publisher, Duke University Press. 6.9. Sampling Rates: The Nyquist Theorem • The highest frequency we can record is about half the sampling rate • To sample a sine wave, at least two points per cycle must be sampled • Trying to sample a frequency higher than the sampling rate leads to confusion: new frequencies are generated below the sampling rate • Example: 44100 Hertz can record up to 22055 Hertz 6.10. The Limits of Sampling • Resampling an audio signal at the audio rate [samplingRate.pd] 6.11. Bit Depth: Dynamic Range • The bit depth determines dynamic range • 16 bit audio has 96 dB dynamic range (65536 discrete volume levels) • 24 bit audio has over 110 dB dynamic range • Each additional bit adds about 6 dB 6.12. A Bit on Bits • Bits are a way of storing binary numbers • The number of bits tells us how many numbers (things, positions, values) available • One bit encodes two possible values: 0/1 • Two bits encode four possible values: 00/01/10/11 • Three bits encode eight possible values: 000/001/010/011/100/101/110/111 • 4 bits encode 16 possible values • 8 bits (or one byte) encode 256 possible values • 16 bits encode 65,536 possible values • 24 bits encode 16,777,216 possible values • In general: 2bits = possible values 6.13. The Sound of Degradation • Sample rate reduction and bit smashing • Both make curves more square • Making curves more square adds high frequencies [samplingReduction.pd] 6.14. Listening: Alva Noto • Listening: Alva Noto, “Xerrox Meta Phaser,” Xerrox Vol. 2, 2008 6.15. Digital Storage • Audio files store digital sound information • Can store multiple channels of digital audio in a single file • Two components necessary • Header information: sampling rate, bit depth, number of channels • Data: a list of amplitude measurements • Playback system must properly interpret header and read sample data • Digital information can be stored on magnetic, optical, or other mediums 6.16. Digital Storage Size • MB / min == Sampling rate * bytes per sample * channels * 60 (s/min) • 44100 * 2 (16 bit) * 2 * 60 == 10 MB / min 6.17. Common PCM Digital Storage Formats • A long list of sample values, with header information describing channels, sampling rate, and bit depth • PCM Formats: AIFF, WAVE, others • May have bit depth from 8 to 32, may have sampling rates from 22050 to 96000 • Compressed, non-PCM formats: MP3, M4A, MWA, OGG 6.18. History of Digital Audio • 1928: Harold Nyquist at Bell Labs develops Nyquist Theorem • 1938: A. Reeves develops first patented pulse code modulation technique for message transmission • 1950s: Max Mathews at Bell Telephone Labs generates first synthetic sounds from a digital computer • First digital one-channel audio recorder demonstrated by NHK in Japan • 1973: Nippon Columbia (Denon) has digital audio recorder based on 1 inch video tape © Denon. All rights reserved. • 1977: first commercial digital recording system: Sony PCM-1: encode 13 bit digital audio onto Sony Beta videocassette recorders © Sony. All rights reserved. • 1978: Sony PCM 1600: uses 16 bit encoders • 1984: AMS NEVE Ltd release AudioFile system: first hard disk audio recording system (Holmes 2008) • 1985: Audio Engineering Society establishes two standard sampling rates: 44.1 and 48 KHz 6.19. Consumer Digital Audio: CD • 1982: Compact Disc becomes available (Philips and Sony) • Within two years sells 1.35 million players and 10s of millions of discs 6.20. Consumer Digital Audio: DAT • mid 1980s: Digital Audio Tape (DAT) released © Wikipedia User:Yaoleilei. License CC BY-SA. • Popular for professional use in studios and field recording; commonly used for higher 48 kHz sampling rate • 1992: Audio Home Recording Act: adds Serial Copy Management system to commercial DATs, adds royalty fees to digital records and blank digital media • 2005: Sony announces that DAT machines will be discontinued 6.21. Consumer Digital Audio: New Formats • • DVD-Audio • Support for 96 kHz, 24 bit, 5.1 surround • Support for 5.1 and alternative formats • Partial compatibility with DVD Video players • Supports 8.5 GB of data • Maximum bit-rate of 9.6 mbps Super Audio CD (SACD) • As of May 2009, 5500 SACD releases, 20 times more titles are available on SACD than other high resolution formats, mostly “Classical” music • Supports 7.95 GB of data • Support for 5.1 surround and alternative surround formats • Hybrid discs are compatible with old CD players • Digital Stream Digital Encoding: not PCM • Pulse-density modulation encoding: significant debate on quality • Direct-Stream Digital (DSD) Encoding: not PCM Uses 1 bit encoding at a sampling rate of 2.8224 Mhz (1 bit times 64 times 44.1 kHz) Over 120 dB of dynamic range and frequency response over 90 kHz Significant debate over quality A type of pulse-density modulation Image: Wikipedia, by Pawel Zdziarski, license CC BY. © source unknown. All rights reserved. 6.22. Professional Digital Audio Multitracks • 1978: 3M with BBC introduces first 32 track digital recorder • 1970s: Soundstream releases first computer-based sound editor and mixer © Soundstream, Inc. All rights reserved.. • 1990s: low cost digital multitracks by Alesis (ADAT) and Tascam © Alesis LLC (top) and TASCAM (bottom). All rights reserved. • 1991: Digidesign releases first version of ProTools, supporting four 16-bit tracks (Holmes 2008) • late 1990s: digital audio workstations (DAW) on personal computers 6.23. Digital Audio Workstations and Non-Destructive Editing • DAWs separate the digital audio and the presentation of that digital audio • Numerous different presentations of the same audio segment are possible, each with different boundaries and at different times in different tracks • Removes risk of over-writing material (overdubs) and necessity of cutting and splicing tape 6.24. Reading: Sterne • Sterne, J. 2006. “The mp3 as cultural artifact.” new media & society 8(5): 825-842. • Why might Sterne describe the mp3, after Mumford and Sofia, as a container for containers? • “... the mp3 has been ascribed the status of a thing in everyday practice” (2006, p. 830); is the mp3 a thing? • In numerous places Sterne assigns agency to the mp3: “the file is designed to figure out what you will not hear anyway and to get rid of the data for that portion of the sound” (2006, p. 832); “The encoder then decides how much data to retain and how much to discard ... The encoder calculates a new timbral measurement for each frame based on what it learned about the shape of the incoming signal” (2006, p. 833); “the mp3 encoding process puts the body on a sonic austerity program. It decides for its listeners what they need to hear and gives them only that” (2006, p. 838). Is this necessary, and does this support or hinder his overall argument? • Sterne states that “Mp3s are designed to be heard via headphones while outdoors, in a noisy dorm room, in an office with a loud computer fan, in the background as other activities are taking place and through low-fi or mid-fi computer speakers” (2006, p. 835): what evidence supports this claim? • What agency do humans have in producing and making mp3s? Does Sterne account for this source of agency? • What other music technologies were “designed for massive exchange, casual listening and massive accumulation” (2006, p. 838)? • Do you agree that the mp3 is a “celebration of the limits of auditory perception” (2006, p. 828)? 6.25. Compression: Non-Lossy (Lossless) • All data is stored and can be retrieved • Various approaches • • Compact redundancies • Translation tables • Analyze and produce algorithmic replacements Examples: zip (LZW), Huffman coding, FLAC, Apple Lossless 6.26. Compression: Non-Lossy (Lossless): Example • Run-length encoding • Compact redundancies by replacing identical data with a code that gives the length of data • CCEEEGGGDAGFGGGGC • 2C3E3GDAGF4GC • In terms of a PCM audio file, silence can be compressed more than noise 6.27. Compression: Lossy • Data is lost • Various approaches • • Data is removed that is less relevant • Data is removed that is statistically predictable and common Examples: jpeg, mp3, aac, ogg 6.28. Compression: MP3 • Psychoacoustic measures are used to remove (supposedly) non-audible signal components [demo/psychoacousticMasking.pd, demo/psychoacousticPanning.pd] • Frequency-domain analysis: sound is broken up into many (32 or more) frequency bands • Encoder analyzes data and removes data that is not expected to be heard based on frequency domain analysis • Encoder performs lossless compression on data • Bitrate (kbps) determines quality: 96 to 320 are common • kbps: refers to kilo-bits per second: transmission rate necessary for real-time audio playback • CD-audio (2 channels of 16 bit 44.1 kHz sampling rate audio) takes 14,000 kbps 6.29. Aural Comparison of MP3 Quality • MP3 distortion is most often audible with complex signals in the mid-range (cymbals and highhats in particular) • Low bit rate encoders often remove high frequencies • Comparison of AIFF, 320, 256, 128, 96, 48, 24