We’ve established that there are a lot of good things about these encoders (dramatically reduce file size, allowed the internet to be the new battleground for music, etc…).

But what are some problems that these encoders can introduce?  Why do they sometimes sound wonky?

This article will be more relevant if you’ve read the other articles on file compression.

I’ll just list major problem areas as headings and then describe them.

Loss of Bandwidth

I talked about this a bit int he previous articles.  Lossy encoders don’t like encoding high frequency information, as it can take a lot of bits to do so, given the detail of the waveforms.  Given the fact that the encoder is limited by its KBPS (kilo-bits-per-second) setting, it needs to adhere to a specific number of bits, no matter what the source material is.  When the encoder doesn’t have enough bits to accurately encoder the whole frame, high frequencies are the first to go.  Check out the frequency content of this mp3, encoded at 96 KBPS (again from Steely Dan’s Hey Nineteen).  Notice how there is nothing above ~12k? (click for full size)

Now, compare that graph to the following one.  Below is a 256kbps mp3 of the same tune.  Notice how there is more high frequency information present (eyeballing looks like at least 16k or more)

Since the second mp3 has been given more than double the bits per second to encode, it has room for more of the high end goodness.

Pre-Echo

Pre-echo is quite an odd concept.  Its probably easier to understand in picture form (below).  When a quantizer is working, it is looking at a frame at a time (let’s say maybe 32 milliseconds long, like the picture).  When it encodes the quantization (and other) distortion/noise, it somewhat spreads it across the entire frame.  In a frame like this with a transient attack (its a castanet), there will be an increase in quantization distortion.  But, the transient doesn’t happen until the middle of the frame!  So, the quantization distortion it induces can actually be encoded before the transient even happens.   Unfortunately this is from an AES paper, and is in black and white (when it should be two different colors).  The bottom line is the level of quantization distortion, the top is the rest of the signal (with the castanet transient)

Brandenberg, Karlheinz. AES, 17th International AES Conference, Florence, Italy. MP3 and AAC Explained. New York, NY: AES, 1999

So notice how the noise line increases in amplitude way before the transient occurs.  This will sound almost like a reverb (think back to the Ogg Vorbis audio example), but it will happen before the attack of the source.  Weird=yes.

Time Smear/Double Speak

I’ll just hit on these briefly, because things start to get really complicated.  (When working on a presentation for the Art of Record Production a few weeks back, I found out what “brain tears” were by thinking about this stuff for too long).

Lossy encoders like to move transients around.  Assuming we are working with stereo audio (2-channel), the encoder is alot happier if the maximum of a given transient occurs at the same time in the L and R channel, rather than having to remember two different positions for the two channels.  Sometimes (especially in lower KBPS settings) the encoder will scoot one channel’s peak over a bit to line up with the other.  However, sometimes this shifting causes other parts of the signal to no longer line up.  If the encoder doesn’t correct this, then there will be a delay between the R and L channel at that point (which could cause the “double” speak).

Without getting close to “brain tears”, its just good to know that these encoders are moving things in time.  This is going to introduce more distortion, leave room for smearing/double speak, alter frequency response/phase, etc…..

In addition to all these^^ problems, more basic problems also occur

• A compressed file can sound “harsh” (also sometimes termed “rough”) because the nature of the noise floor is changing constantly (from encoding window to window)
• It can be noisy!  (especially when the encoder fails to lower the level of the noise below the masking threshold induced by the level of the “good” signal)
• Other non-harmonic distortions as a result of all of the above…

This is just an introduction to problems with lossy encoders.  I use the word “problems” with caution though.  In no way do I mean to say ‘MP3 SUCKS!” etc… These lossy encoders are really great tools for our industry, but it is important to understand what they are doing to the audio and why.

As usual, if there are any questions or need for further explanation, please post in the comments

C

The Studio Files

]]> http://thestudiofiles.com/?feed=rss2&p=525 2 http://thestudiofiles.com/?p=404 http://thestudiofiles.com/?p=404#comments Wed, 19 Nov 2008 21:35:26 +0000 http://thestudiofiles.com/?p=404 Now that I’ve told you all about compression formats, let’s do some listening.

This article will make more sense after reading the rest of this series (Take 1, Take 2, Take 3, Take 4, Take 5, and VBR)

The following audio files were made by converting a CD quality (44.1/16) version of Steely Dan’s Hey Nineteen into compressed formats (MP3, OGG, and FLAC… still tracking down my AAC file…).

Once converted, each file was summed to mono, polarity flipped, and summed with the original WAV file.

Now – this is NOT a representation of what “MP3 takes out”.  Its not that simple.  These compression codecs also introduce new distortion, restructure the bit depth, time shift peaks, raise/lower noise floors, etc… Therefore, this is not all that much of a scientific comparison.  So, then why are we doing it?

In essence, tests like these give you an idea of what the failure mode and distortion (aka, the “yuck”) within the compressed formats sounds like.  Also, you can at least get a good idea of how the compression codec is dealing with the high end of the spectrum.  Generally, the more high-end you hear in these examples, the more the codec cut out of the original file when compression down.  There are 2 specific things to notice in these.

1) Compare the mp3 to the Vorbis.  Notice how the Vorbis sounds more analog-like?  (Much more like natural white-noise-esque analog noise, rather than the harder, digital sounding distortion of mp3)  If you refer back to the article on Vorbis, you’ll remember that Vorbis alters the level of the noise floor to mask nasty-sounding distortion.

2) The FLAC summed file may not be what you expect.  Remember… its lossless (!hint)

(haha, its funny to note that the following files are actually .mp3… This is a great demonstration of why we need these formats… WAV files are too big!)

256 KBPS MP3 flipped and summed with original

256 KBPS Ogg Vorbis flipped and summed with original

FLAC flipped and summed with original

Questions? Disagreements?  Post it in the comments.

C

The Studio Files

aka FLAC (Free Lossless Audio Codec)

Keyword there being: Lossless!  No data is removed from the audio.  Let’s dive in…

Again, if you haven’t read Take 1, Take 2, Take 3, Take 4, and VBR, you may want to…

So far, all the compression codecs we’ve looked at are lossy, meaning they actually remove data from the file (and therefore, you could never accurately convert back from the uncompressed file).  The other encoders work on the principle of masking and psychoacoustic models, looking to “take out” ot de-emphasize the things humans won’t hear.

FLAC doesn’t.  And while similar in some ways, this is a whole different breed of encoder.  Plus (like Ogg Vorbis, its open source = FREE)

For starters, FLAC (like the lossy encoders) breaks the signal into smaller pieces.  The length of each piece is based on a whole slew of different factors, including dynamic level, spectral content, transients, etc… Now, this is the point where the lossy encoders would start axing (leaving out) things.

FLAC’s goal is to formulate its best prediction of how the signal is acting in the frame.  Once it has the best mathematical prediction it can come up with, it compares the prediction to the original, and encodes any differences (called the “residual”).  Check out this picture – its a verrrrrrrrry basic representation of what is going on here:

Now, remember that digital audio is a very finite thing.  At every given sample point, there are only so many different positions (within the bit depth) that the sample can be at.  This is essential to understanding how FLAC could be lossless.  If we were dealing with a signal that did not have a finite number of points (aka, analog), this would not work.  It could be close, but not completely lossless.  By having a mathematical representation of a frame (plus the residual encoding), FLAC can shrink the file size down usually around 30-50% (but more in some circumstances).

When I was first delving into FLAC, I thought “well… how can a compression format that is lossless have different levels of compression???  It must not be lossless…”

It has to do with CPU cycles.  When compressing FLAC as much as possible (say you are going for 50% size reduction), the computer will spend much longer trying to predict what the waveform is doing.  By getting a more accurate prediction up front, there will be much less residual data to encode.  So, the trade-off for a smaller file size is a longer encoding time (and be aware that I have not been able to find a FLAC encoder that can access more than one core of a processor).

That being said, we are only talking a few seconds here.  Where an mp3 may encode in 5 seconds, a fairly high compression FLAC might take 25 seconds.  If you read Part 4, you will have read the introduction to why open source compression formats aren’t amazingly popular right now…  If you google FLAC, you can find a slew of information on encoders, decoders, players, plug-ins… etc.  With newer iPods, I’ve heard there’s even a mod you can do to get FLACs to play (although not on the old ones, the processor requirement is too high) – careful though, I’d bet this is not within any limitations of the warranty or user agreement…

So what the heck is FLAC useful for?  My first thought would be for archiving mix/mastering files.  With 50% reductions on the files, all of a sudden you have to buy half as many backup drives.  It’s also useful for sending high quality audio to clients over the internet with less bandwidth (assuming they also know how to procure a FLAC player)

Hopefully we will see FLAC grow in popularity in the near future!  More to come soon on these compression formats…

C

The Studio Files

]]> http://thestudiofiles.com/?feed=rss2&p=387 0 http://thestudiofiles.com/?p=376 http://thestudiofiles.com/?p=376#comments Wed, 19 Nov 2008 02:21:42 +0000 http://thestudiofiles.com/?p=376 Ogg Vorbis?

Definitely wins the award for the weirdest name among digital audio compression formats.  However, Ogg Vorbis is arguably one of the best sounding and easiest to access of the bunch (it’s open source, aka FREE)

Again, this article assumes you’ve read Take 1, Take 2, Take 3, and hopefully also VBR.

“Vorbis” is actually the audio compression codec.  It was developed as an open source (lossy) compression codec as a substitute for mp3.  “Ogg” is simply the file container in which the Vorbis is held, thus “Ogg Vorbis”.  (Its worthy to note that you can hold lots of other things in Ogg containers, including FLAC).

So what’s different about Vorbis?  For one, its open source.  That means there are no licensing costs associated with using it.  Unfortunately, since people aren’t making millions of dollars off it, the effort to significantly popularize (or even standardize) the format is a hard sell.  MP3 and AAC are very much market-driven formats (i.e. iTunes uses AAC).  Since companies and individuals have incentive to make large amounts of money off of their codecs, its no wonder that MP3 and AAC are overwhelmingly more widely used than Vorbis.  That being said, it is good to know that Vorbis sounds really good at equivalent bit rates, AND its free.  (So although you’d be in the minority, if you are using Ogg Vorbis, you have something that sounds better* and costs nothing to license).

It’s also worth noting that if Vorbis were licensed and AAC open source, we’d likely be saying the same sort of things the other way ’round… this is really just an open discussion.

So, other than being free, what’s good to know about how Ogg Vorbis works?

Vorbis, like AAC, uses an MDCT to begin to break the digital audio into small pieces, and put them through the frequency-analysis process.  Vorbis uses its own psychoacoustic model (which theoretically is better than its predecessors).  Of course, the implication here is that if the psychoacoustic model is “better” than AAC and MP3, it will be getting better sounding results at the same bit rates.

Vorbis is different too in that it is always variable bit rate. (Click here to revisit the article on VBR encoding) While this will yield more pleasing sonic results for Vorbis, it can make it more difficult to compare MP3 and AAC encoders to Vorbis objectively (as some are not VBR).  Now for the big difference…. drum roll please….

Vorbis is different in its “failure mode”.  Here, I am using “failure” to describe what happens with the encoder doesn’t have enough available bits to get the good audio encoded above the distortion.  Remember, the encoder is inevitably going to have to deal with noise from things like quantization distortion, and ideally, all of that will be below the masking threshold (aka, we won’t hear the bad stuff because the good stuff is loud and covers it up).  When, let’s say, and MP3 encoder fails, the distortion gets loud, you may hear metallic swirlies (also called birdies, think “youtube audio”), other non harmonic distortion…

But! When Ogg Vorbis fails, something entirely different can happen.  When the audio is initially going through the filterbank in Vorbis, the encoder separates the noise floor from the audio.  When the encoder would fail, the noise floor is raised.  This gives a much softer (almost analog-noise-like) distortion, rather than birdies and harsh distortions.  So, between an improved psychoacoustic model, the inherent VBR, and the nicer-sounding failure mode, Ogg Vorbis seems like a great choice.

Again, problems with Ogg Vorbis mainly stem from the fact that it is not a market driven format.  There are many plug-ins and stand alone players that will read and encode/decode Ogg Vorbis files, but they are not nearly as numerous as those of MP3 and AAC files.  Most portable music players won’t directly play Vorbis files without some software tweaking.

FLAC is next!

Also coming soon are flipped polarity examples of these encoders with the original files (so you can hear the failure modes, and other “yuck” left over/added from compression conversion)

C

The Studio Files

…which, although commonly referred to as “Apple Audio Codec”, this actually stands for Advanced Audio Coding.

(This article assumes you have read Everything About Compression: Take 2)

Although originally coming out of MPEG-2 specification, AAC is known as Part 3 of MPEG-4. AAC was designed as a newer, “better” sounding alternative to MP3. Generally, the AAC encoder follows the same steps as the MP3 encoder, but let’s take a look at what makes the two formats so different.

First off, the filterbank (used to “chop up” the original audio file and determine the frequency content) in the AAC encoder is not a hybrid filter like the mp3 (which uses both an FFT and and an MDCT). The AAC encoder just uses an MDCT (the most important thing to note here is that an MDCT will typically overlap its bands of analysis, thus reducing any errors that might come at the “edge” points between blocks). AAC’s MDCT also has greater resolution than MP3′s (1024 vs. 576). AAC can also switch its block size (the length of the audio pieces it is analyzing).

The AAC also has a superior psychoacoustic model. The model was developed later (and likely spun from the MP3 original research). Presumably, the encoder with a more accurate perceptual model will be able to achieve better quality at lower bit-rates (kbps). The perceptual model is arguably the most important element in an encoder, so AAC’s improved psychoacoustic model is the most prominent reason why AAC sounds better at identical bit rates to mp3. (Note: “better” is obviously very subjective an open to debate. This is speaking rather theoretically)

Finally, AAC also has some additional tools which can help with efficiency, including Temporal Noise Shaping (TNS) and a Prediction stage. The inner processes of TNS are a little beyond the scope of this entry… but basically its a type of noise shaping and prediction that was originally implemented to help improve the sound of speech at lower bit rates. It helps with the efficiency and quality of the encoding. The prediction stage is not always used, as it is most useful in signals that are easily predictable (sine tones or tone-like soures).

Phew… enough of the technical jumble…

Oh, I should also mention that most AAC encoders will have the option to encode with variable bit-rate (which I went into in the VBR Explained Article).

So that’s AAC…

C

The Studio Files

This is a tangent from the compression articles. I thought I should explain VBR.

Normally, when encoding a compressed audio files (let’s say, an MP3), you need to specify a bit rate in KBPS (kilobits per second). This tells the encoder how many bits it has to work with over the given time period. (See Everything About Compression: Take 2 for info on what the encoder is doing)

But, what if a REALLY complicated second of audio is followed by a really simple second (say a second of metal mathcore followed by a 1k sine wave). In this case, the encoder would struggle to fit all the metal into the allocated bits, but would probably have a lot of wiggle room with the sine wave encoding.

Well, a VBR encoder would realize this, and vary the bit rate for that section. It would allocate more bits to the second of complex signal (the mathcore), leaving less (but more than sufficient) bits available for the sine wave encoding.

To relate it to real life… You ate a sandwich (on wheat bread) that had 300 calories. Now, you have two things on your “to do” list:

1) Stand up

2) Do 9007 jumping jacks

If you were not a Variable Bit Rate encoded person, you would allocate 150 calories for standing up and 150 calories for doing 9007 jumping jacks. What’s going to happen? Standing up is no problem, but sometime after you start jumping jacks, you are going to fail. When the compression encoder “fails”, it does things like not giving enough bits to the audio signal to raise it above the (quantization-induced) noise floor, thus audible distortion.

If you were a VBR encoded person, you could allocate 1 calorie to standing up, and 299 to jumping jacks, thus giving yourself almost twice the chance of finishing them.

A rather ridiculous example… but hopefully it gets the point across.

C

The Studio Files

Loved by many, despised by some…  regardless of where you stand, mp3 is hugely important in the proliferation of internet music.  It is the original method of delivering compressed audio (mainly over the internet).

What does MP3 stand for? How does it compress?  Why is it “good”? Why is it “bad”?  Let’s dive in…

MP3 actually stands for “MPEG-1 Audio Layer 3″. MPEG-1 was originally developed as a lossy compression standard for audio and video, and layer 3 (of the 5 total) is for audio compression.

So I started hinting at psychoacoustic modeling (aka perceptual modeling) in the last post.  In these encoders, these are essentially digital representations of how our (human) hearing system works.  Through rigorous and complicated testing, this model is formed to predict particularly how the hearing system is going to react to sounds.  What the model is mainly interested in is what parts of any given sound signal are going to be masked, both in the time and frequency domains.

I gave an exaggerated quasi-frequency based masking example in the last article, here’s a time based:  Suppose Jim is the fastest drummer in the world.  He can play a flam so quickly on his imaginary drum that the transients of the two hits are .00000001 seconds apart.  As humans, we probably aren’t going to be able to hear this as two separate hits (like a flam, as Jim originally intended to play).  The mp3 encoder might choose to just squish those two transients together.

**Note, this is just a rather-ridiculous example.  It assumes that both hits ring out independently, there is no comb filtering as a result of the addition, and that our sampling frequency is high enough to capture the difference in the first place (DSD anyone?), etc…

So here is a basic explanation of what is happening in the encoder.  Let’s pretend we have a 3 minute rendition of HAPPY BIRTHDAY as a WAV, and you want to compress it to a 128 kbps (kilo-bits per second) :

1) The uncompressed audio is broken up, both in the time and frequency domain, using a combination of FFT and MDCT filterbanks.  Simply put, the encoder breaks the original audio signal into very small blocks and analyzes the frequency content of each block.

2) At the same time, the encoder begins to examine the audio with the psychoacoustic model.  Here it starts to examine which parts of the audio will be masked by our hearing system.  This information is analyzed and fed with the MDCT’d audio (from step 1) into a quantizer.

3) The quantizer’s job is essentially to assign bits of data to all the information.  Since we chose 128 KBPS at the start of this process, the quantizer knows it has 128,000 bits available to assign for each second of audio.  With the information from the psychoacoustic model, the quantizer will know to give many less bits to parts of the sound that we can’t hear, and the majority of the bits to things that we will hear.  However, problems can arise if the encoder runs out of bits (in this case, if it uses up all 128,000 bits in a second and still has data to encode).

When this happens, something needs to happen. One option is to encode things with yet lower bit depths (or the amount of bits used for each sample).  Unfortunately, this may lead to more quantization distortion, inaccurate signal representation, errors in the time domain, other non-harmonic distortion… more bad things.  Another common thing to do is de-emphasize high frequencies.  If the encoder doesn’t have to worry about any frequencies, say, above 12 kHz, that frees up a lot of bits (in the process, destroying the high end of the source material).  Drag an mp3 into a DAW and run it through a spectral analyzer.  The lower the KBPS, the lower the cutoff for the high frequencies will be.

Some of this problem can be solved by using variable-bit-rate (or VBR) options in the encoders.  We’ll get to that in the next article perhaps.

4) Now quantized into a nice array of 1′s and 0′s, the encoder attaches metadata and other tags on the frames and assembles it into a nice mp3 package.  Once here, an mp3 decoder could quickly and easily decode an mp3 into audio.

Again, I apologize if that got more technical that necessary.  Here it is in sum:  The encoder breaks the original file into small pieces and analyzes the frequency content.  A perceptual model of how humans hear tells the encoder what parts of the original signal are unnecessary (or less important).  The encoder assigns bits to the data based on the model, giving less bits to things that we shouldn’t hear.  Then the encoder packs it up, labeling things so that the decoder can recognize and play the file.

Soon, we’ll get into explanations of the other encoders (most of which are fairly similar to mp3), problems with the encoders, etc…

If anyone is seeking deeper technical explanation of these processes, be sure to let me know in the comments.

C

The Studio Files

Ah…. the internet.  One of the greatest inventions of our time, not only for information technology… but also for (you guessed it) the audio industry!

The internet has allowed us to turn the music market upside-down.  With CD sales on the fall and internet music on the rise, a substantial need for good-sounding compression formats has arisen.  But how do these compression codecs actually do what they do?  Why do some sound better than others?  Why do some sound like… well… bad?  We’ll explore this over the series of articles.

Most of the common compression formats you will see online (mp3, AAC…) are lossy codecs.  That means, once compressed, data has been lost.  Once you convert your uncompressed (WAV, AIFF) file down to mp3, you can never convert it back without loss of bits (and therefore quality).

I will get deeper into these concepts in the articles following this one, but its good to know that many of these lossy encoders are using a psychoacoustic model of human hearing to predict what elements of the signal may be masked by our hearing system (usually the masking occurs as a result of critical bands on the basilar membrane… I guess I should give an introduction to the human hearing system as well… noted).  If part of the signal of the original file will likely be masked by our hearing system, then technically, the encoder could lower the volume (speaking in bits) of the masked sound without a difference in the final sound.

Sorry, getting into some wooly passages. I’ll try to simplify with an exaggerated example:

Imagine you have a tune where a 170′ diameter gong gets smashed by the back end of Paul Bunyan’s axe (of course, by Paul himself).  Right after the initial attack of the gong, someone lightly clangs some finger cymbals.  Yes, some of the high frequency information from the finger cymbals may get picked up by a mic in the room, but it will be overwhelmingly covered by the sound of the massive gong.  The encoder (based on psychoacoustic modeling) knows this and can lower the number of bits it is using to encode the sound of the finger cymbal.

So, in a simple summation, the goal of the lossy compression encoders is to predict which sounds we won’t hear, and make them quieter (less bits).

In the next few articles, I’ll get into mp3, AAC, Ogg Vorbis (maybe a few others)… and then… FLAC! (which is a lossless audio compression format)

That’s all for now…

C

The Studio Files