Do Specifications Lie? Do Our Ears Lie? Where does the Truth Lie? An Examination of Headphone Amplifier Performance Specifications

Recording Engineers and Audiophiles often distrust audio measurements and specifications. It is not uncommon to hear claims that a product measures poorly but sounds good. Occasionally we also hear claims that a product measures well but sounds bad. This whitepaper documents significant differences between three headphone amplifiers that have nearly identical published specifications. Each is equipped with a high-quality low-jitter digital to analog converter, and each can accept 96 kHz 24-bit digital audio. All three products have a high-quality headphone output with very low output impedance, high drive capability, low distortion, and a flat frequency response. The published specifications strongly suggest that it should be impossible to hear any difference between these products. Nevertheless, in our informal listening tests, we had the distinct impression that there were significant sonic differences between these products when listening to music.

We replicated the manufacturers' tests and found that the published specifications were accurate and correct. All three manufacturers followed standard measurement procedures and provided accurate specifications.

Figure 1 shows a THD+N vs. frequency sweep of each headphone amplifier. This test confirms the excellent specifications published by each of the manufacturers. In this test there are no significant performance differences between the three products. The THD+N for unit 1 (Benchmark DAC1 HDR) does measure about 9 dB lower than the other two units, but the THD+N should be nearly inaudible in all three units. If we were to stop here, it would be easy to conclude that all three headphone amplifiers provide similar performance.

Figure 2 shows that all three headphone amplifiers have a nearly perfect frequency response. Amplitude variations are so small that it should be impossible to hear any differences. Again, this test fails to identify any significant difference between the three headphone amplifiers.

In search of an explanation for the sonic differences, we conducted an extensive series of additional tests. We added a 60-Ohm resistive load to simulate headphone loading. With the load applied, we began to see major differences in the three headphone amplifiers.

Figure 3 shows the same test as Figure 1, but with a 60-Ohm resistive load added (to simulate a pair of headphones). Units 2 and 3 clearly have difficulty driving a 60-Ohm load at 500 mW. In contrast, the Benchmark DAC1 HDR remains nearly distortion free when driving high-level signals into 60-Ohms.

Our next step was to replace the ideal 60-Ohm load with a pair of 60-Ohm headphones. After all, why not test the headphone amplifier while driving a real pair of headphones? We selected the Sony MDR-V6 headphones because they are a popular pro-audio headphone with a 60-Ohm input impedance (at 1 kHz). As soon as the headphones were connected, we began to measure more differences between the three units.

Figure 4 is the same test as shown in Figure 3, except that the headphone amplifier is now driving 60-Ohm Sony MDR-V6 headphones. All three headphone amplifiers have enough power to drive the MDR-V6 headphones at their 500 mW rated power. Nevertheless, units 2 and 3 were unable to fully damp the mechanical resonances of these popular headphones. Consequently, units 2 and 3 show significant distortion at low frequencies. The low-frequency distortion in unit 2 is 31 dB higher than the DAC1 HDR. Unit 2 also has difficulties at specific mid-range frequencies (note the peaks near 2 kHz and 3 kHz). In contrast the DAC1 HDR shows virtually no change in performance when compared to the no-load test shown in Figure 1, or the 60-Ohm test shown in Figure 3. The Benchmark DAC1 HDR is nearly distortion free when driving this difficult load at very high signal levels.

However, the 500 mW test shown in figure 4 does not accurately represent a typical listening situation. 500 mW into MDR-V6 headphones produces a sound pressure level that is sufficient to cause permanent hearing damage. Nevertheless this test is useful in that it shows potential problems that we should look for when testing at normal listening levels.

To determine a "normal" listening level, we played several commercial CD pressings while adjusting headphone volume for a comfortable listening level. This tended to be near the level that produced 1 mW peak into the Sony MDR-V6 Headphones. This was convenient since Sony's published specifications show that the MDR-V6 headphones produce 106 dB SPL at an input power of 1 mW. This means that our "normal" listening level will produce 106 dB SPL peaks. We do not necessarily recommend this level for extended listening, although the suitability of this level is largely determined by the amount of compression applied to the recording. This level may be too low for many classical recordings, but may be too high for many pop recordings. Overall it represents a reasonable level for simulating typical use.

We continued the testing at a signal level of 5.56 dBu (which produces 1 mW into MDR-V6 headphone at 1 kHz, and 106 dB SPL at the ear cuffs). The load was provided by the MDR-V6 headphones and not by an ideal 60-Ohm resistor (unless otherwise noted).

Our "normal" listening level produces peaks that are 106 dB above the threshold of hearing. Therefore, defects that are -106 dB (or lower) should be inaudible at this listening level. In general, noise that exceeds -106 dB (0 dB SPL) should be audible. In most cases, distortion will need to exceed 0 dB SPL by a significant amount before it will be audible. Distortion only exists while music (or test tones) are playing. Consequently, music tends to mask much of the distortion. It is fair to say that distortion must be significantly higher than -106 dB (0 dB SPL) before it will become audible.

When all three headphone amplifiers are adjusted to produce -5.56 dBu at 0 dB FS (our "normal" listening level), there are remarkable differences in the performance.

Figure 5 shows the large performance differences that were measured under normal operating conditions (MDR-V6 headphones attached and playing at a typical listening level). The plot shows that Units 2 and 3 have poor signal to noise ratios(SNR) at normal listening levels. The SNR of units 2 and 3 is only 73 dB at typical listening levels when using Sony MDR-V6 headphones, and would be worse when using headphones with higher sensitivity. Conversely, units 2 and 3 would improve by 11 dB when driving Sennheiser HD650 headphones. Even with the low-sensitivity Sennheiser headphones, units 2 and 3 would be limited to 84 dB (not even CD quality).

The loss of SNR shown in Figure 5 is due entirely to the volume control circuitry used in units 2 and 3. There is no question that the noise produced by units 2 and 3 is audible, and it is easy to hear when wearing the headphones. The noise is 33 dB above the threshold of hearing under normal listening conditions! In contrast, the Benchmark DAC1 HDR maintained a 96 dB SNR due to its analog gain control, and produced noise that was only 10 dB above the threshold of hearing. The DAC1 HDR is 23 dB quieter than the other two units at typical listening levels. Remember, all three units have similar performance specifications at maximum volume levels, but units 2 and 3 cannot deliver this performance in a typical listening situation.

Another problem is evident in Figure 5: Unit 2 (green curve) shows very high distortion at low frequencies, while the Benchmark DAC1 HDR (blue curve) shows no rise in distortion. What this graph cannot show is that unit 3 (magenta curve) also has a significant low-frequency distortion problem. The high -73 dB noise levels at the output of unit 3 make it impossible to see the distortion in this traditional THD+N plot. We must filter out the noise and plot THD to see the differences. Our ears are very good at performing this function. We are able to hear tones, music, and distortion at levels that are up to 30 dB below the level of a white noise signal. Fortunately, we now have audio analyzers that can remove noise and measure THD (instead of measuring THD+N).

Figure 6 shows a THD vs. frequency plot under the same conditions as Figure 5 (peak SPL is 106 dB).Please note that we are plotting THD, not THD+N. Noise has been removed so that distortion (THD) can be plotted without being obscured by noise. This graph shows dramatic differences in distortion between the 3 amplifiers. The Benchmark DAC1 HDR is essentially distortion free, while the other two headphone amplifiers have significant levels of low-frequency distortion. The distortion in unit 2 is 35 dB above the threshold of hearing. The distortion in unit 3 is 29 dB above the threshold of hearing. There is a high probability that the distortion in units 2 and 3 is audible. In contrast, the DAC1 HDR maintains distortion at or below the threshold of hearing.

In our listening tests, we had the distinct impression that the Sony MDR-V6 headphones sounded much cleaner when driven from the DAC1 HDR. There was also an obvious difference in noise levels. At the typical listening levels used in Figures 5 and 6, the noise from amplifiers 3 was 29 dB above the threshold of hearing, and was clearly audible. Amplifier 3 measured even worse (34 dB above threshold). Under the same conditions, the noise from the DAC1 HDR was only 10 dB above the threshold of hearing and was nearly inaudible. The DAC1 HDR is 18 to 23 dB quieter than the other two units when operating at typical listening levels.

In summary, we tested three high-quality pro-audio headphone amplifiers with built-in D/A converters. All three had similar published specifications. All units are priced between $1000 and $2000. We verified that the manufacturer's specifications were accurate, but we have shown that these published specifications are not sufficient to tell the whole story. In this case, the published specifications were not a good representation of typical operating conditions. Under typical operating conditions, unit 2 and unit 3 suffered from elevated noise levels. These same two units also suffered from distortion problems when driving the difficult load presented by a pair of headphones. In contrast, the Benchmark DAC1 HDR was able to maintain its performance over a wide range of operating conditions. All three products sounded very good at normal listening levels, but differences were noticeable. Our measurements, taken under typical listening conditions, confirmed that audible differences should exist. The published specifications did not lie, the manufacturers did not lie, and our ears did not lie. The truth lies here: Specifications must be measured under typical operating conditions if they are to be useful in predicting audible differences.

Test Conditions for Published Specifications (showing identical performance):
1. High-impedance load (100 K-Ohms)
2. Maximum or near-maximum output level (21 .4 dBu)

Test Equipment: Audio Precision System 2722

Devices Tested:
• Unit 1 - Benchmark DAC1 HDR - D/A Converter with Headphone Amplifier - MSRP = $1895 *
• Unit 2 - D/A Converter with Headphone Amplifier - MSRP = $1695
• Unit 3 - D/A Converter with Headphone Amplifier - MSRP = $1148

* The test results for Unit 1 also apply to the Benchmark DAC1 PRE - MSRP = $1595

Copyright© 2011 Benchmark Media Systems, Inc.