Headphone Sound Quality: What Do Listeners Prefer?

As mobile technology has evolved, allowing people to carry music, movies, books, and a host of other types of infotainment in their pockets, headphones have become more prevalent. However, in the 2010s, as smartphones became ubiquitous, headphone technology remained stuck twenty years in the past. In this episode, we talk to Sean Olive of Harman International about his research regarding the perception of headphone sound quality and how standards can be updated to better reflect listener preferences. 

Read the associated article: Sean E. Olive. (2022) “The Perception and Measurement of Headphone Sound Quality: What Do Listeners Prefer?” Acoustics Today 18(1). https://doi.org/10.1121/AT.2022.18.1.58.

Read more from Acoustics Today.

Learn more about Acoustical Society of America Publications.

Intro/Outro Music Credit: Min 2019 by minwbu from Pixabay. https://pixabay.com/?utm_source=link-attribution&utm_medium=referral&utm_campaign=music&utm_content=1022

Transcript

Kat Setzer  00:06

Welcome to Across Acoustics, the official podcast of the Acoustical Society of America Publications Office. On this podcast, we will highlight research from our four publications. I'm your host, Kat Setzer, editorial associate for the ASA.  Today I'll be talking to Sean Olive of Harman International about his article, "The perception and measurement of headphone sound quality: What do listeners prefer?" which appeared in the spring 2022 issue of Acoustics Today. Thanks for taking the time to speak with me today, Sean. How are you?

Sean Olive  00:39

I'm fine. Thank you. How are you?

Kat Setzer  00:41

I'm good. So tell us a little bit about your research background.

Sean Olive  00:45

Sure. I'm a Senior Fellow in acoustic research at Harman International, and I'm just celebrating my 30th year conducting research. 

Kat Setzer  00:53

Oh, wow.

Sean Olive  00:53

Yeah, and thirty years goes by fast. So I'm in my 30th year conducting research in the perception and measurement of reproduced sound. We study human sound quality preferences of reproduced sound, and then we model and predict their preferences using acoustic measurements. The research findings are then applied to the design and testing of our consumer, professional, and automotive audio products, which include microphones, loudspeaker systems, and more recently, headphones and spatial audio algorithms. Prior to joining Harman in 1993, I worked with Floyd Toole for six years at the National Research Council of Canada, doing research on the perception and measurement of loudspeakers in rooms.

Kat Setzer  01:37

Okay, so makes sense how you got into writing this article then. So in this article, you mentioned that headphone sound quality hasn't kept pace with consumer demands and expectations. Can you tell us some about the history of headphones and how they've changed, or not changed, over time?

Sean Olive  01:51

Sure. The earliest headphones date back to the late 19th century and had nothing to do with music, but were used for radio communication, telephone operators, and later for military communications. Headphones for music didn't really happened until the late 1950s, when the first stereo LP was invented. And an American named John Koss released the first commercial stereo headphone. I bought my first set of headphones when I went to university in the late 1970s, and I moved into a dormitory and shared a room and I needed a privacy. So having headphones were necessary. But as soon as I moved my second year into my own room, those were replaced by loudspeakers. And I really never used headphones much after that. In the 1980s. I remember buying a Sony Walkman, which came with a pair of  pretty lousy headphones. And it wasn't until really 2002 that I started wearing headphones again when the iPod was invented. And suddenly, because you could carry 1000 songs in your pocket and hear music anywhere, headphones were, you know, a necessary accessory. But the earbuds that came with it didn't sound very good. And then when the smartphone came around 2007, along with music streaming services later on, music listening became a truly mobile listening experience, and we saw headphone sales suddenly explode. In 2006 Beats was one of the newest, one of the new headphone companies that anticipated this this trend in headphone growth before anyone else. And by 2011 they had 64% market share of headphones costing more than $100, which was remarkable. Of course, Beats made headphones very popular through using musician endorsements and excellent marketing. And it became a cool fashion accessory. And that was about the time, in 2012, up until then, I'd mostly focused my research on loudspeakers, but I started listening to and measuring different headphones. And what impressed me was the huge variance in sound quality among the different brands and even models within the same brand.  They were all over the map. I started talking to headphone designers, and no one basically agreed on what makes the headphone sound good or how to measure it. And I looked up the headphone standards, and they had been written in the 1990s and not changed. And it was also clear that you know when you look at headphones and measure them and listen to them, no one was following the current standard curve. There was a lack of published research on headphones, very few published tests on, based on listening tests. So in 2012, we undertook this seven-year project into the perception of measurement of headphone sound quality.

Kat Setzer  04:48

Yeah, that's really interesting. It makes sense that headphones would become more popular as mobile technology advanced. So how do you judge headphone sound quality?

Sean Olive  04:57

So we judge headphone sound quality by using both trained and untrained listeners, and they sit down and listen and compare different headphones while listening to different short tracks of music. They're asked to rate each headphone on a 100-point preference scale, according to how much they like or dislike it. So we get a measure of the overall preference as well as the relative magnitude between two headphones. From that we get a measure of how good it sounds. We also want to know the underlying reason why they like or dislike it. So they'll rate the headphones on different attributes, like spectral or timbre balance, the spatial and dynamic qualities, and whether it sounds noisy or distorted.

Kat Setzer  05:40

Okay, interesting. So let's go to this research project you mentioned before. In 2012, you started the research project to improve consistency and sound quality of headphones. So why was there so much inconsistency? Can you explain how you approached the challenge?

Sean Olive  05:53

Yeah, well. Why was there so much consistency, I believe there was a lack of scientific, really good guidance on how to design a headphone to make it sound good. You know, so the first challenge we had was to, you know, figure out what makes a headphone sound good. And whether most listeners agree, because if everyone has a difference in opinion, then we're not starting on a good foot. So one of the challenges we faced first was was the methodology, how do you do a controlled listening test? How do you make it blind so that you're removing all the psychological and expectation biases related to brand price, visual design, and so on? So we already had this hypothesis that the success behind Beats  was not the sound, but the catchy industrial design, the clever marketing, so that was something we wanted to remove from the test. So we developed a virtual headphone method where we could measure a headphone, and then apply a digital filter to another headphone and replicate the sound. And this allowed us to quickly switch between and compare different headphones without knowing what the brand or model was. And this is what a blind test is. We did a lot of validation testing to show that the correlation between the actual headphone and the virtual version was very high. And the correlation was somewhere between .86 and .98, depending on the type of headphone, whether it was over the ear or in ear. And then once we perfected this virtual headphone method, we were able to conduct tests very, very accurately that gave us accurate and repeatable results.

Kat Setzer  07:26

So what is a headphone target curve?

Sean Olive  07:28

So a headphone target curve is a defined frequency response of a headphone that's been shown to satisfy the preferences of the majority of listeners. And I probably should define what the frequency response is. It's basically the level or the amplitude response of a device measured in decibels. So decibels gives you a measure of how loud or what the level is, and it's measured at different frequencies across the audio, audible range from the lowest note, which is generally 20 Hertz, to the highest note or where the highest overtones occur, at 20 kilohertz. So in audio products, the general goal is to have a flat frequency response, which means the amplitude is the same across the entire audio bandwidth. In that way, recorded music is accurately reproduced without altering the level of the various pitches and overtones. 

Kat Setzer  08:22

Okay, that makes sense. 

Sean Olive  08:23

But I was gonna say, the target curve is measured on an industry standard test fixture, which simulates the anthropometric and acoustic features of the human head, outer ear, and ear canal. And because of that, when you see the target curve on this device, it includes the ear canal resonance of the human ear, and it's not flat. So you have to take that into account when you're looking at headphone measurements. 

Kat Setzer  08:52

That's really interesting. Okay, so can you tell us some about the target curves that have been developed over the years?

Sean Olive  08:58

Sure, in the 1970s, the most common headphone target curve was the free-field curve. And a free field is a space where there's no reflections, like being outdoors or in an anechoic chamber. And the free-field frequency response is measured by putting a flat loudspeaker-- flat, meaning it has a flat frequency response in the free field-- and you measure it at the listener's ear, eardrum reference point, which is right where the eardrum is. In this case, the loudspeaker was positioned directly in front of the listener, and what it produces is a frequency response that tends to be very bright and very thin or lacking bass. And since the response is based on a single sound source, the loudspeaker, from only one direction, it tended to sound very unnatural, because most music is enjoyed in reflective acoustic spaces that enhance the bass and the first arrival, or the direct sound, is supported by reflections that arrive at the ear from many directions.  So in the 1980s, there was an alternative target proposed, known as the diffuse-field target. And this puts the loudspeaker in a reverberant field, the total opposite of what a free field is. And the theory behind it was that the frequency response fits based on sounds arriving at the ear from all directions, it'll produce a sound that, a headphone that sounds more natural, and produces a more externalized sound because you can't associate sound from just one direction, which was the case with a single loudspeaker in the free field. So this standard became sort of an international standard in the 1990s. And it remains the standard today. But as our research shows, most manufacturers don't follow it. And the research that we've done, and others, show that most people prefer alternative targets.

Kat Setzer  10:49

Well, and that actually probably leads us really well into our next question, which is, what is the Harman Target Curve? How was it developed, and how does it compare to the previous target curves?

Sean Olive  10:58

Yeah, so the Harman Target Curve is based on the frequency response measured at the eardrum using an accurate loudspeaker in a semi-reflective room. So this is a room that is somewhere in between the free-field and the reverberant field, or the diffuse-field curve. And the hypothesis behind it was that headphones should sound like this, because 99% of all recordings made today are mixed and mastered through these systems. They're not, they're not made or designed to sound good through headphones. So what we did was we went into our reference listening room, which is a semi-reflective room, we put in two accurate loudspeakers and we measured the frequency response at the eardrum, and that, in essence became the Harman Target Curve. So it simulates the response of a loudspeaker in a room. And we went on and did further experiments where people could make slight adjustments to the bass and the treble of that target. But essentially, it's a, it's a loudspeaker in a room.

Kat Setzer  12:05

Okay, that totally makes sense. So how did you figure out what people prefer in headphone performance? And how does the Harman Target Curve relate to this?

Sean Olive  12:13

So we did... So once we had this Harman Target Curve, we did a lot of listening experiments. The first ones were involving listeners comparing this target to other targets. So that included the diffuse-field as well as the free-field curve. And they made comparisons listening to music tracks, and rated each one, and on average, we found that people prefer this new target curve over the existing standard as well as previous standards. And of course, we also compared it to many different models of headphones that were popular on the market at that time.

Kat Setzer  12:54

So were there any noticeable differences in individual headphone sound preferences?

Sean Olive  12:59

Yeah, so one of the studies we did involved tests in four different countries, Canada, US, Germany, and China, involving over 300 listeners. And we looked at factors such as age, gender, listening experience, and how these these factors influence their preferences. And what we generally found was that listeners fall into three different segments based on taste. 64% of the listeners preferred headphones tuned to the Harman Target, and then there were two smaller segments that preferred the Harman Target with more or less bass and treble, and there was a small segment 15% of the sample preferred more bass, and these tended to be younger males. While the people who preferred less bass, this was roughly 22% of the segment of the population, they tended to be females and older listeners. And we think that the older listeners, this could be related to age-related hearing loss, which affects high frequencies first, and by reducing the bass and increasing the treble, this would help enhance speech and vocal intelligibility.

Kat Setzer  14:08

You mentioned ways of predicting listeners' headphone preferences. So how would you do this? And why would you want to?

Sean Olive  14:13

So we found that the frequency response of the headphone was a very strong predictor of how listeners would rate its sound quality, and the more the frequency response deviated from the Harman Target Curve, the lower the headphone was rated, so we came up with a simple model based on these deviations in frequency response. And we were able to predict how it would score in a listening tests based on these deviations with a correlation of .86 between the predicted and the observe preference ratings. And the erroor on a  100-point scale was about 6.7 of a rating, which is roughly the equal to the error of our, of our listeners when they give ratings.  So why would we want to do this? Well, being able to predict the headphone sound quality based on this model saves us a lot of time, effort, and money conducting these very large listening studies, which can take, you know, weeks and cost thousands of dollars. In comparison, the measurement and prediction based on a measurement takes about 10 seconds. So that's a very useful tool when you're, throughout the design and manufacturing of a headphone, because you can quickly make a change and predict how it's going to impact the sound quality.

Kat Setzer  15:28

Okay, yeah, that sounds absolutely useful. Do you have any concluding thoughts?

Sean Olive  15:32

Yeah, I mean, I think the last ten years, there's been a lot of research, there's been a lot of interest in our research. And I think, in part, this has led to a lot of progress in the quality and consistency of the sound of headphones. I've noticed that over the ten years, headphones are becoming better. And they're more similar in how they measure and sound. So that's a good thing. But I see in the future headphones are starting to merge with other devices. they're starting to have some of the functionality of hearing aids, and we're seeing headphones including biometric sensors to measure your health. And it's clear that I think headphones will become something that we perhaps wear all day. And they're becoming more and more personalized to our tastes, to our hearing, to our, you know, our head-related transfer functions, which affect how we localize sound. So I think there's exciting things happening in the next few years in headphones. They should be interesting.

Kat Setzer  16:38

That's very exciting and interesting to think about how headphone technology will change over time. Well, thank you again for chatting with me today. I never realized how little consistency there was among headphones, and I'm sure many of our listeners will appreciate your work to help improve sound quality in headphones. Have a great day.

Sean Olive  16:55

Yes, thank you very much.

Kat Setzer  16:59

Thank you for tuning into Across Acoustics. If you'd like to hear more interviews from our authors about their research, please subscribe and find us on your preferred podcast platform.