When the "paradox of choice" is true

A popular book called The Paradox of Choice argues that the more choices we're confronted with, past a certain point, the more miserable we become because we spend too much time trying to analyze each possible outcome. That gets to be burdensome beyond a small number of options.

For example, if the only choices for coffee are black, with cream, with sugar, and with cream and sugar both, it doesn't stress us out to analyze each and feel content that we've made a good choice. Once there are 50,000 different coffee-based drinks on offer, though, we waste time trying to gather information about each one, or at least imagining to ourselves what each might be like. Even after we've made our choice, we're stung with regret about not having tried one of the other choices that was up there in the ranking -- maybe you should have had the mocha java chip frappuccino with a shot of espresso and the hazelnut syrup rather than with the raspberry syrup.

Wasting more time and still winding up unable to enjoy our choice as we wonder whether one of the others was really better -- sounds like a bad deal! Lots of criticism has followed the book, like how most people don't really sit in front of the Tazo tea section of the supermarket and agonize for hours which one to get -- "Hey, this looks neat, why don't we go with this one?" -- and that, Woody Allen types aside, they generally aren't so neurotic that they poison their happiness with doubts about all those other unrealized choices.

Still, in defense of the basic idea, there is one class of examples where having more to choose from really does ruin the overall experience. I'll use the example of listening to music on an iPod, although there are similar cases too.

Instead of the slight-of-hand way of doing science, where we make a prediction and see if it's true -- and we only do this if we already knew our prediction would come true -- let's start with a simple observation. Compared to people who listened to music on a Walkman or portable CD player, which could have held maybe 20 to 30 songs at most, users of the iPod are much more likely to be futzing around with the damn thing, clearly unhappy with the current song and trying to scroll through their playlists to find something good. (Most current iPods can hold on the order of thousands of songs, though who knows how many the typical user has -- at least on the order of hundreds, though.) So we notice right away that the user's operation of the iPod is a lot less carefree, and that the quality of the typical song is not very good, compared to portable systems that only hold 20 songs (or maybe 40 if they brought a second tape or CD).

This is actually very easy to model. I'm going to refer to the alternative ways of listening as the limited system (that only holds 20 songs) and the wide-open system (that holds 2000), just to focus on the paradox-of-choice differences. Let's even assume that the two systems are the same in all other respects -- i.e., they both play mp3s, both have the same sound quality, etc.

First, the user is constrained in the amount of time they have to listen to music on-the-go, and this time will be the same for users of either system -- using the wide-open system doesn't somehow give you more hours in the day to listen to music. The average length of a song will be the same for both systems, so the total number of songs you get to listen to during your excursion will be the same. Just to make it concrete, let's say you have 1 hour to listen to music -- while out running errands, walking / jogging around the neighborhood, commuting, whatever -- and the average song is 3 minutes long. That means you'll listen to 20 songs, no matter which system you use.

Second, whether you like a particular song is not set in stone -- it often depends on the mood you're in, what activity you're doing, etc. To simplify, let's say you're just rating a song as either like or don't like. A certain song then has some probability of being a good song ("I like it") when it actually streams into your headphones -- you never know for certain whether you'll dig it or not in that exact moment, in that context, in that mood, etc. When we look at the entire collection of songs you're bringing with you, there's some probability that a randomly chosen one will be good.

Last, as you add more and more songs, at least beyond some point, the quality of those songs will start to fall off. In other words, if you only have room for 20 songs, you're going to be pretty choosy and make sure that they're ones that almost always please you. If you have room for 2000, you'll likely have those in the playlist, but you'll also have quite a lot of filler -- these are the songs we see iPod users skipping through trying to find the great ones. So, the chance that a randomly chosen song on the limited system will be good is greater than the chance that a randomly chosen one from the wide-open system will be. The chance could be pretty close to 100% for the limited system, given how selective you'll be, and it could be as low as 10% for the wide-open system (that's with 200 great songs and 1800 filler songs).

Put all of these pieces together. The number of songs listened to is the same for both systems, yet the probability of finding a good song is much higher on the limited system (because of diminishing marginal returns that set in when you dump 2000 songs into your iPod). Therefore, the expected number of good songs you'll hear -- total number heard times probability of there being a good song in the collection -- is greater for the limited system. Being more realistic, scrolling and skipping through all those filler songs probably eats up enough time that you only get to listen to 19 or 18 songs instead of 20, so that drags down the expected number of good songs even further for the wide-open system.

That's just looking at the benefits, but the costs of setting each system up with songs are the same. Grabbing a CD off the shelf, or grabbing a digital album / playlist of 20 songs onto your iPod, takes no more effort than it does to move 2000 songs onto the iPod. You might think it would take more effort for the limited system since those users have to be more choosy, but because they've already had so much experience with their music, they already know which albums or playlists are superior to their other ones. These are the ones that they'd list if you asked them to name an album they'd bring on a deserted island if they could only bring a handful of them.

Again, I didn't start with a model and see if the real world prediction came true -- we already know how distracted iPod users are by having to schlep through their infinitely long playlists to find one measly good song, and then having to repeat this once the good song is over! The model just makes sense of what we've already observed. To reiterate the three points of the model: 1) we're tightly constrained by time in the number of things we can sample, 2) these things only have a probability of being good, and 3) the probability that a randomly chosen thing will be good declines as we add more things into the selection pool (past a certain point anyway). Thus, the more choices we have, the more miserable we are (again past a certain point).

OK, big deal -- does it apply to anything other than iPods? Well, that is a big deal all by itself given how prevalent they are, and how widespread the delusion is that they offer a superior music listening experience over older ways like a portable CD player or a CD player in your car. Clearly people buy iPods not for a good listening experience but for something else -- to fit into their cultural tribe, to show they're in step with fashion, that they appreciate pretty-looking gizmos, etc. That's the source of the kicks they get from using an iPod.

I think this will apply to digital readers, if they ever catch on. That would be more in the context of reading a book on an airplane trip, where time is tight -- not necessarily using it for your general reading purposes. Imagine you had time to read 5 short stories. If your reader only held 5 short stories, you'd pick some really good ones right before the trip and be happy. If your reader held 5,000 as you moved a "My short stories" folder onto the reader's memory, you'd spend most of the flight getting a little into each one and then skipping on to another one that you'd hope was better, just like the iPod user.

Same if you are packing your clothes for a 5-day trip. That's only 5 outfits you'll be wearing, so if you just pack 5, they'll be nice and you'll be set. If you bring more luggage to fit 50 outfits that you'll choose from on-the-fly, you'll spend most of each day shuffling from one outfit to another before finding one that you like.

This also shows why the paradox of choice doesn't necessarily paralyze us before the dazzling variety of stuff at the supermarket. It's not a given that a supermarket that only gives you two kinds of butter to choose from rather than 20 has a higher probability of having a good butter on offer. That was true for mp3s because you made that selection yourself, based on having already experienced the songs for awhile before. From your point-of-view in the supermarket, all of the butters have some smallish chance of being great, so now in order to maximize the number of butters that you'll really dig, you want them to carry as many as is feasible for them. Say that any type of butter had only a 10% chance of turning you on. Then a supermarket that carries 20 butters is expected to have 2 that you'd love, while the one that carries only 2 butters is expected to have 0.2 that you'd love -- that is, not even one. And you're not so pressed for time either -- sure, you'll die some day, but you'll have enough time to try all 20 types of butter at the more varied supermarket, so you'll find the two or so that you really dig.