Reluctance to read new authors, '80s vs. the 2000s

This NYT article discusses the smashing success of the new book The Cuckoo's Calling -- which only started selling like mad once it was revealed that the author was none other than J.K. Rowling, founder of the Harry Potter colony.

Several sources are quoted to the effect that a book needs instant brand recognition these days in order for it to make it big time. They all avoid pointing their finger at today's conformist risk-averse audiences, and treat it like some new constraint on the industry that came from who know's where, but that we've all just got to adapt to.

You see the same thing when music, movie, and TV people talk about the same phenomenon in their own industries. No one wants to see the big picture because blaming the consumer would be bad for business. Still, you'd think at least the critics would point this out -- that folks today are so disturbed by novelty that they won't take a chance on anything unfamiliar. It's not that they're afraid. More like suspicious, not trusting... but also OCD. "Wait, there's something new here -- that messes up the old inventory!" It's like being a picky eater -- OCD to the max.

We're supposed to live in a fascinating new era where so many options are at our fingertips, and everyone is dipping into this and then dipping into that -- a crazy world where no one commits to anything. In reality, people these days rigidly adhere to what is familiar, forever. The utter lack of novelty and variety, the never-ending reign of blandness, is a world apart from cultural life in the good old days.

To put some hard numbers on this, let's stick for now with book publishing (although the same approach will work for pop music or movies, and I'll get around to those later). We want a statistic that will capture how dominant the top authors are over time, vs. how easy it is for new authors to break into the big leagues. So we'll look at Publishers Weekly list of best-sellers by year. Dominance, hegemony, etc. -- these all refer to a certain author appearing time and again, or having multiple hit books in a single year.

Therefore, we'll look at a consecutive 5-year period, and take the top 10 best-sellers in each year. Then ask: how many distinct authors are represented among these 50 best-sellers? Or even better, how many authors had just 1 hit, 2 hits, 3 hits, and so on? Then summarize that distribution with something like the average or median number of hit books by an author of that time period.

Ideally, I'd do this for every 5-year period in the data source. But this is just a pilot study to quickly prove a point (how much things have changed since the '80s). Let's contrast the period of 1984-1988 to 2004-2008. Only 20 years apart, not even a full generation. Mature industries like book publishing couldn't change very much in so little time, right?

Well, in the period centered on 1986, there were 28 distinct authors who wrote the top 50 books, while in the period centered on 2006, there were only 22. The average author in the '80s period could have expected to have 1.8 hit books, while his counterpart in the 2000s period would have had 2.3 hit books. Even looking at the median, which is much more resistant to change than the average, increased from 2 books to 3 books by best-selling authors. In the '80s period, only 2 authors had more than 3 hit books; by the 2000s period, it had risen to 6 authors who were that dominant.

The maximum was, however, the same in both periods -- 7 hit books (Danielle Steel claimed the prize in the '80s, James Patterson in the 2000s). The mode (or most common) number of hit books in both periods was also the same -- 1 book. As in other domains where eminence has meaning, the distribution is geometric -- most have only 1 hit, fewer have 2, fewer still have 3, and so on, until there's only a small elite who have lots of hits.

So, indeed readers have become more reluctant to take a chance on new authors since the 1980s. It would probably look even worse if I chose the late 2000s/early 2010s as the end-point. You'd see the same thing if you looked at how many artists are responsible for the top 10 songs across a 5-year period from the '80s vs. the 2000s. And the same for how many movie stars were in the top 10 movies at the box office. There are so few fresh faces these days.

The fact that these trends span so many different industries means that we can dismiss any explanation that is idiosyncratic to an industry. My take is that this is just one piece of the larger pattern of the decline in novelty-seeking since the early-mid 1990s. That has both a physical side, where people don't go out joyriding, dancing (with another person), having casual sex, or hanging out in public places anymore. But even private consumption shows the pattern, where anything unfamiliar is either suspicious or throwing the existing order outta-whack. I can't tell whether mistrust or OCD is the dominant factor here. Whatever the case, it goes to show that cocooning is not only a spatial phenomenon.

Finally, I suspect today's pattern would also show up during the mid-century, another period widely regarded as one of cultural stultification. The approach used here would let us figure out whether that stereotype is deserved, or whether the Fifties just gets a bum rap. I plan to look into this in the future, for all three major industries (books, music, movies) that have some kind of "best-selling" lists. Stay tuned.