Searching for Red Stockings: The Myth of Information Overload

As the internet advocate Clay Shirky noted, everybody who talks about information overload starts with the graph with the telltale ascending line and the litany of the troubles it entails. As the line informs us, information is increasing exponentially, and we can barely deal with it intellectually and emotionally, or more and more often, we can’t. And the solution?  It is here that the rallying cries diverge. 

Scary Graph  (from Basex.com)

On one side there is Shirky, who assigns the problem to filter failure, and why not? It’s a reasonable thing after all to suppose that if we had better ways to sort out information, we could cull the bad from the good, and be able to significantly reduce the information we have to cope with daily.  Search, social media, and e-commerce firms of course concur, and are rapidly improving their search algorithms (using of course information about you that you voluntarily or involuntarily port over to them) so  you can find what you need the first time.

On the other hand is the internet critic Nicolas Carr, who attributes information overload to filter success. In Carr’s opinion our filters are working all too well, and the problem is that they are getting better and better. 

Thus,

“….The real source of information overload…. is the stuff we like, the stuff we want. And as filters get better, that's exactly the stuff we get more of.

It's a mistake, in short, to assume that as filters improve they have the effect of reducing the information we have to look at. As today's filters improve, they expand the information we feel compelled to take notice of. Yes, they winnow out the uninteresting stuff (imperfectly), but they deliver a vastly greater supply of interesting stuff. And precisely because the information is of interest to us, we feel pressure to attend to it. As a result, our sense of overload increases.”

Implicit in both arguments is this premise:

The information we want is the same as the information we need.

This is an argument for the curing salve of better filters (to fine tune what we want, since our wants are finite) or a call for mass despair (because our wants are infinite, and thus overwhelm us when they are invariably served by the web). This premise derives from an assumption that in our hubris we are wont to make: that humans are rational agents who know what they want and why.

But what if this was not true?  What if we are at root irrational creatures who delude ourselves into thinking that we know what we want and why we want it? What if the information we want is more often than not different from the information we need? If this is true, then to paraphrase Shakespeare, our fate is not in the stars (or rather the cloud), but in ourselves, because if the information that we want is often not the same as the information we need, then we need to be aware of how to distinguish our wants from our needs and how and when  to constrain the former.  In other words, for information overload, the key is to understand how our basic motivations work.

The question that Shirky and Carr beg is thus elemental: Why is information of interest to us, because it is important, or because of something else?  To answer this question, let us illustrate how a basic search was performed over the last few generations by going to our metaphorical sock drawer in search of red stockings.

It’s 1912, and you as t-shirt manufacturer want to begin a production run of commemorative t-shirts of the Boston Red Stockings triumph in the World Series. As soon as the game is over you receive an immediate telegraph of their victory, and it’s off to the races to start production.

It’s 1932, and you as a t- shirt manufacturer want to get started with your commemorative t-shirt run, and so you listen to the game on the radio, and upon its completion, get to work.

It’s 2012, and you as a t-shirt manufacturer want get to cracking on your production run celebrating the Boston Red Sox victory, and you follow the sox from college draft to preseason to all of their games through the World Series, and monitor all the social and news media who have something to say about it.

In all three time frames, the decision point happens in a second at a predetermined moment, namely when (hopefully) the sox win. The narrative of how that final fact (a sox victory in the final game) got there is irrelevant. No matter what era, the decision point is concise, precise, and momentary, and gets to you on time regardless of the media you use and irrespective of its background story.  There is no need to follow the narrative that describes the changing facts that get us to that point, as the point of the last man flying out in the last inning is all we need.

The difference between the three eras is that in the first era we could not follow the narrative that follows the sox on their way to the pennant, but in the latter era we could, thanks to the rapidly declining transaction cost for information that allows us to perceive the changing flow or narrative of information. But following the latter comes at a cost.  By following the progress of the sox we become diverted from other things of value, and suffer regret.  If these diversions are small scale and populate our working day, they become distractions and cause us to lose focus and attention. Finally, as we continually choose between distraction and staying on course, we become tense and nervous.

The metaphor of ‘information overload’ would seem to apply here, as every frame of every moment of the continuous narrative leading to the Red Sox pennant can and is considered by the sox fan. However, like a strip of static frames in a motion picture that give rise to a sense of movement or motion, the story is interesting because of the novel ways the narrative changes, and it is the changes that compel. Thus, although the ending is necessary for us to go about our business, the story that leads to it is compelling not because of what it is, but how it is continually transformed.

We can expand our simple Red Sox narrative to the narratives embedded in all the things we do that are being progressively revealed by the web. We need to know facts, but what obsesses us is the narrative or story that brings us to those facts.  The internet produces not just more information, but more changing patterns of information. We see not a picture, but a movie, not a note, but a score, not a phrase but a speech. Moreover, we conflate the importance of the narrative with the significance of its conclusion, or what we want with what we need. This is a dangerous delusion, for the stuff we want depends upon the narrative or facts in motion, but the stuff we need depends upon the facts sitting still.

We can get the facts we daily need in a half hour, but continually accessing the web to see a moving stock market, an evolving middle east crisis, or what Uncle Charlie is up to are never ending stories that excite us, engage us, but ultimately bring us down. A narrative is of course still important if our behavior necessarily changes in tandem. In this case the narrative is ‘feedback’. Thus, a quarterback’s performance is determined by feedback during the moment to moment course of the game. However, for the stadium audience, this feedback is entertainment, and for those who attend to the ever expanding narrative on the game itself, an unnecessary and harmful obsession.

The Myth of Information Overload

As a metaphor, information overload attributes the psychological effects of the internet to what information is rather than how it is arranged, and ultimately the metaphor of information overload in inadequate because we are not overloaded with information but with ever evolving novel patterns of information or narratives. Because humans are above all novelty-seeking creatures, novelty is enhanced not in the facts but in the stories they tell. Our interest  lies not only in the rational but in the abstract properties of information. So it is not information that overloads, but elemental aspects of information, namely novelty that when produced in infinite abundance by the web leads us to endless distraction, stress, and regret.  Because the explanation for how the web influences us psychologically is based on core assumptions on human motivation that are faulty, namely that we are rational actors, we proceed with our daily lives under a dangerous illusion abetted unfortunately by the perverse incentives of our media providers to keep us hanging onto the story when the conclusion is all we need. Whether or not we can escape this illusion and its dire consequences depends ultimately on not just a better story, but also a better explanation as to how our minds actually work.

Sources:

Finding a better story to describe the emerging fact that wanting and liking aren’t the same thing takes you to the seminal work performed by the neuropsychologist Kent Berridge on the topic, or my own narrative on Berridge’s work and what it means.  Hopefully both make for some interesting explanations.