Clay Shirky on the Need for Better Information Filters

Last month, Clay Shirky gave a presentation with the title "It's Not Information Overload. It's Filter Failure" at the Web 2.0 Expo. ((Web 2.0 Expo, co-produced by TechWeb and O'Reilly Media, "is a global annual gathering of technical, design, marketing, and business professionals who are building the next generation web. Web 2.0 Expo features the most innovative and successful Internet industry figures and companies providing attendees with examples of business models, development paradigms, and design strategies to enable mainstream businesses and new arrivals to the Web 2.0 world to take advantage of this new generation of services and opportunities.")) Shirky admits up front at the start of the talk that the topic is something new that he is exploring, and as a result the ideas are not fully formed. (I get lost in how the last of his three examples applies to the topic at hand, for instance.) But his viewpoint is a refreshing way to look at the issue of "information overload" from a new perspective, and it is worth looking at even in this raw stage. For starters, he says that we've been facing information overload for the past 500 years -- since the introduction of the Gutenburg movable type press gave readers more books than they could possibly read. What has changed in the last decade has been how past information "filters" are no longer effective.

Shirky posits that the expense of printing a book made publishers both the creators of the object and filters for information printed in objects. The relatively high up-front costs of producing the book meant publishers in the position of selecting only the best information to print. Publishers were, in effect, a kind of filter of quality for the onslaught of information as a way of reducing their risks of printing content that no one would want to read. The internet has driven the cost of publishing to near zero, and as such the "pre-publication" filter that publishers provided is no longer in place. (He calls this "post-Gutenburg economics.) In Shirky's words, "the filter for quality is way downstream from the site of production."

Shirky points to some examples of filters and talks about their effectiveness. For inbound communication, the example is e-mail spam and how spam filters must be constantly tuned. This is a pretty clear example of what he is talking about -- the cost of production is cheap and the assessment of quality is done by the reader, not the producer. The second example is one of outbound communication; Shirky tells the story of a colleague who attempted to use Facebook privacy settings to slowly disseminate the fact that she had broken up with her colleague. (That isn't what happened. P.S.: Karen Schneider -- your name pops up briefly in one of Clay's screenshots!) The third example is that of a student that faced expulsion from a Canadian university because he started a Facebook homework group. Shirky's point with this example seems to be that a filter-of-inconvenience was removed through the use of technology -- that a study group of 147 students wouldn't actually occur in real life but was replicated on Facebook.

Some other quotes that caught my ear:

• "Managing your privacy practices is an unnatural act... Privacy is a way of managing information flow... The big question we're facing around privacy now is that were not moving from one engineered system to another engineered system with different characteristics. We're moving from an evolved system to an engineered system."
• "The inefficiency of information flow wasn't a bug, it was a feature. That's what privacy was."
• "What the internet does is allows large systems that are free-rider tolerant rather than free-rider resistant."
• "It really is about rethinking the [higher education] institutional model. You have to have group conversation. You have to have individual effort. You have to design a system that accommodates both."
• "If you have the same problem for a long time, maybe it's not a problem. Maybe it is a fact." --Yitzhak Rabin