Data Mobility Group, LLC - High Definition Analytics and Technology Market Insight

Are we prisoners of our preconceptions?

March 12th, 2011

We wish to express our gratitude to Jonah Lehrer for his permission to share an excerpt from Chapter 7, The Brain Is An Argument, in his 2009 bestseller “How We Decide“.  To fully understand the context of the excerpt I recommend that you read the book.

Mr. Lehrer wrote, “There’s something unsettling about seeing the brain as one big argument. We might like to believe that our decisions reflect a clear cortical consensus, that the entire mind agrees on what we should do. And yet, that serene self-image has little basis in reality.”

Having spent the last ten years in an industry filled with pundits serving clients who have equally large egos and preconceptions, Lehrer’s coverage of Philip Tetlock’s research results struck a chord with me. I hope to revisit this topic in a future post and discuss how it affects decision-making and our willingness to consider/accept the opposing views and opinions of others.

In 1984, the University of California at Berkeley psychologist Philip Tetlock began what he thought would be a brief research project. At the time, the Cold War was flaring up again—Reagan was talking tough to the ‘evil empire’—and political pundits were sharply divided on the wisdom of American foreign policy. The doves thought that Reagan was needlessly antagonizing the Soviets, while the hawks where convinced that the USSR needed to be aggressively contained. Tetlock was curious which group of pundits would turn out to be right, and so he began monitoring their predictions.

A few years later, after Reagan left office, Tetlock revisited the opinions of the pundits. His conclusion was sobering: everyone was wrong. The doves had assumed that Reagan’s bellicose stance would exacerbate Cold War tensions and had predicted a breakdown in diplomacy as the USSR hardened its geopolitical stance. The reality, of course, was that the exact opposite happened. By 1985, Mikhail Gorbachev was in power.  The Soviet Union began implementing a stunning series of internal reforms. The ‘evil empire’ was undergoing glasnost.

But the hawks didn’t do much better. Even after Gorbachev began the liberalizing process, hawks tended to disparage the changes to the Soviet system.  they said the evil empire was still evil; Gorbachev was just a tool of the politburo. Hawks couldn’t imagine that a sincere reformer might  actually emerge from a totalitarian state.

The dismal performance of these pundits inspired Tetlock to turn his small case study into an epic experimental project. He picked 284 people who made their living ‘commenting or offering advice on political  and economic trends’ and began asking them to make predictions about future events. He had a long list of pertinent questions. Would George Bush be reelected? Would there be a peaceful end to apartheid in South Africa? Would Quebec secede from Canada? Would the dot-com bubble burst? In each case the pundits were ask to rate the probability of several possible outcomes. Tetlock then interrogated the pundits about their thought processes so he could better understand how they’d made up their minds. By the end of the study, Tetlock had quantified 82,361 different predictions.

After Tetlock tallied the data, the predictive failures of the pundits became obvious. Although they were paid for their keen insights in world affairs, they tended to perform worse than random chance. Most of Tetlock’s questions had three possible answers; on average, the pundits had selected the right answer less than 33 percent of the time. In other words, a dart-throwing chimp would have beaten the vast majority of professionals. Tetlock also found that the most famous pundits in his study tended to be the least accurate, consistently churning out overblown and overconfident forecasts. Eminence was a handicap.

Why were these pundits (especially the prominent ones) so bad at forecasting the future? The central error diagnosed by Tetlock was the sin of certainty, which led the ‘experts’ to mistakenly impose a top-down solution on their decision making processes. In chapter 2, we saw examples of the true expertise that occurs when experience is internalized by the dopamine system. This results in a person who has a set of instincts that respond quickly to the situation at hand, regardless of whether that’s playing backgammon or staring at a radar screen. The pundits in Tetlock’s study, however, distorted the verdicts of their emotional brains, cherry-picking the feelings they wanted to follow. Instead of trusting their gut feelings, they found ways to disregard the insights that contradicted their ideologies. When pundits were convinced they were right, they ignored any brain areas that implied they might be wrong. This suggests that one of the best ways to distinguish genuine from phony expertise is to look at how a person responds to dissonant data. Does he or she reject the data out of hand? Perform elaborate mental gymnastics to avoid admitting error? Everyone makes mistakes; the object is to learn from these mistakes.

Tetlock notes that the best pundits are willing to state their opinions in ‘testable form’ so they can ‘continually monitor their forecasting performance.’ He argues that this approach makes pundits not only more responsible—they are forced to account for being wrong—but also less prone to bombastic convictions, a crucial sign that the pundit isn’t worth listening to. (In other words, ignore those commentators that seem too confident or self-assured. The people on television who are most certain are almost certainly going to be wrong.) As Tetlock writes, ‘The dominant danger [for pundits] remains hubris, the vice of closed-mindedness, of dismissing dissonant possibilities too quickly.’Even though practically all professionals in Tetlock’s study claimed that they were dispassionately analyzing the evidence—everybody wanted to be rational—many of them were actually indulging in some conveniently cultivated ignorance. Instead of encouraging the arguments inside their heads, these pundits settled on answers and then came up with reasons to justify those answers. They were, as Tetlock put it, ‘prisoners of their preconceptions.’

How will Iron Mountain respond to Elliott Management’s letter to its BoD?

March 11th, 2011

Chris Mellor over at The Register published an interesting article this morning titled “Iron Mountain Hit By Hedge Fund Attack” in which he described the effort of Elliott Management’s Paul Singer to allegedly persuade Iron Mountain to consider forming a Real Estate Investment Trust (REIT).  At least two of the nominees for IRM’s Board of Directors, Schulweis and Antenucci, have been involved with forming REITs in the past mostly in the context of industrial warehousing (Catellus) and multifamily rental properties (Town and Country)—a far cry from using an REIT as a corporate tax shelter when rentals and property management isn’t one’s primary business. Read the rest of this entry »

Revisiting the Value of Information

March 10th, 2011

Back in November 2008 we published an opinion about determining the value of information titled “Outcomes and the Value of Information“. Recently, Martin Sumner-Smith published a post titled “The Implicit Value of Content is Realized Through Business Process” to his blog Martin’s Fulcrum Musings in which he talks about the impact of process on the value of information assets.  I felt this would be the perfect opportunity to update our opinion on the subject. Read the rest of this entry »

Understanding the real cost of your business applications

September 23rd, 2010

Alan Pelz-Sharpe of the Real Story Group, a very reputable analyst firm serving the web and enterprise technology markets, recently published a post titled The High Cost of Support about the annual support fees levied by CMS vendors.

In response to Alan’s post I wrote: Read the rest of this entry »

On information sharing and value generation

August 28th, 2010

Jack Vinson of Knowledge Jolt with Jack recently published a post titled Share it, don’t just store it. We share Jack’s perspective that just storing information is pointless. The value is in knowing what you have, where it can be found and putting it to use.

In response to Jack’s post I wrote: Read the rest of this entry »

Myopia and self-interest hurt government, business, community and family

August 26th, 2010

Vinnie Mirchandani over at Deal Architect wrote another great post titled Alarming IT data points. In it he discusses everything from external dependencies and a lack of innovation at IT companies, to what he believes to be out-of-control costs and a shift in CIO focus.

In response to Vinnie’s post I wrote: Read the rest of this entry »

Relationships are broken, not just in Big Tech

August 21st, 2010

Vinnie Mirchandani over at Deal Architect recently wrote a post titled Big Tech is Broken – badly. In it he wrote that he has “never seen this much acknowledgement that our big technology companies are inefficient organisms”

The disappointment and discontentment (with product and process) has always been there. I experienced it nearly every day in my former career as a software developer and business analyst working on behalf of clients.

Employees are dissatisfied with what they perceive as management’s “screwed-up” priorities, and clients express dissatisfaction not only with vendor processes and priorities but their own as well.

Why have we not seen significant change? Why do broken relationships persist and thrive in and outside business? Here are just a few of the many reasons that come to mind: Read the rest of this entry »

Devil’s advocate on open source

August 19th, 2010

I just finished reading Mark Taylor‘s recently published blog post Open Source: It’s not just cheap!–one of thousands written about the benefits and pitfalls of open source software.

In his post, Mark referenced a Frank Scavo quotation from Chris Kanaracus’s July 2010 article about Open Source ERP. Apparently Frank, an IT consultant, wrote the following comment in an email to Chris:

“A lot of companies are getting fed up with the restrictive licensing, forced march to new versions, and exorbitant fees for ongoing maintenance. For organisations that are willing to take responsibility, open source provides the ultimate in flexibility, low cost, and control of one’s own destiny.”

Mark remarked that Frank’s opinion is not the usual “Open Source is better because it’s cheaper,” but “Open Source is better because you retain control.”

Nearly all justifications for the use of open source are based on notions about cost or control. However, as I pointed out back in 2008, perceived cost-savings and control are often illusions. We’ve thought about the subject quite a lot since then and we are thoroughly convinced that the attraction of open source is rooted in an illusion of control. More about that in a moment.

Read the rest of this entry »

  © 2002-2009 Data Mobility Group, LLC. All Rights Reserved. terms of use privacy copyrights