Sherlock Holmes as Enemy of Confirmation Bias

Further to my recent paper on Sherlock and the ontology of ficta:

http://www.opticon1826.com/article/view/opt.bs/304

– which was kindly tweeted by Dr Watson:

– I was also pointed by Dr Watson towards some very interesting Holmes quotes aimed at showing that he is a fan of data-driven decision making:

That looked like a decent case, but what struck me more about the five well-chosen quotes is that they really show that Holmes is very well aware of the problem of Confirmation Bias. This is prevalent everywhere in everyone and completely bedevils our reasoning abilities. Given that this is very modern psychology, it is remarkable that Holmes was on to it so quickly.

I will proceed as follows. I will give you the quotes; I will tell you what Confirmation Bias is; I will show how the quotes show that Holmes is aware of the problem, and I will close with some brief remarks as to why Confirmation Bias is a problem.

Quotes from Sherlock

Here are the quotes; again courtesy of the Umbel blog.

1. “There is nothing like first-hand evidence.”

2. “The world is full of obvious things which nobody by any chance ever observes.”

3. “It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”

4. “I never guess. It is a shocking habit,—destructive to the logical faculty.”

5. “‘Data! Data! Data!’ he cried impatiently. ‘I can’t make bricks without clay.’”

What is Confirmation Bias?

First cut: Confirmation Bias is the tendency to confirm what you already believe.

This of course is the enemy of good hypothesis formation. You should instead attempt to falsify what you believe. That is the only way of proving anything, because attempting to prove what you already believe just gives you an endless series of facts which are consistent with your hypothesis. You can have an infinite series of consistent observations but that proves nothing; whereas a single disconfirmatory observation disproves the hypothesis!

Given the remarkable asymmetry in power of potential observations, it is remarkable that few people ever look where they ought to. Of course, one reason for that is that if you falsify a hypothesis you already hold, you will have to track through the ramifications of that for your whole belief structure. If for instance, you find out that the man in the hot is not Moriarty, you will have to discard a large number of other beliefs. If you saw the man in the hat at the station, you now have to believe that the man at the station was not Moriarty, and so on, with potentially significant consequences for your picture of the world. This takes time and energy so people don’t want to do it.

Confirmation Bias comes in three main forms: a) not looking for disconfirmatory evidence; b). ignoring disconfirmatory evidence if it is pressed upon one; c) discounting disconfirmatory evidence.

Holmes on the Case

The key is quote 3, which is basically a statement of the problem of Confirmation Bias. The facts you actually see are twisted by what you are expecting to find, and so you will ten inexorably to find what you were expecting. For that reason, guessing is a mistake, as Holmes points out in quote 4. Because a guess does not stand in a vacuum. It is formed from currently existing half-beliefs and things you are prepared/want to believe. SO it is biassed. Worse still, the guess becomes a hypothesis which by the twisted magic of Confirmation Bias will now find ways of becoming your truth. Holmes is right to call this a shocking abuse of logic.

Quote 2 speaks to the problem of ignoring data. Many obvious things are unremarkable merely because we have seen them so often. Take gravity. Why do we stick to the earth? Isn’t that odd? No-one thinks so, but how can it be explained? (Incidentally I object to the latest TV version having Holmes say he doesn’t know that the earth goes around the sun because it changes nothing here. We would, for example, be shocked by his failure to expose as an impostor a scientist who claimed the sun goes round the earth. So Holmes needs an excellent theory of the world in order to have the excellent Theory of Mind that he clearly enjoys.)

Quotes 1 and 5 speak to the primary importance of data, which as I have been saying must be impartially collected and not merely what makes it through after Confirmation Bias.

Why is Confirmation Bias a problem?

Think about just two things: religion and politics. Imagine that you have been trained from a young age to believe a set of random hypothesis and have then had a lifetime exercising Confirmation Bias to back up these hypotheses. Some people move on from religious fairy tales, but many do not. Also, have you noticed that most people vote the way their parents did? They seem to know *without listening* that everything that the other political party says is wrong. This sort of factor gives you the political polarisation currently visible in America and elsewhere.

This is not a good thing and Holmes is right to warn us strongly against it. Beware Confirmation Bias!