Cognitive biases are systematic deviations from the standards of rational thinking and decision-making. Cognitive biases influence individual as well as collective decision-making (Sunstein & Thaler 2008, Kahneman 2012). Human cognition, perception and memory are adaptive and have been shaped by evolutionary pressures (Leakey 1991, Maslin & Lewis 2018). Biases (and heuristics) may not always lead to optimal decisions in terms of rationality. But, on average, they appear to translate into ‘good enough’ decisions and indeed are often more cost-efficient than more complex decision-making rules and procedures (Klein 1999). This is why intuition (biases, heuristics) sometimes fall short in important ways. Humans find it difficult to grasp the phenomenon of exponential growth and humans typically lend too much weight recent events and the near future than what a rational calculus would suggest. The existence of biases and heuristics simply confirms other research that demonstrated that human behaviour deviates from perfectly rational behaviour. Experiments have shown that individuals act less than rationally in a (game-theoretical) non-repeated dictator game. Completely self-interested behaviour is in fact the exception rather than the rule (Ariely 2009).
Cognitive biases and fallacies are ubiquitous. Biases often get divided into individual and social (and sometimes memory biases). Individual and social biases involve a flawed understanding of statistics and probability calculus, or their complete disregard. Cognitive biases lead people to hold mistaken (non-rational) beliefs and commit fallacies. Some fallacies can be squarely attributed to biases (e.g. base rate fallacy, conjunction fallacy), while other fallacies are perhaps simply due to a limited understanding of statistics and probability (e.g. birthday paradox, Simpson paradox, prosecutor’s and defence attorney’s fallacy). In all cases, human ‘intuition’ leads people to hold mistaken beliefs and/ or biased views and often leads them to make non-rational choices. For example, many people ignore the survivorship bias in data (e.g. Wald’s problem) or the they commit a conjunction fallacy (e.g. Linda problem). Training in probability and inferential statistics can help people correct for such biases (King, Keohane & Verba 1994).
Biases often make it more difficult - and often impossible - to reach rational consensus. Finding consensus may be difficult due to different preferences, interests or values. Jonathan Haidt (2013) has shown that Republicans and Democrats tend to subscribe to different sets of values. The commitment to these values informs the intellectual arguments used in the defence of these values (not vice versa). But even if the defence rested solely on the intellectual merit of its arguments, different premises/ values would lead to different conclusions. In other words, different premises combined with deductive rules lead to different conclusions. (The conclusions are true, if the premises are true.) Values (like tastes) do therefore not lend themselves to rational debate. Or at least they do not lend themselves to rational consensus. De gustibus non disputandum. In practice, of course, disagreement about values and morality rarely involve formal deductive inference. Such discussions are more often about asserting one’s view and expressing affective opposition to the other person’s view.
In principle, it should be easier to find consensus with respect to empirical questions, in spite of cognitive biases. This is because in principle such questions concern the problem of how empirical evidence bears on the question (hypothesis) and should be more amenable to agreement in light of the evidence. Like in a deductive inferential situation, one may start with a different view (hypothesis). But as evidence accumulates, one is forced to update once initial belief. Naturally, empirical questions do need to overcome cognitive and affective biases. “It is difficult to get a man to understand something, when his salary depends on his not understanding it”, Upton Sinclair once remarked. This may explain why (most) investment bankers believe in low taxes and in trickle-down economics even though there is no empirical evidence to support the latter. Agreement on whether low taxes are more desirable than high taxes is a normative issue, of course. But it should be possible to get to a certain degree of agreement about what relevant effects lower or higher or taxes have. In principle, this question should be amenable to approximate agreement. Such questions can only be settled in a empirical-inductive manner, in light of evidence that bears on the hypothesis.
Extra-epistemic factors do influence how willingly individuals engage in a critical evaluation of hypotheses. For example, the rally-around-the-flag is very common during wars and national emergencies (Mueller 1970). When uncertainty is high, the need for leadership, authority and sense-making helps people deal with anxiety. Stability and reassurance then becomes more important than critical engagement with claims made by the authorities. (It can also make for strange proverbial (?) bedfellows.) Authority in general can limit people’s inclination and ability to assess views and claims critically. Often people are swayed by non-epistemic factors. Investment bankers dress well for a reason and US presidents and CEOs are disproportionately tall. Unless nice suits or tallness is correlated with greater intellect, people behave irrationally in the sense of letting themselves be influenced by non-epistemic factors (Economist, September 27, 2017). Research has shown that Democrats/ Republicans are more likely to support/ oppose the very same legislative proposal depending on whether it has been proposed by Democrats/ Republicans. Framing matters, too.
People tend to overestimate their intellectual grasp of issues and think themselves smarter than experts. This is related to the so-called knowledge illusion: people believe they understand things, but they don’t (Fehrenbach & Sloman 2017). Overestimating one’s own knowledge underestimating the knowledge of experts is also called the Dunning-Kruger effect (Dunning & Kruger 1999). Remember Michael Gove’s remark that “people have had enough of experts”. (Gove later clarified that he only referred to economists (BBC, February 27, 2017).) Interestingly the way to redress this is to ask people to explain how something works instead of why they support or oppose it (e.g. social security). This tends to nudge people towards a more reasonable view of their own abilities and makes them more willing to engage in fact-based debate that is more likely to lead a degree of agreement. If, on the other hand, one confronts people with a contrary view on a – actually or potentially – value-laden issue, people’s position tends to harden (Economist, December 8, 2018). This may be related to group biases. Either way, none of this makes it more likely that people will reach agreement.
Research has shown that experts are on average better than laymen in terms of making the correct predictions and good decisions (Klein 1999). However, not everybody who is called an expert is an expert. Philip Tetlock (2005) distinguishes between hedgehogs and foxes. Hedgehogs claim expertise but are often fairly ideological individuals with pre-set views and an unwillingness to learn from their mistakes (Tetlock 1998). (They make for good television in an age of polarisation.) Foxes, on the other hand, know many little things, are willing to learn from their mistakes, seek out diverse information and adjust their probabilistic forecasts in light of new information. Foxes do in fact outperform hedgehogs prediction-wise (Tetlock 2015). Polymath John Maynard Keynes would not have been surprised: “When the facts change, I change my mind. What do you do, sir?”
Experts have a greater awareness of their cognitive biases, constraints and limitations. Good analysis requires an openness to new information as well as the continued questioning of assumptions, analytical frameworks and hypotheses as well as information. Good analysts are aware of cognitive, group and memory biases (CIA 1979). This awareness helps them avoid avoidable mistakes (King, Keohane & Verba 1994). Psychologically, this requires precisely what most people find uncomfortable: being comfortable dealing with cognitive incoherence and refusing cognitive closure. F. Scott Fitzgerald quipped that a great mind can hold two contradictory ideas in the mind at the same time. Good analysts do exactly that.
Scientists, whose training should attune them to the existence of biases, are also influenced by extra-epistemic factors in their evaluation of hypotheses and theories. In spite of their training in methodology and epistemology, scientists frequently remain wedded to so-called Kuhnian paradigms (or Lakatosian research programmes), theories and even individual hypotheses (Kuhn 1962, Lakatos 1978). Max Planck once quipped that science progressed one funeral at a time. This suggest that even scientists find it difficult to adjust their prior beliefs in light of new evidence. Extra-epistemic factors may be at work, including interests, biases, cognitive coherence. A brief look at Bayes’ theorem and holism suggests that this reluctant may not be indefensible, epistemologically speaking.
Could a Bayesian framework help foster a convergence of views and keep at bay biases? In a nutshell, Bayes theorem allows individuals to start out with different subjective beliefs (priors). The priors are linked to a conditional probability P (H/E), that is, the probability of the hypothesis given the evidence. To the extent that a piece of evidence raises the probability of the hypothesis, a rational agent will need to adjust P (H/E) upwards. If P (H/E) is not updated accordingly, an agent acts irrationally – and a diachronic Dutch book can be made against him. In principle, this allows two agents to start off with different views but end up with converging views as evidence accumulates.
There are some well-known problems with the subjective interpretation of Bayes' theorem. Convergence of beliefs is only possible if the priors are different from zero. This is not too much of a problem. Extremists and perhaps hedgehogs are likely to assign a zero probability to the priors. Most 'reasonable' people wouldn't. Priors only “wash out” in the long run. This is something that may keep philosophers up at night, but it is far from a decisive argument against Bayesian convergence. Even the likelihood of the evidence given the hypothesis P(E/H) that is used to help update P (H/E) is not too troublesome. Often the hypothesis entails the evidence. Surprisingly, the most problematic part of Bayes’ theorem is to do with the probability of the evidence P(E) = P (E/H) and P(E/-H), that is, the denominator. While P (E/H) is not too problematic, as just mentioned, the term P(E/-H) is a problem for -H is a catch-all hypothesis and may include any number of alternative hypothesis. This makes it difficult to assign probabilities to it. Then again, this is something philosophers (rightfully) quibble over. In practice, ‘reasonable’ people might be able to assign reasonably similar probabilities to P(E). Granted, unreasonable people won’t – and this is the point. At first sight, Bayes' theorem suggests a way to update one’s views in light of new evidence and a way for epistemic consensus to emerge among individuals in spite of very different priors. It turns out that this will only work if individuals are ‘reasonable’ to be begin with in the sense of not assigning off-the-charts probabilities to P(E) as well as not assigning zero probabilities to either of the priors.
It is not clear that Bayes’ theorem can deliver convergence and consensus, even in principle. Scientists do not simply or quickly drop their hypothesis just because a piece of evidence bears negatively on it. (Popper’s fallibilism is too extreme.) Scientists often adjust other parts of their theory to account for such an ‘anomaly’. This is exactly what non-scientists do. They hold on to their prior beliefs (hypotheses) in spite of contrary evidence. They do so by changing the probabilities they assign to various parts of the theorem after the evidence is in. This would allow one to make a diachronic Dutch book and it does not feel legitimate to change posterior probability by changing P(E/H) or P(E). However, scientists (or philosophers of science) can invoke Quinean holism and the underdetermination of the theory by the evidence. Holism says that hypotheses can never be disconfirmed individually, but only as part of a wider theory or web of belief. Auxiliary hypotheses and background assumptions can always be adjusted to salvage the hypothesis. This also causes problems for Bayesian confirmation. To the extent that background assumptions are adjusted, the probabilities plugged into Bayes theorem may change. Again this looks illegitimate, hence the diachronic Dutch book argument. But this is often how salvaging one’s prior belief works, whether one is a scientist or a non-scientist. Aside from the problems related to Bayes’ theorem discussed above, this weakens the useful of Bayes' theorem as a tool to generate consensus. At the very least, it does so in practice. Subjective Bayesians naturally regard such a move as inadmissable. Holism weakens the usefulness of Bayes’ theorem by weakening the very constraints that - in the Bayesian view - were supposed to rein in biases and force a convergence of posterior beliefs in spite of potentially wildly different subjective prior beliefs.