The Science of Knowing What You Don’t Know

Our new research reveals a better way to detect who actually listens, learns, and updates their beliefs.

Philosophers have a long history of debating our epistemic vices and virtues. Virtues such as intellectual humility involve the tendency to seek out and respond to evidence and the testimony of others in ways that are conducive to the acquisition, maintenance, and transmission of knowledge. People who have this trait recognize the limits of one’s own knowledge and remain open to alternative perspectives.

In the past decade, there has been an explosion of interest on the topic of intellectual humility(see the figure below). This had led to the development of at least 10 new scales to measure intellectual humility and hundreds of new studies on the topic. If you agree with statements like “I welcome different ways of thinking about important topics” and disagree with statements like “I believe my ideas are superior to others’ ideas” then you would be considered an intellectual humble individual.

It turns out that intellectual humility is a great trait to have. People who score high on these measures are less likely to spread conspiracy theories or misinformation, express out-group prejudice, and feel polarized. They are more likely to follow public health guidelines, reason carefully about political issues, and update their beliefs in light of novel information.

However, the variety of different measures makes it difficult to know which scale to use or how to interpret different results produced by competing scales. Thus, creates the worrisome possibility that scholars in this area are committing the jingle fallacy (using different labels for the same thing) or the jangle fallacy (using the same label for different things) in this research.

Therefore, Jay conducted a series of ten studies (including over 5,900 participants) with Philip Parnamets and Mark Alfano to generate a single unified measure of intellectual humility. We called it the “Collected Intergroup Intergroup Humility Scale”. You can download our paper here if you want to read the details from our research.

One key feature of our measure is that we created a subscale to measure reasoning in the context of intergroup conflict. Prior measured focused on individual reasoning in a social vaccuum or merely in the context of dyadic disagreements. This is a remarkable omission because the biggest challenge for many people is responding with humility in the face of criticism from out-group members. Therefore, we created a number of items to capture the ability to question or challenge their own group and remain open to ideas from outsiders

Overall, we found five key factors of intellectual humility:

1. Open-mindedness: being open to new perspectives and evidence, especially when they are not consistent with one’s existing ideas. People who score high on this scale agreed with items like “I like finding out new information that differs from what I already think is true”.

2. Intellectual defensiveness: reacting emotionally or negatively to disagreement and the possibility of one’s own cognitive limitations. People who score high on this scale disagree with items like “I feel uncomfortable when someone points out one of my intellectual shortcomings".

3. Intellectual arrogance: indifference to truth or the possibility of being wrong. People who score high on this scale disagree with items like “I don’t really enjoy gaining new knowledge”.

4. Sense of intellectual superiority: believing you or your group is smarter or better at reasoning than other individuals or groups. People who score high on this scale disagree with items like “My ideas are usually better than other people’s ideas”.

5. Ingroup criticism: actively seeking out and correcting errors in reasoning by one’s in-group. People who score high on this scale agreed with items like “To avoid group-think, it’s important to be extra critical of your own group’s ideas”.

We include all the items below so you can measure your own intellectual humility of give it to your friends and colleagues!

Although these factors were all correlated with one another, they seemed to capture distinct sub-scales (see figure below). We also found that intelligence, numeracy, and reasoning skills were only weekly correlated with these five factors (all correlations were below r = .16). This suggests that intellectual humility is not simply reducible to basic cognitive abilities. We also suspect that intellectual humility might be easier to cultivate than basic intelligence, which is highly stable over time within individuals.

You can see how you score on each of these items to assess your own degree of intellectual humility. You are also free to share our measure with your friends, family and coworkers and discuss their scores. Have them answer each question below on a 7-point scale their level of agreement with each item, with 1 =“strongly disagree” to 7 = “strongly agree.”