G8 Guru / Judging Someone You Cannot Check

Follow the incentives, and check whether they have ever been wrong

Two questions with public answers: who profits from your belief, and what happened last time.

Interest does not make someone wrong. People with a stake in a question are often the ones who know it best, and a rule that dismissed anybody who benefits from being believed would eliminate almost every useful source. But knowing the shape of the interest tells you where to look harder. If a person's income, standing or identity depends on you accepting a particular conclusion, that conclusion deserves more scrutiny than the parts of their account that cost them something to say. Ask what they lose if they are wrong, and whether anyone is positioned to notice.

The second question is whether they have ever publicly changed their mind. Anyone who has been making claims for years has been wrong repeatedly, because everyone is. The only variable is whether they have said so. Look for a correction, a revisited prediction, a note acknowledging that an earlier position did not survive. Finding one is strong evidence of good faith. Finding an unbroken record of vindication across a long career is not evidence of accuracy, it is evidence that nothing is being recorded.

Both checks share a virtue: they require no expertise in the subject. You are not assessing whether the claim is true, which you cannot do. You are assessing whether the person's conduct is consistent with someone who cares about being right. That is a weaker test, and it will sometimes clear a person who is honestly mistaken and flag a careful one who simply has no public archive. Used with judgement rather than as a formula, it still filters out a great deal.