“So the real scandal is: Why did anyone ever listen to this guy?”

John Fund writes:

[Imperial College epidemiologist Neil] Ferguson was behind the disputed research that sparked the mass culling of eleven million sheep and cattle during the 2001 outbreak of foot-and-mouth disease. He also predicted that up to 150,000 people could die. There were fewer than 200 deaths. . . .

In 2002, Ferguson predicted that up to 50,000 people would likely die from exposure to BSE (mad cow disease) in beef. In the U.K., there were only 177 deaths from BSE.

In 2005, Ferguson predicted that up to 150 million people could be killed from bird flu. In the end, only 282 people died worldwide from the disease between 2003 and 2009.

In 2009, a government estimate, based on Ferguson’s advice, said a “reasonable worst-case scenario” was that the swine flu would lead to 65,000 British deaths. In the end, swine flu killed 457 people in the U.K.

Last March, Ferguson admitted that his Imperial College model of the COVID-19 disease was based on undocumented, 13-year-old computer code that was intended to be used for a feared influenza pandemic, rather than a coronavirus. Ferguson declined to release his original code so other scientists could check his results. He only released a heavily revised set of code last week, after a six-week delay.

So the real scandal is: Why did anyone ever listen to this guy?

I don’t know. It’s a good question. When Ferguson was in the news a few months ago, why wasn’t there more discussion of his atrocious track record? Or was his track record not so bad? A google search turned up this op-ed by Bob Ward referring to Ferguson’s conclusions as “evidence that Britain’s political-media complex finds too difficult to accept.” Regarding the foot-and-mouth-disease thing, Ward writes, “Ferguson received an OBE in recognition for his important role in the crisis, or that he was afterwards elected a fellow of the prestigious Academy of Medical Sciences.” Those sorts of awards don’t cut much ice with me—they remind me too much of the U.S. National Academy of Sciences—but maybe there’s more of the story I haven’t heard.

I guess I’d have to see the exact quotes that are being referred to in the paragraphs excerpted above. For example, what did Ferguson exactly say when he “predicted that up to 150,000 people could die” of foot-and-mouth disease. Did he say, “I expect it will be under 200 deaths if we cull the herds, but otherwise it could be up to 2000 or more, and worst case it could even be as high as 150,000?” Or did he flat out say, “150,000, baby! Buy your gravestone now while supplies last.”? I wanna see the quotes.

But, if Ferguson really did have a series of previous errors, then, yeah, Why did anyone ever listen to this guy?

In the above-linked article, Fund seems to be asking the question rhetorically.

But it’s a good question, so let’s try to answer it. Here are a few possibilities:

1. Ferguson didn’t really make all those errors; if you look at his actual statements, he was sane and reasonable.

Could be. I can’t evaluate this one based on the information available to me right now, so let’s move on.

[Indeed, there seems to be some truth to this explanation; see P.S. below.]

2. Nobody realized Ferguson had made all those errors. That’s true of me—I’d never heard of the guy before all this coronavirus news.

We may be coming to a real explanation here. If a researcher has success, you can find evidence of it—you’ll see lots of citations, a prestigious position, etc. But if a researcher makes mistakes, it’s more of a secret. Google the name and you’ll find some criticism, but it’s hard to know what to make of it. Online criticism doesn’t seem like hard evidence. Even published papers criticizing published work typically don’t have the impact of the original publications.

3. Ferguson played a role in the system. He told people what they wanted to hear—or, at least, what some people wanted to hear. Maybe he played the role of professional doomsayer.

There must be something to this. You might say: Sure, but if they wanted a doomsayer, why not find someone who hadn’t made all those bad predictions? But that misses the point. If someone’s job is to play a role, to speak from the script no matter what the data say, then doing bad work is a kind of positive qualification, in that it demonstrates one’s willingness to play that role.

But this only takes us part of the way there. OK, so Ferguson played a role. But why would the government want him to play that role. If you buy the argument of Fund (the author of the above-quoted article), the shutdowns were a mistake, destructive economically and unnecessary from the standpoint of public health. For the government to follow such advice—presumably, someone must have been convinced of Ferguson’s argument from a policy perspective. So that brings us back to points 1 and 2 above.

4. A reputational incumbency effect. Once someone is considered an expert, they stay an expert, absent unusual circumstances. Consider Dr. Oz, who’s an expert because people consider him an expert.

5. Low standards. We’ve talked about this before. Lots of tenured and accoladed professors at top universities do bad work. I’m not just talking about scandals such as pizzagate or that ESP paper or epic embarrassments such as himmicanes; I’m talking more about everyday mediocrity: bestselling books or papers in top journals that are constructed out of weak evidence. See for example here, here, and here.

The point is, what it takes to be a celebrated academic is to have some successes. You’re defined by the best thing you did, not the worst.

And maybe that’s a good thing. After all, lots of people can do bad work: doing bad work doesn’t make you special. I proved a false theorem once! But doing good work, that’s something. Now, some of these celebrity academics have never done any wonderful work, at least as far as I can tell. But they’re benefiting from the general principle.

On the other hand, if the goal is policy advice, maybe it’s better to judge people by their worst. I’m not sure.

Not that we’re any better here in the U.S., where these academics have had influence in government.

Taking the long view, organizations continue to get staffed with knaves and fools. Eternal vigilance etc. Screaming at people in the press isn’t a full solution, but it’s a start.

P.S. There seems to some truth to explanation 1 above, “Ferguson didn’t really make all those errors; if you look at his actual statements, he was sane and reasonable.” From Tom in comments:

Mad Cow paper:
https://www.ncbi.nlm.nih.gov/pubmed/11786878

From abstract:
“Extending the analysis to consider absolute risk, we estimate the 95% confidence interval for future vCJD mortality to be 50 to 50,000 human deaths considering exposure to bovine BSE alone, with the upper bound increasing to 150,000 once we include exposure from the worst-case ovine BSE scenario examined.”

Consistent with the “up to 50,000” quote but the quote fails to mention the lower bound.

See also Vidur’s comment which discusses some of the other forecasts.

Read the article here: https://statmodeling.stat.columbia.edu/2020/05/08/so-the-real-scandal-is-why-did-anyone-ever-listen-to-this-guy/

One of the paradoxes of the coronavirus crisis is that the need for public scrutiny of government policy has never been greater, but there’s less tolerance for dissent than usual. That’s particularly true of the work of Professor Neil Ferguson and his team at Imperial College. Anyone questioning Professor Ferguson’s analysis is likely to be met with howls of disdain. Witness the furious reaction provoked by Professor Sunetra Gupta and her team at Oxford when they published a paper suggesting that the Imperial model might have underestimated the percentage of the population that has already been infected. The Financial Times printed a critical letter co-signed by a group of scientists that was reminiscent of left-wing academics denouncing one of their colleagues for dissenting from woke orthodoxy. They used the word “dangerous” in their description of the Oxford research, as if merely challenging Imperial’s model would cost lives, and Professor Ferguson has made the same argument to condemn other critics of his work. “It is ludicrous, frankly, to suggest that the severity of this virus is comparable to seasonal flu – ludicrous and dangerous,” he said.

Professor Neil Ferguson

A more prudent approach would be for the Government not to place too much confidence in any one model, or set of models, but to encourage different teams of experts, working independently, to come up with predictions of their own and challenge their rivals. That’s the tried-and-tested scientific method and it has been bizarre to see respected pundits simultaneously argue that we should be strictly guided by “the science” and that any scientist expressing dissent from the prevailing orthodoxy is behaving “irresponsibly”. That was the same argument used by the Chinese authorities for silencing the doctors who first raised the alarm in Wuhan. They were arrested and forced to confess to “spreading rumours” that “severely disturbed the social order.” Shutting down dissent during an actual war might make sense, but in a war against a virus it is vital that we should stick to the scientific method. As Sir Karl Popper said: “The point is that whenever we propose a solution to a problem, we ought to try as hard as we can to overthrow our solution, rather than defend it.”

We don’t want to repeat the mistakes we made during another viral outbreak, namely the 2001 foot and mouth epidemic. Tony Blair’s government adopted a strategy of pre-emptive culling which led to the death of more than six million cattle, sheep and pigs, with an estimated cost to the UK economy of £9 billion. That strategy was informed by predictive modelling produced by a team at Imperial College led by, among others, Professor Ferguson. Like today, there wasn’t much appetite for questioning his predictions. But we now have good reason to believe his analysis was wrong. Michael Thrusfield, professor of veterinary epidemiology at Edinburgh University, has written two critical reports about the government’s response to that epidemic, concluding that the Imperial College modelling was “severely flawed”.

One person who’s sceptical of Professor Ferguson’s modelling is Anders Tegnell, the epidemiologist who’s been advising the Swedish Government. “It’s not a peer-reviewed paper,” he said, referring to the Imperial College March 16th paper. “It might be right, but it might also be terribly wrong. In Sweden, we are a bit surprised that it’s had such an impact.”

Further Reading

“Carnage by Computer”: The Blackboard Economics of the 2001 Foot and Mouth Epidemic’, David Campbell and Robert Lee, republished in Lockdown Sceptics, originally published in 2003

Use and abuse of mathematical models: an illustration from the 2001 foot and mouth disease epidemic in the United Kingdom‘, Michael Thrusfield et al, Edinburgh Research Explorer, 2006

Physical interventions to interrupt or reduce the spread of respiratory viruses‘, T Jefferson et al, NCBI, July 2011

A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data‘ by John PA Ioannidis, Stat, March 17th 2020

Neil Ferguson, the scientist who convinced Boris Johnson of UK coronavirus lockdown, criticised in past for flawed research‘ by Katherine Rushton and Daniel Foggo, The Telegraph, March 28th 2020

Complicated Mathematical Models Are Not Substitutes for Common Sense‘ by Philippe Lemoine, National Review, March 30th 2020

Dissent over coronavirus research isn’t dangerous – but stifling debate is‘ by Toby Young, The Spectator, April 4th 2020

Predictive Mathematical Models of the COVID-19 Pandemic: Underlying Principles and Value of Projections‘, Nicholas P Jewell et al, JAMA Networks, April 16th 2020

The Tyranny Of Models‘ by William M Briggs, wmbrigs.com, April 17th 2020

After Repeated Failures, It’s Time To Permanently Dump Epidemic Models‘ by Michael Fumento, Issues & Insights, April 18th 2020

POLICY IMPLICATIONS OF MODELS OF THE SPREAD OF CORONAVIRUS: PERSPECTIVES AND OPPORTUNITIES FOR ECONOMISTS‘, Christopher Avery et al, National Bureau of Economic Research, April 2020

How Wrong Were the Models and Why?‘ by Phillip W. Magness, American Institute for Economic Research, April 23rd 2020

The Bearer of Good Coronavirus News: an Interview With John Ioannidis‘ by Allysia Finley, Wall St Journal, April 24th 2020

Imperial College Model Applied to Sweden Yields Preposterous Results‘ by Philip Magness, American Institute for Economic Research, April 30th 2020

The Fatal Hubris of Professor Lockdown‘ by Toby Young, Critic, May 6th 2020

Further Viewing

Read the full article here: https://lockdownsceptics.org/how-reliable-is-imperial-colleges-modelling/