- Dr. Eyal Shahar, an epidemiology professor emeritus at the University of Arizona, says flu vaccine estimates from the Centers for Disease Control and Prevention (CDC) cannot be trusted.
- He argues the CDC’s test-negative study design is biased because it only includes people who sought care and got tested.
- Jay Bhattacharya, director of the National Institutes of Health, delayed a CDC COVID-19 vaccine study over the same design concern.
- A Cleveland Clinic preprint found vaccinated employees had a 27% higher flu risk in an early version.
- Shahar also says excluding early post-vaccination infections can inflate vaccine effectiveness estimates.
For years, Americans have relied on federal estimates of how well the annual flu shot works. But an epidemiologist now argues those numbers are fundamentally untrustworthy, pointing to what he calls an inherent bias in the research method the Centers for Disease Control and Prevention has long relied upon. Dr. Eyal Shahar, a professor emeritus at the University of Arizona’s Mel & Enid Zuckerman College of Public Health, contends that the CDC’s favored study design produces biased and incoherent results.
How the test-negative design can skew results
Shahar’s central complaint is the “test-negative design,” a method the CDC has used for years to calculate vaccine effectiveness. The approach looks only at patients who visited a doctor with flu-like symptoms and got a flu test, then compares how many in the positive and negative groups had been vaccinated.
The problem, Shahar explained to The Defender, is that this approach excludes everyone who did not seek medical care or receive a test. That creates selection bias. More troubling, it can also introduce “collider bias,” a statistical distortion that can skew results.
“There are many supporters of the test-negative design, and the CDC simply adopted that design long ago,” Shahar said. “There are CDC-associated networks, all of which exclusively use that design to estimate effectiveness. It is time for a change.”
Shahar, the author of nearly 200 papers and two books, including an epidemiology and statistics textbook, argues that a well-designed cohort study tracking both symptomatic and asymptomatic flu cases would yield more accurate data. But such studies require more funding and effort.
NIH director delayed a CDC study over the same flaw
Shahar is not the only one raising concerns. Jay Bhattacharya, director of the National Institutes of Health, who served as acting CDC director this year, has voiced similar skepticism about the test-negative approach.
Earlier this year, Bhattacharya delayed publication of a COVID-19 vaccine study in the CDC’s Morbidity and Mortality Weekly Report because it relied on the test-negative design. The journal later rejected it. Bhattacharya defended the decision on X, saying the design is statistically flawed and that no one “should view the paper as providing an accurate measure of vaccine efficacy.”
Cleveland Clinic preprint found higher flu risk among vaccinated workers
A Cleveland Clinic cohort study posted last year as a preprint found vaccinated employees had a 27% higher risk of flu in the 2024-25 season, according to an early version of the paper. An author has said the findings do not justify pulling the vaccine from the market. The authors listed avoiding the test-negative design as a strength, writing that such studies “systematically overestimate true vaccine effectiveness.”
Brian Hooker, chief scientific officer at Children’s Health Defense, told The Defender that the flu shot has never really prevented transmission and that its efficacy against flu was negative in some years, including last year. “Flu would blow through a vaccinated population with nearly the same ease as it would an unvaccinated population,” he said.
Excluding early infections can inflate effectiveness, Shahar says
Shahar’s analysis of a 2022-2023 study from a CDC-associated network revealed another flaw: “immortal time bias.” The study excluded patients who were vaccinated fewer than 14 days before their test or visit. Shahar says removing those early infections, which occur before the body builds immunity, can inflate the vaccine’s apparent effectiveness.
Yale epidemiologist Dr. Harvey Risch added that the problem extends deeper, arguing that infections in the first two weeks after a shot should count as a possible harm, not be discarded. “If vaccines cause harm, accounting for that risk starts immediately upon vaccination,” Risch told the Epoch Times.
Shahar has shared with The Defender a preprint on a newly described bias in the test-negative design. “I can’t get it published,” he said. He maintains that the CDC employs epidemiologists who know of better methods but keeps using a bias-prone design. Americans deciding each fall whether to roll up a sleeve deserve to know how solid the numbers behind that advice really are.
Sources for this article include:
Please contact us for more information.