Can you spot the problem with these headlines? (Level 1) – Jeff Leek & Lucy McGowan


“New drug may cure cancer.” “Aspirin may reduce risk of
heart attacks.” “Eating breakfast can help
you lose weight.” Health headlines like these
flood the news, often contradicting each other. So how can you figure out what’s a
genuine health concern or a truly promising remedy, and what’s less conclusive? In medicine, there’s often a disconnect between
news headlines and the scientific research they cover. That’s because a headline is designed
to catch attention— it’s most effective when
it makes a big claim. By contrast, many scientific studies produce
meaningful results when they focus on a narrow,
specific question. The best way to bridge this gap is to look at the original research
behind a headline. We’ve come up with a simplified
research scenario for each of these three headlines
to test your skills. Keep watching for the explanation
of the first study; then pause at the headline
to figure out the flaw. Assume all the information you need
to spot the flaw is included. Let’s start with this hypothetical
scenario: a study using mice to test
a new cancer drug. The study includes two groups of mice, one treated with the drug,
the other with a placebo. At the end of the trial, the mice that receive the drug are cured, while those that received
the placebo are not. Can you spot the problem
with this headline: “Study shows new drug
could cure cancer” Since the subjects of the study were mice, we can’t draw conclusions about
human disease based on this research. In real life, early research on new drugs
and therapies is not conducted on humans. If the early results are promising, clinical trials follow to determine
if they hold up in humans. Now that you’ve warmed up, let’s try a trickier example: a study about the impact of aspirin
on heart attack risk. The study randomly divides a pool
of men into two groups. The members of one group
take aspirin daily, while the others take a daily placebo. By the end of the trial, the control group suffered significantly
more heart attacks than the group that took aspirin. Based on this situation, what’s wrong
with the headline: “Aspirin may reduce risk of heart attacks” In this case, the study shows evidence
that aspirin reduces heart attacks in men, because all the participants were men. But the conclusion “aspirin reduces risk
of heart attacks” is too broad; we can’t assume that results found in
men would also apply to women. Studies often limit participants based on
geographic location, age, gender, or many other factors. Before these findings can be generalized, similar studies need to be run
on other groups. If a headline makes a general claim, it should draw its evidence from a diverse
body of research, not one study. Can you take your skills from the first
two questions to the next level? Try this example about the impact
of eating breakfast on weight loss. Researchers recruit a group of people
who had always skipped breakfast and ask them to start
eating breakfast everyday. The participants include men and women
of a range of ages and backgrounds. Over a year-long period, participants lose an average
of five pounds. So what’s wrong with the headline: “Eating breakfast can help
you lose weight” The people in the study started eating
breakfast and lost weight— but we don’t know that they lost weight
because they started eating breakfast; perhaps having their weight tracked inspired them to change their eating
habits in other ways. To rule out the possibility that
some other factor caused weight loss, we would need to compare
these participants to a group who didn’t eat breakfast
before the study and continued to skip it during the study. A headline certainly shouldn’t claim the
results of this research are generally applicable. And if the study itself made such
a claim without a comparison group, then you should question its credibility. Now that you’ve battle-tested your skills on these hypothetical studies
and headlines, you can test them on real-world news. Even when full papers aren’t available
without a fee, you can often find summaries of
experimental design and results in freely available abstracts, or even within the text
of a news article. Individual studies have results that don’t necessarily correspond
to a grabby headline. Big conclusions for human health issues require lots of evidence accumulated
over time. But in the meantime, we can keep on top of the science,
by reading past the headlines.

About the author

Comments

  1. I'm happy you talked about this issue. I see biased interpretations of studies on a daily basis. It's amazing how two people can see the same data and interpret it so vastly different. Going down the rabbit hole you'll find that many studies themselves are performed with the intention of proving/disproving an idea to such an extent that there is an overwhelming amount of bias in the research methods and test subject selection. It's even become acceptable in some fields to publish "social research" based on your own limited personal experience in some unique situation. This is absurd, and needs to be stopped.

  2. I'm currently reading "Thinking, Fast and Slow", which really helps to see past things like that and mentions lots of examples where studies need to be examined before drawing conclusions.

  3. Also, in the third one, they've lost 5 lb "on average", meaning that if some of them have lost more than 5, some of them have even weight!

  4. "It's not always easy to be interesting and accurate, but…
    it's better then being exciting and wrong."
    The Detective Pickachu movie was interesting, slightly exciting, but inaccurate,
    and got so much stuff wrong, I wonted to walk out and get a refund.

    I've been a "Pokemon Trainer" for 23 years now.

    Here is a list of some things I noticed in the movie that annoyed me.

    Day-time only pokemon out at night, (like Jigglypuff)
    night-time pokemon out in the day. (like Morelull)

    All the Pokemeon in the movie sounded like they should, except Bulbasaur.
    Bulbasaur just made weird cute "squee" noises.

    Climate Change joke when the ground is moving around them crazy.
    The Pokemon world doesn't have a Climate Change problem,
    so not only does the joke not fit the setting, it doesn't fit the world as a whole,
    and becomes a 4th-wall breaking thing.

    SPOILERS

    My fave scene was the Mr.Mime scene because it was funny,
    but Mr.Mime would have gotten away since the things it mimes are "real".
    Mr.Mime can run, it doesn't need to "Mime" running.
    If Mr.Mime "mimed" a motor-bike, he'd have driven off and gotten away.

    Mewtwo didn't see that "the guy trying to capture him",
    was the one who sent his kid to look for Mewtwo.

    The R-Gas makes pokemon go into a rage.
    But not when used on Ditto, instead makes Ditto faint, for some reason.

    In the movie some guy made a thing that fit's on your head that can control Mewtwo's mind.
    Alakazam's mind is faster and more powerful,

    then the world's most powerful super computer that is multi-stories tall.
    Mewtwo can overpower it's mind effortlessly,
    and can even keep people and pokemon as mind-slaves with even less effort.
    What should have happened is, the guy should have went brain dead.

    They say Mewtwo can move/re-move/exchange souls between pokemon only in a state of rage,
    but non-mind-controlled Mewtwo could do mind-body-soul transfers into pokemon not raging.
    (main character's dad's mind/body/soul transferred into his pickachu)

  5. Your approach is flawed because there is another very obvious reason that each headline is misleading.

    That, of course, is the way each was phrased. The use of 'may' and 'might' also made each one highly suspect.

  6. This is also why we have people trying to ban guns and pitbulls. We really need to repeal the NFA and Hugh's Amendment.

  7. As a dentist I see this all too often. The general public needs more information like this otherwise gaps of information get created in society. This is probably my favourite video of yours and I'd totally look forward to more. This also helps anyone doing an article critique!

  8. Interesting and informative, although it might be helpful to pause a little to give viewers the chance to think of what might be wrong with the headline to encourage critical thinking?

  9. On the newspaper I read, the latest scientific research are like this "Scientists have found a breakthrough in cancer treatment" or to that effect. Which as far as I'm aware is accurate enough when trying to cure cancer.

  10. How dare you talk about just two genders? It is 2019, a person can be a bicycle of they want!

    You deserve the 135 dislikes (against about 6,5 k likes)

  11. The headlines are counterintuitive since they are needlessly presenting "measurements" as "estimations." Since most news outlets pay a greater attention to political affairs, their words for any promising scientific discoveries resemble those of the daily horoscope. "May, might, ifs and buts" so on…

  12. the thing is though completely airtight studies are basically impossible thats actually why a prominent statistician didn't believe that smoking caused cancer because of lack of placebo control and choice.

  13. um isn't that herme's cadeus? Im pretty sure that the healing god's snake is the actual healing thing.

  14. So was the scepter of hermes at 0:37 intentional because it's supposed to represent misleading reports on medicine, or was it a real oopsie?

  15. 0:36 and 4:44 , that is caduceus, not to be confused with the rod of asclepius, which has one snake and no wings, which is the correct association with medicine.

  16. 0:39 Actually, this representation isn’t entirely correct either.

    What’s actually depicted here is the staff of Hermes (also Caduceus) who held divinity over merchants/trade and thieves. Because of how it looks it’s often confused with the staff of Asclepius (also Asklepios) which is a single serpent coiled around a staff without wings. This latter staff is the symbol of medicine and hygiene which the animators wanted to show here presumably.

  17. at 0:35, that is the caduceus. It is a symbol for hermes and has nothing to do with medicine.
    The symbol for medicine is the rod of asclepius, which consists of a single snake wrapped around a rod/stick that does NOT have wings.

  18. Maybe an idea for a future episode based on this one, the difference between causality and correlation. Way to many people use correlation to sell you on the idea of something. I have yet to find a causal relationship between insulin spike and drinking diet beverages, only correlations, but bloggers and doctors alike take it as a causal relationship. Please, for anybody reading this, I want to find a paper about the causal relationship between artificial sweeteners and insulin spike, I have yet to find it myself.

  19. 0:37 THAT IS THE STAFF OF HERMES THE MESSENGER GOD. ACCORDING TO PERCY JACKSON, THE SNAKES NAMES ARE GEORGE AND MARTHA. BUT, WHEN LEO, JASON AND PIPER GO TO ASCLEPIUS, THE GOD OF MEDICINE, HE CLEARLY STATES THAT HE HATES WHEN PEOPLE CONFUSE HIS ONE-SNAKED STAFF IS THE HEALING STAFF.

  20. Omg did you just say that men and women’s bodies are different??? You are about to get banned by all liberals and youtube will demonetize you.

  21. As for those who aren't awared, this is basicly the foundation of analizing the crediblity of a scientific report.
    The research paper is also needed to come from a credible sourse/publisher. (this is what wikipedia sadly lacks)

  22. I'm glad you covered the breakfast-eating one, that one was such BS, and it annoyed me that news organizations reported it that way.

    I feel like number 2 was reaching though. There doesn't exist a large enough physiological difference in the way men's cardiovascular system works versus women's, so I don't think #2 was misleading. In fact, I think adding the "in men" is overly pedantic.

  23. Also, if you randomly divide two groups of men and conducted one experiment, you don't know if the men in one group were predisposed to heart attacks already. It doesn't talk about the health of the individual, prior tests were not done. Had they already been taking aspirin previous to this? It's difficult to conduct ONE experiment on ONE group of people and conclude this. It's an incredibly general statement.

  24. based on the information given, maybe it’s also possible in the 2nd headline that people in the control group was just naturally more prone to heart attacks due to… disorders or age or eating habits in the first place, since they were divided randomly and supposedly not controlled on those aspects? and this could have affected risk of heart attacks alongside aspirin, which could affect the validity of results?

  25. If you're a Psych student doing research, the part of finding out what's wrong would be under the limitations section for a literature review

  26. What?? I believe every headline I read. I saw a headline at the supermarket checkout magazine, lose 20 lbs in just 2 days. All you have to do is put the work in. LOL

  27. I personally found this kind of too easy and uninspiring. It felt more like a very limited English reading quiz. Might be good for Junior School students though…

  28. Well i know one headline in the internet
    Its say "Florida man arrest after eating pancakes in the middle of the road"

  29. I think what we should learn from this is that you can only fit several words into a headline; so, it's going to be an abbreviation. For example, "Cure cancer" will always mean "cure this one particular kind of cancer that they studied". Also, "may" (meaning might) gives an indication that it hasn't been studied all that long and if it ever pans out (and most things don't), it's going to be a long time before it will be used on the general public. And yeah, just know that there will be hype.

  30. Because this a channel about education, I'm going to point out a factual mistake this video makes, not to be annoying but because it was a mistake I used to make and had to research to correct.

    Around 0:36 you show a rod with wings and two intertwined snakes. This symbol appears immediately after you say the word "medicine", implying it is a symbol for medicine.

    That symbol is called caduceus. A staff carried by Hermes in Greek mythology, but has nothing to do with medicine. It is usually (as in the video) confused with the Rod of Asclepius, which is, in fact, associated with healing and medicine, but it has only one snake, and no wings.

  31. I always love the animation in these videos. I'm aspired to be someone who can make these kind of animations

Leave a Reply

Your email address will not be published. Required fields are marked *