Unit 4: How to Think Critically about Psychological Science

Unit 4: Assignment #1 (due before 11:59 pm Central on Monday September 6):

  1. To appreciate why critical thinking is crucial to understanding psychological science,
    1. Watch Crash Course’s (2014) YouTube, “Psychological Research.” Because the narrator of the video speaks quite rapidly, you might need to watch the video at least twice (or use the speed-controller in the settings options on YouTube).
    2. Read Halonen’s (1996) article, “On Critical Thinking [in Psychology].” Note that in Unit 2, we worked on what Halonen refers to as “Practical” critical thinking skills. In this Unit, we will be working on what Halonen refers to as “Methodological” critical thinking skills.
    3. Read the first page of Dewey’s (2007) chapter, “Critical Thinking [in Psychology].”
    4. Read Stafford’s (2014) article, “What It Means To Be Critical [about Psychological Research],” which is more about how to be critical of psychological science than why it’s important to be critical, but Stafford’s article will prepare you for other Assignments in this Unit.
  2. To continue developing your skill for writing five-paragraph essays, write a five-paragraph essay of 400 to 500 words arguing either in favor of or against the statement, “Critical thinking is crucial to understanding psychological science.”
    1. You may write either a Reasons/Arguments Essay OR an Examples Essay.
    2. For either type essay:
      • Remember to begin by jotting down somewhere your three Reasons/Arguments or your three Examples.
      • Next, you should write your three Reasons/Arguments paragraphs or your three Examples paragraphs.
      • Then, you should write your Thesis Statement.
      • Next, write your Introduction Paragraph, including a hook.
      • The last step is to write your Conclusion Paragraph, in which you restate your Thesis Statement and end with something witty or profound
    3. Remember that all five of your paragraphs, your Introduction Paragraph, each of your three Reasons/Arguments Paragraphs or each of your three Examples Paragraphs, and your Conclusion Paragraph, need to have
      • a Topic Sentence;
      • three Supporting Sentences; and
      • a Conclusion Sentence.
  3. Save your essay as PDF and name the file YourLastname_CriticalThinkingEssay.pdf.
  4. Go to the Unit 4: Assignment #1 Discussion Board and make a new Discussion Board post to which you attach your essay, saved as a PDF. Remember to “Attach” your essay’s PDF (don’t embed your file or use use the “File” tool; instead, use the “Attach” tool).

Unit 4: Assignment #2 (due before 11:59 pm Central on Tuesday September 7):

  1. To learn the critical questions that should be asked about psychological science (or any type of science) reported in the news:
    1. Read about the first (“Sensationalized Headlines”), second (“Misinterpreted Results”), third (“Conflicts of Interest”), and twelfth (“Non-Peer Reviewed Material”) indicator of bad science in Compound Interest’s (2015) infographic “A Rough Guide to Spotting Bad Science.”
    2. To see examples of the first (“Sensationalized Headlines”) and twelfth (“Non-Peer Reviewed Material”) indicators of bad science, watch Above the Noise’s (2017) YouTube, “Top 4 Tips To Spot Bad Science Reporting.”
    3. Read Ossola’s (2017) article, “Can You Tell If a Health Story Is Total BS?” Ossola’s indicators of “Check the Label” and “Control the Spin” are like Compound Interest’s “Sensationalized Headlines” indicator; however, Ossola presents a novel indicator “Beware the Animal Study.”
    4. To see examples of these indicators of bad science, watch Last Week Tonight with John Oliver’s (2016) YouTube, “Scientific Studies.” Warning: John Oliver is a late-night comedian/TV host. Therefore, this video contains adult content, adult language, and extreme irreverence toward a wide swath of people. The video presents numerous examples of bad science indicators; however, if you’d prefer not to watch the video, then please don’t.
  2. From the Internet, find three news reports, each of which reports about a different research study that is characterized by at least one of these indicators of bad science:
    1. “Sensationalized Headlines”
    2. “Misinterpreted Results”
    3. “Non-Peer Reviewed Material”
    4. “Beware the Animal Study”
    5. “Conflicts of Interest”
  3. Be sure to find three different news reports, each of which reports about a different research study, rather than three news reports all of which report about the same research study.
  4. Go to the Unit 4: Assignment #2 and #4 Discussion Board and make a new post, of at least 200 words. In your post:
    1. Describe each of the three news reports, preferably each in its own paragraph.
    2. Identify which indicator of bad science characterizes each news report.
    3. Provide for each news report either its URL (using the technique you learned from the Course How To) or, if a video, its YouTube or Vimeo (using the technique you learned from the Course How To).

Unit 4: Assignment #3 (due before 11:59 pm Central on Wednesday September 8):

  1. This assignment will focus on the fourth indicator of bad science in Compound Interest’s (2015) infographic, (Text-Only Word version available here) which is confusing “Correlation with Causation.”
    1. Watch TEDxDelft’s (2012) YouTube, “The Danger of Mixing Up Causality and Correlation.”
    2. Watch PsychU’s (2015) YouTube, “Correlation vs. Causation – PSY 101.”
    3. Because PsychU momentarily confuses the term “hypothesis” with the term “theory,” watch PBS’s (2015) YouTube “Fact vs. Theory vs. Hypothesis vs. Law … Explained!”
    4. Make sure you know the meanings of, and differences among, the four terms: Fact, Theory, Hypothesis, and Law. Not only will you will need to know these terms and their differences throughout the rest of this course, but everyone should know these terms and their differences.
  2. Back to understanding the problem of confusing “Correlation with Causation”:
    1. Watch AsapScience’s (2017) YouTube, “This ≠ That.”
    2. Watch Khan Academy’s (2011) YouTube, “Correlation and Causality.”
  3. Make sure you understand what correlation means, what causation means, and why correlation cannot be used to prove causation.
    1. Jot down at least six examples of correlation not proving causation from the videos you watched. One example is the correlation between the amount of ice cream purchased (during each month of the year) and the number of drowning deaths (during each month of the year) not proving that ice cream causes drowning.
    2. Make sure you understand that two variables (e.g., ice cream purchases per month and drowning deaths per month) might both be caused by another variable (e.g., season of the year). That other variable is often called a confounding variable.
    3. Make sure you understand that the correlation between two variables (e.g., pool drownings per year and Nicholas Cage films per year) might simply be due to coincidence.
    4. Make sure you understand that rather than one variable (e.g., skipping breakfast) causing another variable (e.g., obesity), the causation might be reversed.
  4. Teach three separate people why they should not confuse correlation with causation. You can teach each person via email, phone, text, Facebook, Skype, in person, or any other communication medium. But you must teach three separate people at three separate times why they should not confuse correlation with causation.
    1. When you are teaching each person, provide examples of correlations that do not prove causation, using the examples you saw in the videos.
    2. To make sure that each of the three people learned why correlation should not be interpreted as causation, ask each person to tell you another example (an example that you did not tell them) of correlation not proving causation.
    3. Also teach each of the three persons the difference between hypothesis and theory.
  5. Go to the Unit 4: Assignment #3 Discussion Board and make a new Discussion Board post of at least 200 words in which you
    1. describe how you taught the three persons that correlation cannot be interpreted as causation;
    2. state each of the three persons’ initials (e.g., MG) and their approximate age; and
    3. report the examples each person told you of correlations that should not be confused with causation.

Unit 4: Assignment #4 (due before 11:59 pm Central on Thursday September 9):

  1. To cement your learning about critical thinking and psychological science, read Tesler’s (2016) article, “Can You Believe It? Seven Questions to Ask About Any Scientific Claim.”
  2. Go to the Unit 4: Assignment #2 and #4 Discussion Board and read all the posts made in Unit 4: Assignment #2 by the other members (or member) of your Chat Group.
  3. If you are in a Chat Group with two other members, choose one of the studies that each of your two Chat Group members found and posted (for a total of two studies). If you are in a Chat Group with only one other member, choose two of the studies that the one other member of your Chat Group found and posted.
    1. If you are in a Chat Group with two other members, and one of the two other Chat Group members hasn’t yet posted their Unit 4: Assignment #2 — and the due date for Unit 4: Assignment #2 has passed — you can choose two studies from the Chat Group member who has already posted their Unit 4: Assignment #2.
    2. If you are in a Chat Group with only one other member, and that other member hasn’t yet posted their Unit 4: Assignment #2 — and the due date for Unit 4: Assignment #2 has passed — you can choose two studies from students who are not in your Chat Group.
  4. Now, pretend that instead of your Chat Group member(s) finding and posting these two studies on our class Discussion Board as examples of bad science, two relatives or acquaintances of yours posted the two studies on Facebook, Twitter, or other social media as recommendations to their friends and followers (of good science).
  5. Still pretending that instead of your Chat Group member(s) posting these two studies, your relatives or acquaintances did, write a response to your relatives or acquaintances explaining to them why the two studies they posted are examples of bad science.
    1. You are required to make two separate Discussion Board response posts (replies); each response post should be at least 200 words, and each post should be in response to only one of the two studies.
    2. Even if you are replying to the same student, you must make two separate Discussion Board posts.
    3. Each of your two response posts must incorporate what you learned from Tesler’s (2016) article.
    4. Address your two responses not to your Chat Group Member(s) but instead to fictitious people, such as “Dear Aunt Bessie” or “Hey, Freshman Roommate.”

Unit 4: Assignment #5 (due before 11:59 pm Central on Saturday September 11):

  1. To understand what is often referred to as the “Hierarchy of Scientific Evidence,”
    1. download (to your own computer) and save The Logic of Science’s (no date) Hierarchy of Scientific Evidence graphic (PDF format); (Text-Only Word format), and
    2. read Brunning’s (2015) article, “A Rough Guide to Types of Scientific Evidence.”
  2. Identify where on the Hierarchy of Scientific Evidence case reports (which are also called case studies) lie.
    1. Then, read about some of the most famous psychology case reports in Jarrett’s (2015) article, “Psychology’s Greatest Case Studies – Digested.”
    2. Think about why it was important for the various scientists who reported these case studies to report these case studies.
  3. Identify where on the Hierarchy of Scientific Evidence randomized controlled studies lie.
    1. Then, read about an example of a randomized controlled study in Fradera’s (2017) article, “How Much Are Readers Misled by Headlines that Imply Correlational Findings Are Causal?”
    2. Think about why it was important for these scientists to conduct a randomized controlled study on this topic.
  4. Identify where on the Hierarchy of Scientific Evidence meta-analyses and systematic reviews lie.
    1. Then, to see an example of a journal article reporting a meta-analysis, read the abstract of Hyde and Lynn’s (1988) article, “Gender Differences in Verbal Ability: A Meta-Analysis.”
    2. Think about why it was important for these scientists to conduct a meta-analysis on this topic.
  5. Write one five-paragraph essay of 400 to 500 words arguing either in favor of or against the statement, “All scientific evidence is equally strong.”
    1. You may write either a Reasons/Arguments essay OR an Examples essay. For either type essay, remember the following steps:
      • Begin by jotting down your three Reasons/Arguments or your three Examples.
      • Then, write your three Reasons/Arguments Paragraphs or your three Examples Paragraphs.
      • For each of your three Reasons/Arguments Paragraphs or each of your three Examples Paragraphs, write a Topic Sentence, three Supporting Sentences, and a Conclusion Sentence.
      • Then, write your essay’s Thesis Statement.
      • Then, write your essay’s Introduction Paragraph, including a hook.
      • Then, write your essay’s Conclusion Paragraph, in which you restate your Thesis Statement and end with something witty or profound.
      • Remember that all five of your paragraphs, your Introduction Paragraph, each of your three Reasons/Arguments Paragraphs or your three Examples Paragraphs, and your Conclusion Paragraph, must have a Topic Sentence, three Supporting Sentences, and a Conclusion Sentence.
    2. Save your essay as a PDF and name the file YourLastname_EvidenceEssay.pdf.
  6. Go to the Unit 4: Assignment #5 Discussion Board and do the following:
    1. Make a new Discussion Board post to which you attach your essay, saved as a PDF. This is described in the Course How To (under the topic, “How to Save and Attach a Group Text Chat Transcript”).

Unit 4: Assignment #6 (due before 11:59 pm Central on Monday September 13):

  1. Lack of replication is another red warning flag about the quality of research.
    1. To learn what replication means in research, read CK-12’s (no date) article, “Replication.”
    2. To get a quick overview of replication courtesy of the NBC comedy TV show, “The Good Place,” read Harnett’s 2019 tweet.
    3. To learn why replication is important in research, read an excerpt from Dewey’s (2007) article, “The Importance of Replication.”
      • Be sure to notice in the excerpt from Dewey’s article the methodological reasons he gives for why a result might not replicate.
    4. To learn a statistical reason why a result might not replicate, read through xkcd’s cartoon, “Significance,” and its explanation.
      • Be sure to read the entire cartoon, frame by frame.
      • Although it might seem repetitive, the point is to read closely through all 20 of the frames that report the scientists’ results.
      • Then read the final page’s “Explanation.”
    5. Now that you understand what p-hacking means, read Hobbes’s (2019) tweet about the problems with studies that report a link between “a single food” (e.g., eggs) and “a single health outcome” (e.g., heart disease).
    6. To ensure that you understand the role of chance and statistical probability in producing a result that might not replicate, read Hanna’s (2012) article, “Why Is Replication So Important?”
  2. To understand the difference between the term replication and reproduction, read the handout, “Replication (and Replicability) versus Reproduction (and Reproducibility).”
  3. To refresh your memory about false positives, false negatives, true positives, and true negatives, look at this handout.
    1. To understand the role of false positives in research, watch Veritasium’s (2016) YouTube, “Is Most Published Research Wrong?”
    2. This YouTube is a more complex video than some of the others you’ve watched in this Unit.
    3. Try to understand as much as you can. You’ll have a chance to discuss what you don’t understand in this video with your Chat Group.
  4. Plan to meet online with your small Chat Group for a one-hour text-based Chat using the same chat procedures you used for your Group Chat during Unit 3. Prior to your Chat Group meeting, every member of your Chat Group must do one of the following:
    1. If your last name comes first alphabetically in your Chat Group, watch NOVA’s (2017) video “What Makes Science True?”
    2. If your last name comes last alphabetically in your Chat Group, read all the way through Naro’s (2016) comic strip, “Why Can’t Anyone Replicate the Scientific Studies from Those Eye-Grabbing Headlines?”
    3. If your last name comes neither first nor last in your Chat Group (or if you are in a Chat Group with only two members), read Carroll’s (2017) article, “Science Needs a Solution for the Temptation of Positive Results” and Leyser et al.’s (2017) article, “The Science ‘Reproducibility Crisis’ – and What Can Be Done about It?”
  5. During your one-hour text-based Chat:
    1. Begin by reviewing the articles and videos that everyone in your Chat Group read and watched.
    2. Next, take turns telling the other member(s) of your Chat Group about the article, video, or comic strip that individual Chat Group members read.
    3. Then,
      1. as a group:
        • identify one statistical (probability) reason;
        • identify one researcher motivation; and
        • one news media motivation that could lead to the public seeing headlines such as these two conflicting headlines.
      2. And as a group:
        • propose one solution to the statistical (probability) reason you identified;
        • propose one solution to the researcher motivation you identified; and
        • propose one solution to the the news media motivation you identified.
  6. At the end of your one-hour Chat:
    1. Nominate one member of your Chat Group (who participated in the Chat) to make a post on the Unit 4: Assignment #6 Discussion Board that summarizes your Group Chat in at least 200 words.
    2. Nominate a second member of your Chat Group (who also participated in the Chat) to save the Chat transcript as a PDF, as described in the Course How To (under the topic, “How to Save and Attach a Group Text Chat Transcript”).
      1. Then, this member of the Chat Group needs to make a post on the Unit 4: Assignment #6 Discussion Board and attach the Chat transcript, saved as a PDF, to that Discussion Board post.
    3. Nominate a third member of your Chat Group (who also participated in the Chat) to make another post on the Unit 4: Assignment #6 Discussion Board that states the name of your Chat Group, the names of the Chat Group members who participated the Chat, the date of your Chat, and the start and stop time of your Group Chat.
    4. If only two persons participated in the Chat, then one of those two persons needs to do two of the above three tasks.
    5. Before ending the Group Chat, arrange the date and time for the Group Chat you will need to hold during the next Unit (Unit 5: Assignment #6).
  7. All members of the Chat Group must record a typical Unit entry in their own Course Journal for Unit 4.

Congratulations, you have finished Unit 4! Onward to Unit 5!

 

Open-Access Active-Learning Research Methods Course by Morton Ann Gernsbacher, PhD is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. The materials have been modified to add various ADA-compliant accessibility features, in some cases including alternative text-only versions.
Permissions beyond the scope of this license may be available at http://www.gernsbacherlab.org.