Forensic Research Archive of Manipulation and Exploitation
Illustration: a First World War field-intelligence collage of trench maps, wireless transmission logs and aerial reconnaissance photographs.

Learn · Countering propaganda

Media literacy

Illustration, not an archive object

In one line: teaching people to judge sources and claims produces measurable short-term gains in classroom and survey experiments, but the gains often fade, some field trials found none, several studies find extra distrust of true news, and national rankings that are often cited as proof measure background conditions rather than the effect of teaching.

Evidence: mixed. Two meta-analyses and several randomised trials find positive average effects, but field trials in India and Pakistan found nulls for standard formats, effects faded within weeks in some samples, and no study links a national programme causally to lower susceptibility.

Background

Organised teaching about propaganda in the United States goes back at least to the The Institute for Propaganda Analysis (1937-1942) (1937 to 1942), founded at Columbia University's Teachers College by Clyde R. Miller, Kirtley Mather and Edward A. Filene. By the late 1930s it said one million schoolchildren were using its methods, and it corresponded with some 2,500 teachers, according to Columbia's own history. Those figures are the Institute's and measure reach, not effect.

Edward A. Filene, in a studio portrait by Harris & Ewing

Edward A. Filene, department-store owner and co-founder of the Institute for Propaganda Analysis. Harris & Ewing studio portrait, undated (the Commons record gives 1905; the firm worked 1905 to 1945), Library of Congress (LCCN 2016862257). Public domain, via Wikimedia Commons.

Main Hall, Teachers College, Columbia University, seen from West 120th Street

Main Hall (Main-Zankel Hall), Teachers College, Columbia University, built in 1894, at 525 West 120th Street, New York. The Institute for Propaganda Analysis was founded at Teachers College in 1937. Photo: Bmzuckerman, 23 August 2024, CC BY 4.0, via Wikimedia Commons.

The modern field is broad. A 2019 RAND report by Alice Huguet and colleagues found three different framings of media literacy education (economic, civic and information-quality) and said one limitation of the research base is the variety of ways studies define and measure outcomes. That is still the main problem when comparing results: "media literacy" can mean a one-page list of tips, a six-lesson school unit or a national curriculum.

What was tried

  • Tips. In April 2017 Facebook, with the nonprofit First Draft, placed "Tips to Spot False News" at the top of news feeds in 14 countries and ran them as full-page newspaper advertisements in the United States, the UK, France, Germany, Mexico and India.
  • Lateral reading. Sam Wineburg and Sarah McGrew (Stanford History Education Group) compared 10 PhD historians, 10 professional fact-checkers and 25 Stanford undergraduates evaluating websites (published in Teachers College Record, 2019). Fact-checkers left an unfamiliar site quickly and opened new tabs to see what others said about it ("reading laterally"). They reached more warranted conclusions in a fraction of the time. Historians and students were more often misled by official-looking logos and domain names. The group built this into a curriculum called Civic Online Reasoning.
  • Diagnosing the problem. The group's 2016 report collected 7,804 student responses to 56 tasks across 12 states. A 2021 national study (Breakstone et al., Educational Researcher) gave 3,446 high-school students live-internet tasks: 96% never learned that a climate website was tied to the fossil-fuel industry, two-thirds could not tell news stories from adverts on a home page, and more than half said an anonymous Facebook video shot in Russia was "strong evidence" of US voter fraud.
  • Mandates. New Jersey's governor Phil Murphy signed a law in January 2023 requiring information literacy in the K-12 curriculum, described by his office as the first in the nation. California's AB 873 (2023) directs the state to add media literacy to its curriculum frameworks in several subjects. Fast Company (January 2024) lists California, Delaware, Illinois, New Jersey and Texas as states with such laws. In the EU, the 2018 revision of the Audiovisual Media Services Directive (Article 33a) requires member states to promote media literacy and report on it to the Commission, with the first reports covering 2020 to 2022.
  • International promotion. UNESCO has marked a Global Media and Information Literacy Week since 2011; its member states proclaimed it officially in 2019, and the UN General Assembly adopted it in 2021.

Measured effects

Study Design N Finding Caveats
Jeong, Cho and Hwang 2012, Journal of Communication Meta-analysis of media literacy interventions (all topics) 51 interventions Average positive effect d = 0.37 on knowledge, criticism, attitudes and behaviour; more sessions worked better, more components worse Predates most misinformation research
Huang, Jia and Yu 2024, Communication Research Meta-analysis of misinformation experiments 49 studies, 81,155 people Overall d = 0.60: belief in misinformation d = 0.27, discernment d = 0.76, sharing d = 1.04 Stronger with multiple sessions and among college students than crowdsourced online adults
Guess et al. 2020, PNAS Preregistered survey experiments modelled on Facebook's tips US national sample plus two Indian samples Discernment rose 26.5% (US) and 17.5% (highly educated Indian online sample). Share rating a false headline accurate fell from 32% to 24% in wave 1 Effect still measurable after a median of about 20 days in the US, not in India; no effect in a rural northern Indian sample
Wineburg et al. 2022, Journal of Educational Psychology Matched-control field study, six 50-minute lessons in a district government course 271 treatment, 228 control Treatment students grew significantly in judging the credibility of digital content Not randomised; developers of the curriculum evaluated it
Breakstone et al. 2021, HKS Misinformation Review Online college course, four one-hour modules, pre-post 87 Students reading laterally rose from 3 of 87 to 67 of 87 Pre-post only; developers evaluated it
Moore and Hancock 2022, Scientific Reports MediaWise for Seniors, one-hour self-directed modules, 2020 US election 143 treatment, 238 control Correct discernment rose from 64% to 85%; control 55% to 57%; improved on both true and false items Programme run by the Poynter Institute's MediaWise; short-term
Badrinathan 2021, American Political Science Review In-person field experiment during India's 2019 election 1,224 An hour-long training (reverse image search, fact-check sites) did not significantly improve fake-news identification on average Supporters of the ruling party became less able to spot pro-attitudinal fake news
Ali and Qazi 2023, Journal of Development Economics Randomised experiment in urban Pakistan not recorded here General educational videos had no significant effect; adding personalised feedback on past fake-news engagement improved identification by 0.14 standard deviations Negative but non-significant effect on identifying true news

Bar chart of two studies: students leaving a website to check other sources rose from 3 of 87 to 67 of 87; older adults' correct discernment of fake versus true news rose from 64 percent to 85 percent against 55 to 57 percent in a control group

Two rows of the table above, redrawn. Both studies were run by the programmes' developers and use different measures. Other rows of the table found no significant average effect (Badrinathan 2021) or none for standard video formats (Ali and Qazi 2023). Drawn by FRAME from Breakstone et al. 2021 and Moore and Hancock 2022. CC BY 4.0 (FRAME).

Null results, decay and critiques

  • Distrust of true news. Hoes and colleagues (2024, Nature Human Behaviour) ran three experiments in the United States, Poland and Hong Kong (total n = 6,127). Fact-checks, media literacy tips and news coverage of misinformation all reduced belief in false claims, but all also lowered the credibility of factual information. The authors argue that strategies should also build trust in reliable news. In Guess et al. (2020), after a January 2023 wording correction, the share rating mainstream headlines accurate fell only from 57% to 55% in wave 1 and 59% to 57% in wave 2, far less than the drop for false headlines.
  • Decay and context. In Guess et al. the effect had faded in India by the second wave, and there was none in the rural sample, where social media use was far lower. Badrinathan's and Ali and Qazi's nulls come from in-person or video formats aimed at the kinds of populations that tip campaigns target.
  • Judging is not sharing. Sirlin, Epstein, Arechar and Rand (2021, HKS Misinformation Review, N = 1,341) found digital literacy predicted better accuracy judgments but was not a robust predictor of intentions to share true rather than false headlines. This is a correlational survey, not a test of teaching.
  • Partisan use of skills. Jang and Baek (New Media & Society, panel survey of 808 people during South Korea's 2020 election) found that among people exposed to partisan YouTube channels, higher news literacy went with more biased processing: accepting congenial misinformation and rejecting uncongenial claims.
  • "Did media literacy backfire?" In January 2017 danah boyd of Data & Society argued that a culture of doubt and critique, taught partly through media literacy, feeds conspiracy thinking, and that neither media literacy nor expert fact-checking addresses deeper social divides. In a 2018 SXSW EDU talk she said denialists use media literacy's methods, such as questioning speakers' motives. Renee Hobbs of the Media Education Lab replied that media literacy can backfire when taught badly, that boyd's essay cited no media literacy pedagogy or practice, and that it left out the role of platforms.

danah boyd, photographed at a conference in 2007

danah boyd of Data & Society, author of "Did Media Literacy Backfire?" (January 2017), photographed on 18 September 2007. Photo: Joi Ito, CC BY 2.0, via Wikimedia Commons.

Who uses it and who criticises it

Promoters. UNESCO, the European Union (through the AVMSD reporting duty), state legislatures in the United States with bipartisan votes (per Fast Company), the Stanford group's Civic Online Reasoning curriculum, the Poynter Institute's MediaWise, and Facebook with First Draft. Finland's national programme is described in Finland's media literacy model, and Baltic efforts in Baltic counter-disinformation: Estonia, Latvia and Lithuania.

Critics from the right. In Colorado's 2021 debate on HB21-1103, which set up a state resource bank of media literacy materials, state senator Paul Lundeen (Republican) said "the state becomes the arbiter of the information that you should gather", and other Republicans raised free-speech concerns; Republican senator Kevin Priola backed the bill as a way to counter foreign disinformation. Allum Bokhari of the Foundation for Freedom Online wrote in The American Spectator an article headlined "'Media Literacy': The New Partisan Threat in Education". Fast Company reports that some Republican lawmakers call state bills vague and a vehicle for "woke" ideas.

Critics from the left. Nolan Higdon, a lecturer at UC Santa Cruz and a Project Censored judge, argues for "critical" media literacy and says the current disinformation debate has become a "moral panic" in which calling something fake news is used to silence outsiders. He warns against letting governments and corporations decide what is true.

Critics on method. Hoes et al. and the Guess et al. data show the scepticism spillover described above. danah boyd's critique is about culture rather than measured outcomes.

The Media Literacy Index. The Open Society Institute Sofia has published a Media Literacy Index since 2017. It ranks countries on their "potential for resilience" to disinformation by weighting indicators of media freedom (Freedom House, Reporters Without Borders), education (PISA scores, with reading literacy weighted most), trust in people (Eurostat) and e-participation (UN). Finland ranked first in every edition from 2017 to 2023 and shared first place with Denmark, Ireland and the Netherlands (71 points each) in the 2026 edition. Because the inputs are press freedom, test scores and social trust, the index shows which countries have those conditions. It does not measure whether any media literacy programme changed how people there handle false information.

What the evidence supports and doesn't

  • Supported: short lessons and tips improve performance on headline-rating and source-evaluation tasks soon afterwards, in several countries, with average effect sizes in two meta-analyses from moderate to large.
  • Supported: some effects fade within weeks, and some formats had no effect in field settings in India and Pakistan.
  • Supported: several interventions also lower trust in true news, though usually by less than they lower belief in false news.
  • Not shown: that school or national programmes reduce what people believe or share in daily life over years. Country rankings are correlational.
  • Not settled by research: who chooses the curriculum and its examples, a political argument made by named critics on the right and the left.

See also

See also in Learn

Images

Image Source Licence
Edward A. Filene, Harris & Ewing studio portrait Wikimedia Commons, from the Library of Congress Public domain
Main Hall, Teachers College, Columbia University Wikimedia Commons, Bmzuckerman CC BY 4.0
Two studies of media literacy courses (chart) Drawn by FRAME from Breakstone et al. 2021 and Moore and Hancock 2022 (make_counter_media_literacy_chart.py) CC BY 4.0 (FRAME)
danah boyd, 2007 Wikimedia Commons, Joi Ito CC BY 2.0

Image gaps

  • Photos of Wineburg or McGrew. None found on Wikimedia Commons under an open licence.
  • Period classroom photographs (for example FSA/OWI school photographs of the 1940s). Not used: the Library of Congress catalogue could not be searched from the build environment, and none tied to the page's cases was verified.
  • Facebook's 2017 tips or the Stanford assessment tasks. Not included: copyrighted works with no openly licensed copy found.

Sources

Related