Methodology: State of the News Media | Pew Research Center – Pew Research Center’s Journalism Project

The State of the News Media fact sheets use a range of different methodologies to study the health of the U.S. news industry, including custom analysis of news audience behavior, secondary analysis of industry data and direct reporting to solicit information unavailable elsewhere.

State of the News Media industry data

The State of the News Media fact sheets consist of data originally generated by other individuals or organizations that Pew Research Center then collected and aggregated.

For the data aggregated from other researchers, Pew Research Center’s team took several steps. First, Center researchers tried to determine what data had been collected for the media sectors studied and by whom for the media sectors studied. In many cases, this included securing rights to data through licensing fees or other means, and often included paying for use of the data.

Next, researchers studied the data closely to determine where elements reinforced each other and where there were apparent contradictions or gaps. In doing so, Pew Research Center endeavored to determine the value and validity of each dataset. That, in many cases, involved going back to the sources that collected the data in the first place. Where there were conflicts, the Center included all relevant sources and tried to explain their differences, either in chart notes or in the text.

All sources are cited in chart and graphic notes or within the text of the report. Some data providers may update data for past and future years on an annual basis.

Analysis of Comscore television audience data

In 2018, Pew Research Center began using data from Comscore StationView Essentials® and TV Essentials® for our TV news audience analysis, and, as such, the current fact sheets include figures from 2016 onward. Local TV news audience data is from Comscore StationView Essentials®, while cable and network TV audience data is from Comscore TV Essentials®.

Earlier versions of the three TV fact sheets included data stretching back to 2007 and used data from Nielsen Media Research. That data is available in the 2017 archived fact sheets. Because of differences between the two sets of data points, figures from prior years are not directly comparable with the Comscore data shown here.

Analysis of Comscore digital audience data

In order to provide as clear a sense as possible of the digital audience data of U.S. news organizations, researchers took several steps using digital audience measurement data from Comscore, a cross-platform measurement company.

After the initial selection process, each Comscore entry representing an outlet or website (“entity”) was individually vetted by researchers and unusual shifts in data over time were checked with Comscore analysts. Using feedback from analysts, entities that registered increases or declines in unique visitors over the time period studied (October to December of each year from 2014 to 2018) that were likely due to inorganic changes in measurement or site structure were considered to not have data that could be trended and were removed from the analysis.

For all outlets, entities that are representative of an entire web domain (“total domain entities”) were used whenever possible. When domain entities were incomplete – i.e., when they did not represent traffic to an entire web domain – custom entities that represent the total domain were used when available. Multiple entities of the same URL that were the result of a structure change in the Comscore database over time were considered comparable. Researchers consulted Comscore analysts throughout the entity selection process.

Selection

Newspapers: Researchers first assembled a list of the top 49 newspapers by average Sunday circulation for Q3 2015, Q3 2016, Q3 2017, Q3 2018, Q3 2019, and the six months ending Sept. 30, 2020, according to the Alliance for Audited Media data. The Wall Street Journal was then added to the list, as it is one of the largest papers in the U.S., but does not have a Sunday edition, to produce a list of the top 50 newspapers by circulation. Each was matched with its associated total-domain entities in Comscore. Researchers then analyzed the Comscore data for October, November and December in each year. The following 50 entities were used in 2020:

AJC.COM

AZCENTRAL.COM

BALTIMORESUN.COM

BOSTONGLOBE.COM

BUFFALONEWS.COM

CHICAGOTRIBUNE.COM

CHRON.COM

CLEVELAND.COM

COURANT.COM

DALLASNEWS.COM

DENVERPOST.COM

DISPATCH.COM

ELNUEVODIA.COM

FREEP.COM

INDYSTAR.COM

INQUIRER.COM

JSONLINE.COM

KANSASCITY.COM

LATIMES.COM

MERCURYNEWS.COM

MIAMIHERALD.COM

MLIVE.COM

MYSANANTONIO.COM

NEWSDAY.COM

NJ.COM

NYDAILYNEWS.COM

NYPOST.COM

NYTIMES.COM

OCREGISTER.COM

OMAHA.COM

OREGONLIVE.COM

ORLANDOSENTINEL.COM

PILOTONLINE.COM

REVIEWJOURNAL.COM

SACBEE.COM

SANDIEGOUNIONTRIBUNE.COM

SEATTLETIMES.COM

SFCHRONICLE.COM

STARADVERTISER.COM

STARTRIBUNE.COM

STLTODAY.COM

SUN-SENTINEL.COM

SUNTIMES.COM

SYRACUSE.COM

TAMPABAY.COM

TIMESUNION.COM

TWINCITIES.COM

USATODAY.COM

WASHINGTONPOST.COM

WSJ.COM

Digital-native news outlets: Researchers assessed all domains from 11 Comscore categories (Business to Business, Directories/Resources, Entertainment, Games, Lifestyles, Multi-Category, News/Information, Services, Social Media, Sports and Technology) with at least 10 million average monthly unique digital visitors in the fourth quarter of 2018. From that set of entities, they then selected digital-native news outlets using the following criteria:

  1. Must be “born on the web,” i.e. not the website of a legacy news brand (note: may be owned by a legacy media company).
  2. It is a publisher of original content about news, defined as current events affecting public life (can include both original reporting and commentary/analysis). Sites are judged by an assessment of the material appearing on their homepage. A review of top stories on the homepage must render some evidence of original reporting, such as interviews, eyewitness accounts or referral to source documents, by a dedicated reporter/editorial staff. Sites are also judged as news publishers if they self-describe as an organization that produces news, either in the subject headers/navigation bar and/or in their “about” or advertising section through usage of terms like “news,” “journalism,” “covering” or “informing.”
  3. It is not entirely focused on reviews, advice, recipes or unedited raw data.
  4. It is not primarily a user-generated or aggregated content platform (such as Medium, Reddit or Wikipedia). Branded content such as NBA.com was also excluded.

The following 37 entities were used for 2018:

247SPORTS.COM

90MIN.COM

BGR.COM

BLEACHERREPORT.COM

BUSINESSINSIDER.COM

BUSTLE.COM

BUZZFEED.COM

BUZZFEEDNEWS.COM

CNET.COM

COMICBOOK.COM

DEADSPIN.COM

DIGITALTRENDS.COM

EATER.COM

ELITEDAILY.COM

ENGADGET.COM

GIZMODO.COM

HOLLYWOODLIFE.COM

HUFFINGTONPOST.COM

IGN.COM

INVESTOPEDIA.COM

MARKETWATCH.COM

MASHABLE.COM

MAXPREPS.COM

POLITICO.COM

POLYGON.COM

REFINERY29.COM

SBNATION.COM

SLATE.COM

TECHRADAR.COM

THEDAILYBEAST.COM

THEROOT.COM

THEVERGE.COM

THISISINSIDER.COM

THRILLIST.COM

TMZ.COM

UPROXX.COM

VOX.COM

A fresh cohort was chosen in each year using the criteria above. Audience information was calculated based on the cohort for that year to reflect the characteristics of the most popular digital-native sites at the time.

Analysis

For each website, minutes per visit and unique visitors for October to December of each year come from the Comscore Media Metrix® Multi-Platform U.S. database for Total Digital Population.

Comparisons year over year are between monthly averages of October to December data in each year.

For sites that didn’t meet the reporting threshold for one month out of a quarter, researchers averaged the two months for which data was available.

Digital-native news outlet audit

Researchers studied several outreach avenues that digital-native news outlets could take to engage with their audiences. For mobile apps, researchers searched the Google Play and iOS App Store for official apps from each outlet. For newsletters, researchers searched each outlet’s site for a sign-up form. For Apple News, researchers searched the Apple News app for official channels for the outlet. For Flipboard, researchers searched the Flipboard app and website for official pages for the outlet. For podcasts, researchers searched the iTunes podcast store for podcasts from the outlet and performed a search on each outlet’s site. An outlet was determined to support comments if at least one of the first five stories on its homepage supported comments at the time of analysis. For social media outreach, researchers searched for official pages, accounts or channels for the outlet on each platform, as well as on the outlet’s primary website.

Digital economic analysis

Economics data for the digital fact sheet comes from eMarketer’s U.S. Ad Spending estimates. Data from previous years is updated annually. In previous years, eMarketer included data about Amazon’s display ads. This data was not included in eMarketer’s 2018 display ads dataset, so Amazon is not broken out separately in any year for this analysis. For 2017 and 2018, eMarketer adjusted its format definitions, substantially changing the way the categories were structured. Because of these changes, data from 2016 and earlier is not included in this report but can be found in archived versions of the fact sheets.

Employment and wage data

Employment and wage trends are based on data from the Bureau of Labor Statistics (BLS) Occupational Employment and Wage Statistics (OEWS) survey. Pew Research Center uses the OEWS data to estimate employment and median wages for four media-related occupations in five information-producing industries. The OEWS survey produces annual estimates of occupational employment and wage rates for full- and part-time wage and salary workers (excluding the self-employed) in nonfarm establishments.

The Center’s analyses focus on five industries: 1) newspaper publishers, 2) television broadcasting, 3) cable and other subscription programming, 4) radio broadcasting and 5) other information services, which is the best match for digital-native publishers. (See below for more information on this category.)

Within each industry, the number of newsroom employees is limited to four occupations associated with news production: 1) news analysts, reporters and correspondents; 2) editors; 3) photographers (e.g., photojournalists) and 4) television, video, and motion picture camera operators and editors (e.g., news videographers, television news video editors).1 This allows a focus on newsroom staff rather than those on the business or distribution side, such as advertising sales agents, printing press operators and delivery truck drivers. (It does not allow for the inclusion of layout artists, designers or digital producers, as there are no occupation codes for employees doing this work specifically in the group of media and communication equipment workers.)

The OEWS survey produces employment and wage estimates annually for over 800 occupations. National occupational estimates for specific industries are also available. The wage data is adjusted for inflation using annual averages from the BLS Consumer Price Index Research Series (CPI-U-RS) with the latest year in the trend as the base year.

Using this single source of data allows for comparable employment estimates across the industry groups by standardizing the occupations included, rather than relying on estimates that are either specific to certain sectors of the news media industry or are not produced on an annual basis.

OEWS data, however, does carry some limitations. For example, OEWS uses the federal government’s Standard Occupational Classification system, making the occupations for which they have wage data not quite as fine-grained or industry-specific as the salary data published by private sources, such as the Radio Television Digital News Association (RTDNA), which uses a classification system tailored to include specific occupations in the television and radio industries, such as weathercaster, sports anchor or tape editor.

Additionally, OEWS uses the federal government’s North American Industry Classification System (NAICS), which subsumes the category most closely corresponding to digital-native publishers – internet publishing and broadcasting and web search portals – under the broader other information services industry, and only allows the data to be broken out for this umbrella category.

The estimates of this broader industry still reflect the employment and wage trends of digital-native publishing for two reasons. First, internet publishing and broadcasting and web search portals make up about three-quarters (79%) of the other information services industry, according to the 2018 Current Employment Statistics survey. Second, it has represented 60% or more of the other information services industry since 2008.

However, the number of digital-native newsroom employees over time may be slightly elevated, as it includes newsroom employees from other categories in the broader other information services industry, such as those from the news syndicates category. It also may include legacy news organizations that are now published exclusively on the web. Because of these limitations, and given that digital news is a rapidly evolving industry, the most valuable information derived from this data is not annual employment and wage levels, but trends over time.

Note that the employment data for the other information services industry is shown starting from 2008. This is because OEWS switched from the 2002 NAICS to the 2007 NAICS beginning with the May 2008 estimates, and the other information services industry is defined differently under the two systems. Data for the newspaper, radio, television and cable industries were not similarly affected. Additionally, OEWS may withhold from publication employment and wage data for some occupations due to, for example, failure to meet Bureau of Labor Statistics quality standards or the need to protect the confidentiality of survey respondents. Gaps in the trend data presented reflect these limitations in the OEWS data.

The OEWS survey produces annual estimates by combining six panels of data collected over a three-year period. Every year, two new panels of data are added, and the two oldest panels are dropped, resulting in a moving average staffing pattern. The three years of employment data is benchmarked to represent the total employment for the reference period. Because annual estimates from overlapping three-year periods are based on nearly the same data, it is difficult to make conclusive year-to-year comparisons. Comparisons are best made between non-overlapping periods. OEWS data is not designed for making comparisons through time, and such comparisons should be interpreted with caution.

Acknowledgments

Pew Research Center is a subsidiary of The Pew Charitable Trusts, its primary funder. This is the latest report in Pew Research Center’s ongoing investigation of the state of news, information and journalism in the digital age, a research program funded by The Pew Charitable Trusts, with generous support from the John S. and James L. Knight Foundation.

The state of the news media fact sheets are the product of decades of work by current and former Pew Research Center staff members. For the current batch of fact sheets, assistance in data analysis was provided by Kirsten Worden and Ricki Wood. Sara Atske provided web producing and graphic support, and Shannon Greenwood and David Kent provided copy editing support, while Hannah Klein, Rachel Weisel and Andrew Grant provided communication support.