Reading research papers

(adapted from an exercise by Deborah Salon)

Reviewing the literature

• The easiest research project is the one you don’t have to do
• Most reports will require a literature review

Dissemination of research findings

• Research is intended to be generalizable
• This requires disseminating findings

Ways research is disseminated

• Peer-reviewed journal articles
• Technical reports
• Published plans
• Popular press
• Memoranda

The academic publishing process

• Author(s) do research and write a paper
• They determine based on subject matter, personal experience, etc., what journal to send it to
• The journal editor may determine it is not a good fit, and desk reject it
• Otherwise, it is sent to several peer reviewers
• Peer reviewers provide feedback to the author on the scientific content, and recommend rejection, revision, or acceptance (rare)
• If rejected, authors usually send paper to another journal
• If revisions are requested, authors revise the paper and send it back
• At any point, authors may generally post pre-prints
  • Pre-prints may not be peer-reviewed

Evaluating the quality of a publication

• Just because an article is published, doesn’t mean it’s good
• Does the research question make sense?
• Are the data and methods appropriate?
• Does the publication refer to existing literature?
• Where the article is published can be a proxy for quality, but does not necessarily imply it
  • There are good articles in poorly-ranked journals, and poor articles in highly-ranked journals
• Some journals are considered “predatory”—they take fees from desparate or unsuspecting authors, and publish with few or no checks on quality

Reference lists are your friend

• Every academic article, and most research reports, will list the works they cited
• Often, these references are very valuable for additional context
• Many tools (including Google Scholar) let you see which articles cited a given article
• This is very useful for finding more recent research

Learning to read, again

• Most things you’ll read promote active reading
  • They hold your interest and are easy to understand
• Research articles generally do not
  • Very little editing goes into research articles

Learning to read, again

• It’s easy to get distracted when reading a research article
• You can read but not comprehend the words, and have to go back and read again

Learning to read, again

• You need to intentionally and critically engage with the material
• What is the author trying to answer? What do you think they will find? Is that what they find?

How to read a research article

Hubbard and Dunbar (2017)

The structure of a research article

• Title
• Abstract
• Introduction
• Literature review
• Methods
• Results
• Discussion
• Conclusion
• References
• Supplemental materials

Differentiate literature review from findings

How to read a research article

1. Read the title, abstract, and introduction
2. Identify the question or questions the author is asking
3. Think about what you think the answer is
4. Think about what variables might be important to the question
5. Read the body of the article
   • Take notes and compare results to your hypotheses as you go

Let’s read together

“Post-pandemic relocation preferences of remote tech workers” by Tan, Fang, and Lester, in Urban Findings

https://go.unc.edu/plan720reading

What is the research question?

• How interested are tech workers in relocation due to remote work eligibility?

What do we think the answer is?

• What factors are likely to be important?

Additional questions to ask ourselves as we read

• What data did the authors use?
• What answer did the authors find? Did it match our expectations?
• What methods did the authors use?
• Do you have any reason to doubt the findings?
• How should the results impact policy, if at all?

Additional questions to ask ourselves after we read

• Are there opportunities for further research? If so, what?

© xkcd

Now, in groups

• Find a paper to read on findingspress.org
• Read it and answer these questions
• Be prepared to present answers to the class in 20-30 minutes

Writing up your research

• Regardless of how it is disseminated, you will need to write up your research
• Different audiences may need different formats
• Different levels of detail required for different people, but enough information to replicate results should be written down somewhere

The literature review

• Synthesis is the goal
• What is the literature saying overall, not what is each study saying individually

The literature review

• Each paragraph will generally cite many sources, saying roughly the same thing
  • Or contrasting things
• What is the takeaway from this group of articles?

The literature review

Bad

Infranca (2019) examined “a new generation of state interventions in local zoning and land use policy, particularly in the realm of housing.” He reviewed theory behind land use regulation, summarized past housing interventions, summarized new housing interventions, and compared them. California recently considered a bill that would allow much higher densities near transit stations, and Massachusetts encourages municipalities to create “smart growth” zoning districts.

The literature review

Good

Recently, a number of US cities and states have relaxed or are considering relaxing their zoning ordinances, to promote additional housing supply and by extension housing affordability (Infranca 2019; Dillon 2020; Mervosh 2018; Bliss 2019; Phillips 2020)

Assignment 1

• Answer these questions about a (longer) paper
• Available now, due September 3rd

Hubbard, Katharine E., and Sonja D. Dunbar. 2017. “Perceptions of Scientific Research Literature and Strategies for Reading Papers Depend on Academic Career Stage.” PLOS ONE 12 (12): e0189753. https://doi.org/10.1371/journal.pone.0189753.