Reading a paper

We open with a discussion on how to read research papers. We build on S. Keshav’s three-pass method, described in How to Read a Paper [1]. Read this short article before the first session; where it differs from what follows, follow the version given here.

The method involves reading a paper in up to three passes, each with a distinct purpose and time allocation. For many papers, one pass is sufficient. Learning to recognise which papers warrant closer attention is an essential part of the process.

One difficulty many readers face is that they read academic literature in a depth-first way. This means you start reading a paper until you find terms you do not understand. You then spend time learning what those terms mean before continuing with the rest of the paper.

However, while working out those terms, you often come across new terms that you have not seen before. You then go deeper and deeper down the rabbit hole. This can mean spending an enormous amount of time getting to grips with background terms that may or may not be important to the paper itself.

You may also spend a great deal of time working through a paper at this detailed level, only to discover that it is not relevant to your work. That time is then wasted.

By reading a paper in multiple passes, you can focus on a different task each time. You can go progressively deeper into the paper as necessary.

When doing research, you will want to read in a much more breadth-first way. This means reading widely before reading in depth. You can read 100 papers at a surface level to get a sense of the field. Then, based on the techniques and data that recur across papers, it becomes easier to identify which are important, influential, and relevant to your work. Those are the papers that earn a second pass.

Only a handful of papers are ever worth reading at the deepest level. Working breadth-first ensures that by the time you study a paper in that much detail, it will benefit both you and your project.

The three passes

First pass — five to ten minutes — “I know what this is about”.
Read the title, abstract, and introduction, then the section and subsection headings and the conclusions. Briefly review the references, noting those you have already read. By the end of this pass, you should be able to answer five questions:

What problem are the authors trying to solve?
What techniques do they propose or use?
What data do they use, and are those data publicly available?
What do they conclude?
What is the next step?

Use your answers to decide whether the paper warrants a second pass.

Second pass — up to an hour — “I could explain this to someone else”.
Read the main text carefully, setting aside technical details such as proofs, dense algorithms, and mathematics. Examine the figures closely. Are the axes labelled? Are error bars included where appropriate? Are the comparisons fair? Do the captions make claims that the figures support? Note relevant references for further reading. By the end of this pass, you should be able to explain the paper’s main argument and supporting evidence to someone who has not read it.

Third pass — several hours — “I could implement this”.
Work through how you would reconstruct the study, starting from the authors’ stated assumptions. Compare your reconstruction with the paper to identify assumptions and dependencies that remain implicit. By the end of this pass, you should be able to assess the work’s strengths and weaknesses and identify experiments needed to support or extend its conclusions.

Questions 2 and 4 look redundant and are not. When an incremental method carries a broad conclusion, comparing the two answers catches the overclaim in the first ten minutes.

Question 3 is the one the reproduction turns on. Without access to the data, you cannot reproduce the result, and access alone does not make it feasible. Check before you commit to a paper.

Question 5 changes meaning as you go. First it asks what the authors propose next; later, whether that is the useful next step or only the convenient one. Project ideas come from the second reading.

Two further ideas deserve particular attention in this strand. Context concerns how a paper relates to existing research. It is essential to the Reviewer 2 role: a well-founded critique requires an understanding of where the paper fits within the field. Build it during the second pass, when following references helps establish those relationships. The Deep Research features of frontier LLMs are typically very useful for this kind of task. Clarity concerns how effectively the paper communicates its ideas and evidence; the referee report asks you to assess it directly.

What the LLM cannot do for you

Reading a paper is how you come to understand the research landscape: how other people have tackled the problem you care about. The survey itself is not the useful artefact — it is worth less now that anyone can generate one — and producing it without understanding the space yourself will not help you in real research.

LLMs are very good at finding relevant literature, drawing a narrative around a field, pointing you at the papers that matter, and helping you digest them. Use them for all of that. The understanding has to be yours, and the parts of this course that are defended orally are where its absence shows.

PaperBlitz provides practice in the first pass. Each session begins with an unfamiliar paper, a fixed time limit, and the five questions. The paper is released during the session, and you read it and prepare your answers there.

# Reading a paper {.unnumbered} ## Reading a paper We open with a discussion on how to read research papers. We build on S. Keshav's three-pass method, described in [*How to Read a Paper*](https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf) [@keshav2007]. Read this short article before the first session; where it differs from what follows, follow the version given here. The method involves reading a paper in up to three passes, each with a distinct purpose and time allocation. For many papers, one pass is sufficient. Learning to recognise which papers warrant closer attention is an essential part of the process. One difficulty many readers face is that they read academic literature in a depth-first way. This means you start reading a paper until you find terms you do not understand. You then spend time learning what those terms mean before continuing with the rest of the paper. However, while working out those terms, you often come across new terms that you have not seen before. You then go deeper and deeper down the rabbit hole. This can mean spending an enormous amount of time getting to grips with background terms that may or may not be important to the paper itself. You may also spend a great deal of time working through a paper at this detailed level, only to discover that it is not relevant to your work. That time is then wasted. By reading a paper in multiple passes, you can focus on a different task each time. You can go progressively deeper into the paper as necessary. When doing research, you will want to read in a much more breadth-first way. This means reading widely before reading in depth. You can read 100 papers at a surface level to get a sense of the field. Then, based on the techniques and data that recur across papers, it becomes easier to identify which are important, influential, and relevant to your work. Those are the papers that earn a second pass. Only a handful of papers are ever worth reading at the deepest level. Working breadth-first ensures that by the time you study a paper in that much detail, it will benefit both you and your project. ::: {.callout-note icon=false} ## The three passes **First pass — five to ten minutes — "I know what this is about".**\ Read the title, abstract, and introduction, then the section and subsection headings and the conclusions. Briefly review the references, noting those you have already read. By the end of this pass, you should be able to answer five questions: 1. What problem are the authors trying to solve? 2. What techniques do they propose or use? 3. What data do they use, and are those data publicly available? 4. What do they conclude? 5. What is the next step? ::: {.column-margin} **Questions 2 and 4** look redundant and are not. When an incremental method carries a broad conclusion, comparing the two answers catches the overclaim in the first ten minutes. **Question 3** is the one the reproduction turns on. Without access to the data, you cannot reproduce the result, and access alone does not make it feasible. Check before you commit to a paper. **Question 5** changes meaning as you go. First it asks what the authors propose next; later, whether that is the useful next step or only the convenient one. Project ideas come from the second reading. ::: Use your answers to decide whether the paper warrants a second pass. **Second pass — up to an hour — "I could explain this to someone else".**\ Read the main text carefully, setting aside technical details such as proofs, dense algorithms, and mathematics. Examine the figures closely. Are the axes labelled? Are error bars included where appropriate? Are the comparisons fair? Do the captions make claims that the figures support? Note relevant references for further reading. By the end of this pass, you should be able to explain the paper's main argument and supporting evidence to someone who has not read it. **Third pass — several hours — "I could implement this".**\ Work through how you would reconstruct the study, starting from the authors' stated assumptions. Compare your reconstruction with the paper to identify assumptions and dependencies that remain implicit. By the end of this pass, you should be able to assess the work's strengths and weaknesses and identify experiments needed to support or extend its conclusions. ::: Two further ideas deserve particular attention in this strand. **Context** concerns how a paper relates to existing research. It is essential to the Reviewer 2 role: a well-founded critique requires an understanding of where the paper fits within the field. Build it during the second pass, when following references helps establish those relationships. The Deep Research features of frontier LLMs are typically very useful for this kind of task. **Clarity** concerns how effectively the paper communicates its ideas and evidence; the referee report asks you to assess it directly. ::: {.callout-important icon=false} ## What the LLM cannot do for you Reading a paper is how you come to understand the research landscape: how other people have tackled the problem you care about. The survey itself is not the useful artefact — it is worth less now that anyone can generate one — and producing it without understanding the space yourself will not help you in real research. LLMs are very good at finding relevant literature, drawing a narrative around a field, pointing you at the papers that matter, and helping you digest them. Use them for all of that. The understanding has to be yours, and the parts of this course that are defended orally are where its absence shows. ::: **PaperBlitz** provides practice in the first pass. Each session begins with an unfamiliar paper, a fixed time limit, and the five questions. The paper is released during the session, and you read it and prepare your answers there.