Reading & Reviewing
The lecture slides do not lend themselves to printing. Those who want pre-printed notes will find the majority of the information in four places: Writing for Computer Science Chapters 10 and 12, Fred Brooks' note on How to do a technical reading program (see link below), and my two pieces on The nature of papers and A way to read (also below).
Videos of the 2012 lectures
You can view
the 2012 versions of the lectures on the University of Cambridge
streaming media service:
Lecture 1: how to read and how to find papers to read.
Lecture 2: how to referee a paper and the reviewing process.
- How to do a technical reading program (1 page) by Prof. Fred Brooks
- How to review a technical paper (5 pages, 90kB PDF) by Alan Meier, with comments by Neil Dodgson
- Mendeley: an online way to manage your bibliography, read and annotate papers, and collaborate with your colleagues.
- Google Scholar — not every paper is on the web and not every paper that is on the web can be found by Google Scholar, but it is a good place to start
- The Computer Laboratory Library resources include a useful list of all bibliographic databases and electronic journals including:
- The University Library
- Universal Catalogue of all library holdings in Cambridge
- eresources@cambridge — a list of many (but not all) electronically available research resources
- ejournals@cambridge — a searchable index of all of the journals to which the University has electronic subscriptions; the search mechanism can be idiosyncratic
- How to Run a Paper Mill (103 pages, 865kB PDF) by John Woodwark — a lighthearted view of the research process
The nature of papers
The starting research student assumes that all research papers are well-written and describe good research that is both correct and important. This is not true.
Research papers can report poor research. The review process is supposed to prevent this, but it is not a perfect process and there is a fair amount of published work out there that is not good. The publication venue is a guide to likely quality. A top-rank journal or conference is unlikely to publish poor research, because the review process is fierce. A lower-rank journal or conference is more likely to let poor research through.
Published research is supposed to be correct. However, there are cases where the authors and reviewers have honestly failed to notice that their data, assumptions, proofs, or conclusions are wrong.
A big mistake made by many new researchers is to think that every single paper is important. Of course, it was important to the authors, but it may well be reporting on a dead end, a trivial result, a system that does not work well, or an idea that has been superceded. So you will find that many papers are unimportant, at least with respect to your particular research area.
Even if a paper is good research, correct, and important, it may not be well written. Sadly, there are some papers that are incomprehensible. It is wrong to assume that the authors are trying to hide something in their poor writing, though this may be the case. Often it is just poor writing skills. Part of this course is to help you to learn how to ensure that your papers are well written.
A way to read
CEM Joad wrote Teach Yourself Philosophy in 1944. His advice can be applied to reading scientific papers. He says:
- Make judicious selection. In philosophy Joad says "Never try to read a whole book at once." In our case, do not try to read all the papers on a subject at once. Joad says "pick out certain chapters, two or three perhaps, which deal with matters that seem to you to be of particular importance or which relate to topics on which you have already read something." We need to pick out those papers that are of most importance to the problem in hand. Of philosophy books, Joad says "Use the introduction and preface to help you identify these." In our case, we need to use something else to determine which papers we should attend to. See Fred Brooks' note (above) for more on that.
- Indulge in intelligent skipping. You do not have to read everything. You do not need to read every word of every paper. Do not be embarrassed to skip. You cannot possibly read every paper in the world; you need to ensure that the ones you spend your time on are the important ones. For some papers you need only look at the introduction and conclusion, with a glance elsewhere. For some, you will need to read the whole thing, but maybe not today. Joad says "omit the unintelligible; skip the repetitions; jump over the boring bits". You are allowed to do this!
- Ensure you understand. Joad says stop reading when you stop understanding, but he also says to interpret "understand" generously. If you do not understand, there is sometimes point in pressing on a little further to see if it becomes clear, but if you really do not get it, there is no point in reading on for the sake of reading on. You need to stop and say "why do I not understand?"; "is this important enough that I need to learn something else so that I can understand?"
- Make notes. The process of writing notes helps enormously with remembering. The process of having to summarise someone else's work helps enormously with understanding. Exercise 1 asks you to precis the content of a four page paper in just one page. This is a great skill to have. Also you need to ensure that you have made good enough notes that you will be able to find any paper again in two years time when it turns out to be important. Too many times have I heard the cry "I know that I have read a paper on exactly this problem, but I cannot remember where!"