“Unscrupulous writers often count on the fact that most people don’t bother reading footnotes or tracking down citations” – Daniel Levitin
Today on Core Ideas, we continue to explore the spread of Contagious Ideas with a look at influential scientific articles. We ignore all the problems associated with distilling nebulous concepts into a single metric, and do it anyway, asking, ”What does ‘highly cited’ mean within paleolimnology?”
The ultimate purpose of publishing scientific papers is to share new knowledge, so it can be read and learnt from. However, with no way to measure how often something is read, it is difficult to quantify whether a particular paper is significant or influential. The number of times a paper is “cited” (referred to as a source in other papers) is often used as a metric to compare papers. There are many caveats associated with reducing a broad concept into a single number, and it is unclear whether citation impact is actually a good way to measure influence; nevertheless “if all you have is a hammer, everything looks like a nail.”
For a sense of what it means to be “highly cited”, our scale needed to be calibrated. A 2014 analysis of the Web of Science database, revealed the highly skewed distribution of citations, with most papers cited just once, or not at all, while the highest cited paper (Lowry et al. 1951) had over 300K citations. For a more recent citation value, we checked Google Scholar only to find that Lowry et al. 1951 has a mere 222,572 citations in that database. This is a real issue when comparing citation numbers, as it involves databases of databases with incomplete overlap. Raising the question, is it even possible to determine a true number?
Another concern with citations is variability among fields, as the number of researchers and average publication record vary wildly across disciplines. So we checked some famous scientists and scientific works:
- Charles Darwin – On the Origin of Species (1859) ~52K citations
- Albert Einstein – Quantum mechanics (1935) ~20K citations
- Watson and Crick – Physical structure of DNA (1953) ~14K citations
With these limitations in mind, we got paleolimnology specific and looked at the citation records of the past and present Presidents of the International Paleolimnology Association and Editors-In-Chief of the Journal of Paleolimnology on Google Scholar and then clicked through their research network of co-authors to determine what might qualify as a highly cited paleolimnology paper (i.e. we excluded textbooks and IPCC reports).
At the time we were only able to find a single paleolimnology paper with over a 1000 citations, a paper about the collapse of the Maya civilization published in Nature. However, after recording the show, it was brought to our attention that we missed a big one published in the Journal of Paleolimnology about the use of LOI to estimate organic and carbonate content of sediments that has over 3700 citations.
As expected, books, methodological papers, and reviews tend to be well cited, but our little exploration definitely highlighted the ambiguous nature of comparing citation counts. While arctic studies tend to figure prominently among individual researchers highest cited works, we were quite surprised by the lack of a presence from large projects with extensive author lists (e.g. PIRLA, SWAP). In hindsight this absence makes sense, as the impact of those types of studies will typically be spread over many papers.
Our final conclusion from the exercise was that to really launch your citation numbers to stratospheric heights, contribute to an IPCC assessment report.