A huge quantity of academic research that seeks to understand how science works relies on citation counts to measure the value of knowledge created by scientists. This measure of scientific impact is so deeply embedded in the literature that it's absolutely crucial to know if it’s reliable. So today I want to look at a few recent articles that look into this foundational question: are citation counts a good measure of the value of scientific contributions?
This podcast is an audio read through of the (initial draft of the) post Do Academic Citations Measure the Impact of New Ideas?, originally published on New Things Under the Sun.
Articles Mentioned:
Teplitsky, Misha, Eamon Duede, Michael Menietti, and Karim R. Lakhani. 2022. How Status of Research Papers Affects the Way They are Read and Cited. Research Policy 51(4): 104484. https://doi.org/10.1016/j.respol.2022.104484
Gerrish, Sean M., and David M. Blei. 2010. A Language-based Approach to Measuring Scholarly Impact. Proceedings of the 26th International Conference on Machine Learning: 375-382. http://www.cs.columbia.edu/~blei/papers/GerrishBlei2010.pdf
Gerow, Aaron, Yuenig Hu, Jordan Boyd-Graber, and James Evans. 2018. Measuring Discursive Influence Across Scholarship. Proceedings of the National Academy of Science 115(13): 3308-3313. https://doi.org/10.1073/pnas.1719792115
Poege, Felix, Dietmar Harhoff, Fabian Guesser, and Stefano Baruffaldi. 2019. Science Quality and the Value of Inventions. Science Advances 5(12). https://doi.org/10.1126/sciadv.aay7323
Yin, Yian, Yuxiao Dong, Kuansan Wang, Dashun Wang, and Benjamin Jones. 2021. Science as a Public Good: Public Use and Funding of Science. NBER Working Paper 28748. https://doi.org/10.3386/w28748
Card, David, and Stefano DellaVigna. 2020. What do Editors Maximize? Evidence from Four Economics Journals. The Review of Economics and Statistics 102(1): 195-217. https://doi.org/10.1162/rest_a_00839
Tahamtan, Iman, and Lutz Bornmann. 2019. What do Citation Counts Measure? An Updated Review of Studies on Citations in Scientific Documents Published Between 2006 and 2018. Scientometrics 121: 1635-1684. https://doi.org/10.1007/s11192-019-03243-4
Kousha, Kayvan, and Mike Thelwell. 2016. Are Wikipedia Citations Important Evidence of the Impact of Scholarly Articles and Books? Journal of the Association for Information Science and Technology 68(3): 762-779. https://doi.org/10.1002/asi.23694