Statistical analyses of digital collections: using a large corpus of systematic reviews to study non-citations

Tove Faber Frandsen, Jeppe Nicolaisen


DOI: http://dx.doi.org/10.15291/libellarium.v9i2.253

Abstract


Using statistical methods to analyse digital material for patterns makes it possible to detect patterns in big data that we would otherwise not be able to detect. This paper seeks to exemplify this fact by statistically analysing a large corpus of references in systematic reviews. The aim of the analysis is to study the phenomenon of non-citation: Situations where just one (or some) document(s) are cited from a pool of otherwise equally citable documents. The study is based on more than 120,000 cited studies, and a total number of non-cited studies of more than 1.6 million. The number of cited studies is found to be much smaller than the number of non-cited. Also, the cited and non-cited studies are found to differ in age. Very recent studies tend to be non-cited whereas the cited studies are rarely of recent age (e.g. within the same year). The greatest differences are found within the first 10 years. After 10 years the cited and non-cited studies tend to be more similar in terms of age. Separating the data set into different sub-disciplines reveals that the sub-disciplines vary in terms of age of cited vs. non-cited references. Some fields may be expanding and the number of published studies is thus growing. Consequently, cited and non-cited studies tend to be younger. Other fields may be more slowly progressing fields that use a greater proportion of the older literature within the field. These field differences manifest themselves in the average age of references.

Keywords


bibliometrics; digital collections; systematic reviews; citation analysis; non-citations

Full Text:

PDF EPUB HTML

References


Brooks, T. A. 1985. "Private acts and public objects: An investigation of citer motivations." Journal of the American Society for Information Science 36, 4: 223-229. https://doi.org/10.1002/asi.4630360402

Brooks, T. A. 1986. "Evidence of complex citer motivations." Journal of the American Society for Information Science 37, 1: 34-36. https://doi.org/10.1002/asi.4630370106

Cole, J. R., and S. Cole. 1972. "The Ortega hypothesis." Science 178, 4059: 368-375. https://doi.org/10.1126/science.178.4059.368

Collier, A., L. Heilig, L. Schilling, H. Williams, and R. P. Dellavalle. 2006. "Cochrane skin group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology." British Journal of Dermatology 155, 6: 1230-1235. https://doi.org/10.1111/j.1365-2133.2006.07496.x

Dienes, I. 2012. "A meta study of 26 "how much information" studies: Sine qua nons and solutions." International Journal of Communication 6: 874–906.

Frandsen, T. F., and J. Nicolaisen. In Press. "Citation behavior: A large-scale test of the persuasion by name-dropping hypothesis." Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.23746

Garfield, E. 1977. "To cite or not to cite: A note of annoyance." Current Contents 35, August 29: 5–8.

Gilbert, G. N. 1977. "Referencing as persuasion." Social Studies of Science 7: 113-122. https://doi.org/10.1177/030631277700700112

Higgins J. P. T., and S. Green. 2011. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. www.cochrane-handbook.org.

Hilbert, M. and P. López. 2012. "How to measure the world's technological capacity to communicate, store and compute information? Part I-II: Results and scope." International Journal of Communication 6: 936-955; 956-979.

Kitchin, R. 2014a. "Big data, new epistemologies and paradigm shifts." Big Data %26 Society 1: 1-12.

Kitchin, R. 2014b. The data revolution: Big data, open data, data infrastructures and their consequences. London: Sage. https://doi.org/10.4135/9781473909472

Latour, B. 1987. Science in action. Cambridge, MA: Harvard University Press.

MacRoberts, M. 1997. "Rejoinder." Journal of the American Society for Information Science, 48, 10: 963.

MacRoberts, M. H., and B. R. MacRoberts. 1984. "The negational reference: Or the art of dissembling." Social Studies of Science, 14: 91-94. https://doi.org/10.1177/030631284014001006

MacRoberts, M. H., and B. R. MacRoberts. 1986. "Quantitative measures of communication in science: A Study of the formal level." Social Studies of Science 16: 151-172. https://doi.org/10.1177/030631286016001008

MacRoberts, M. H., and B. R. MacRoberts. 1987a. "Another test of the normative theory of citing." Journal of the American Society for Information Science 38: 305-306. https://doi.org/10.1002/(SICI)1097-4571(198707)38:4<305::AID-ASI11>3.0.CO;2-I

MacRoberts, M. H., and B. R. MacRoberts. 1988. "Author motivation for not citing influences: A methodological note." Journal of the American Society for Information Science 39: 432-433. https://doi.org/10.1002/(SICI)1097-4571(198811)39:6<432::AID-ASI8>3.0.CO;2-2

MacRoberts, M. H., and B. R. MacRoberts. 1989a. "Problems of citation analysis: A critical review." Journal of the American Society for Information Science 40: 342-349. https://doi.org/10.1002/(SICI)1097-4571(198909)40:5<342::AID-ASI7>3.0.CO;2-U

MacRoberts, M. H., and B. R. MacRoberts. 1989b. "Citation analysis and the science policy arena." Trends in Biochemical Science 14: 8-10. https://doi.org/10.1016/0968-0004(89)90077-7

MacRoberts, M. H., and B. R. MacRoberts. 1996. "Problems of citation analysis." Scientometrics 36: 435-444. https://doi.org/10.1007/BF02129604

MacRoberts, M. H., and MacRoberts. 1987b. "Testing the Ortega hypothesis: Facts and artefacts." Scientometrics 12: 293-295. https://doi.org/10.1007/BF02016665

Moed, H. F., and E. Garfield. 2003. "Basic scientists cite proportionally fewer "authoritative" references as their bibliographies become shorter." Proceedings of the 9th International Conference on Scientometrics and Informetrics: 190-196.

Moretti, F. 2013. Distant Reading. London: Verso

Moseley, A. M., M. R. Elkins, R. D Herbert, C. G. Maher, and C. Sherrington. 2009. "Cochrane reviews used more rigorous methods than non-Cochrane reviews: Survey of systematic reviews in physiotherapy." Journal of Clinical Epidemiology 62, 10: 1021-1030. https://doi.org/10.1016/j.jclinepi.2008.09.018

Narin, F. 1987. "To believe or not to believe." Scientometrics 12, 5-6: 343-344.

Nederhof, A. J., and A. F. J. Van Raan. 1987. "Citation theory and the Ortega hypothesis." Scientometrics 12, 5-6: 325-328.

Nicolaisen, J. 2007. "Citation analysis." Annual review of Information Science and Technology 41: 609-641. https://doi.org/10.1002/aris.2007.1440410120

Ranasinghe, I., A. Shojaee, B. Bikdeli, A. Gupta, R. Chen, J. S. Ross, F. A. Masoudi, J. A. Spertus, B. K. Nallamothu, and H. M. Krumholz. 2015. "Poorly cited articles in peer-reviewed cardiovascular journals from 1997 to 2007: Analysis of 5-year citation rates." Circulation 131, 20: 1755-1762. https://doi.org/10.1161/CIRCULATIONAHA.114.015080

Seglen, P. O. 1997. "Why the impact factor of journals should not be used for evaluating research." British Medical Journal 314, 7079: 498-502. https://doi.org/10.1136/bmj.314.7079.497

Small, H. 1987. "The significance of bibliographic references." Scientometrics 12, 5-6: 339-341.

Weale, A. R., M. Bailey, and P. A. Lear. 2004. "The level of non-citation of articles within a journal as a measure of quality: A comparison to the impact factor." BMC Medical Research Methodology 4: 14. https://doi.org/10.1186/1471-2288-4-14

White, H. D. 2001. "Authors as citers over time." Journal of the American Society for Information Science and Technology 52, 2: 87-108. https://doi.org/10.1002/1097-4571(2000)9999:9999<::AID-ASI1542>3.0.CO;2-T

White, H. D. 2004b. "Reward, persuasion, and the Sokal hoax: A study in citation identities." Scientometrics 60, 1: 93-120. https://doi.org/10.1023/B:SCIE.0000027313.91401.9b




DOI: http://dx.doi.org/10.15291/libellarium.v9i2.253

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.
X




Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Libellarium (Online). ISSN 1846-9213 © 2008

ERIH PLUS
doaj.png



Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.