- Individual researchers benefit from online impact assessment
Individual researchers benefit from online impact assessment
Online measuring of impact not yet suited for research assessment exercises
Forward this SURF newsitem.
Fields marked with a * are required and must be filled in
Individual researchers are very interested in evidence of the impact of their publications. Research institutes and independent organisations assessing research have a special interest when comparing groups and organisations for research assessment. Thanks to the possibilities of web based publishing it is now possible to gauge the impact of some publications under certain conditions. New information filters and tools are helping researchers to assess their own progress and to find responses of others to their publications.
The report Users, Narcissism and Control, which was commissioned by SURF, offers a comprehensive overview of the current tracking tools of online publications. The report shows to what extent it is possible to follow in real-time how research results are being downloaded, read, cited, and applied.
Stricter protocols required
The fact that researchers can use these tools does not necessarily mean that this technology is also a legitimate source of information for research assessment. For this application, they need to adhere to a far stricter protocol of data quality and indicator reliability and validity (for example; what does a download imply on the use of the research results). Most new tools do not yet comply with these stricter quality criteria required for them to be used in research assessments.
The authors of the report ‘Users, narcissism and control’, Paul Wouters and Rodrigo Costas (CWTS), explore the explosion of tracking tools that has accompanied the surge of web based information instruments. A total of 16 quite different tools have been assessed. The authors also highlight the potential risks and disadvantages of the new tracking tools. Just some of the tools discussed are: Google Scholar, Microsoft Academic Research, Total-Impact, PlosONE altmetrics, en F1000.
Frank van Harmelen, professor of Knowledge Representation and Reasoning at the VU University Amsterdam writes on the report: "New web-based metrics for scientific impact will make it possible to observe the developments in science in near real-time. I'm very happy to see that such web-based measures of scientific impact are now being considered by leading scholars involved in the science of measuring and analysing science (Scientometrics)".
Research assessment exercises
The report also recommends to start a concerted research programme investigating the dynamics, properties, and potential use of new web based metrics. This programme should relate these new measures to the already established indicators for scientific and scholarly impact. It can provide insight in how these developing metrics could be applied in research assessment exercises.