How is impact measured?
Researchers work extremely hard (most of the time) to produce a thorough analysis and provoke discussion on a particular topic. It would be very frustrating if it ever felt like all of that hard work did not pay off and had little impact. Because of that, many new ways of engaging with a wider audience and fostering a larger and wider impact have been thought out. The biggest challenge in all of this, is to find a way of accurately monitoring how these changes have improved impact – have more people really been engaged? This gives rise to two main methods of measuring and analysing impact – bibliometrics and altmetrics.
Bibliometrics are focused on measuring academic impact in the form of publication data. They provide a normalised and quantitative view on the full life cycle of a publication, and is not just simply monitoring the number of citations a publication may have. Thing 16 discussed this process in some detail, and outlined the four main stages that bibliometrics focus on for a publication. These are:
- Point of publication – Which journal has a publication been accepted into? This is an initial indication of how the community/industry will perceive the research, without actually reading or assessing the research directly.
- Post publication – How many citations has a publication received? This would appear to be a very black and white situation, but this has two main issues. Firstly, this process takes place over a number of years and secondly, different metrics provide different numbers for the same publication. Which metric should we actually trust?
- Transfer of knowledge – What is the citation chain for a publication? A publication can bridge across many industries and applications and be the tool for new ideas.
- Collaboration – What does my collaboration network look like? When collaborating a network much like a family tree is created. The size and depth of such a network can be another measure of how impactful a publication or author is.
This is an interesting topic and something that should be considered in order to make sure your research reaches the largest possible audience. One aspect that I had never considered is the time scale for being able to fully assess the impact of a particular publication. In order to have a quick look on bibliometrics for myself, I searched three publications written by Dr Cyril Cayron who is a researcher and lecturer at EPFL, Switzerland on three different metrics. The research of Dr Cyril Cayron can be found at: https://www.researchgate.net/profile/Cyril_Cayron. The differences in these metrics can be seen in the table below. The number of citations from Google Scholar always appears to be higher than the number from the other metrics.
Alternative metrics look at other ways of measuring the impact of research, and do not focus on traditional publication impact. This is a relatively new topic, but provides a valid way of assessing the impact of many of the other ways of sharing research that 23Things has already touched on, such as presentations, software and even blogs such as this one. Bibliometrics typically make use of normalised data to ensure a fair comparison from one publication to the next can be made. This is an area that is still being improved for altmetrics, and defining how to make a fair comparison is still being debated. ImpactStory and altmetric.com are two of the main resources for looking at the alternative impact.
What metrics would I use?
Looking into how impactful your research is can only be a good thing. I personally think bibliometrics provide a strong quantitative measure of how much impact your research has. It does take a long time, and as a new researcher this is something that I need to start monitoring when I first publish and continue to monitor as I continue as a researcher. I think having a metric of choice is a good idea, because there are differences between them despite the overall trend looking the same. By monitoring the impact over time as well as at particular times in the publication life cycle can be helpful for future publications. Altmetrics are sometimes a little less clear cut, but still provide a measure of impact. Research can be shared in a variety of ways and impact can be experienced in a number of ways. I personally think the world of altmetrics will grow, and will be adopted by more and more researchers. There will even be researchers researching about altmetrics…