Research Assessment


Twitter account by knowing that they have 20 followers, than we can judge the value of a research paper by knowing that it has 20 citations. While 20 citations may be considered above average in fields such as the social sciences and mathematics, it would be far less impressive in areas such as molecular biology or immunology, where there tend to be higher rates of citation. Similarly, if we are to consider a researcher’s


impact on Twitter in terms of how many followers they have, then we need to compare like with like. This is something most people would intuitively recognise if a comparison was made between the number of followers an average researcher has and the number of followers that Barack Obama has.


Followers depend on use However, comparing like with like not only means comparing individuals or organisations in similar fields, but also comparing Twitter streams that are being used in similar ways. In the traditional research environment, it may be considered sufficient to compare the impact of an author’s work with the average number of citations in their specific field, with other factors that may influence the number of citations, such as the type of paper, given less consideration. In comparison, the purpose of a researcher’s Twitter account may have a significant impact on the elements that can be measured, such as the number of followers. A Twitter user’s number of followers is,


in many ways, the simplest indicator of a user’s impact. At first glance it would seem reasonable to suggest that a researcher with a Twitter stream of high-quality content could be expected to have far more followers than one of lower-quality content, at least once we have eliminated factors such as ‘celebrity’ and focus solely on the research community. However, it is very easy to see how such


an assumption quickly falls down because researchers will use Twitter in different ways and researchers are rarely all things to all people. The researcher who uses Twitter for engaging with a small number of colleagues online is likely to have far fewer followers than the researcher who is far more of a generalist and appeals to a wider community. At the other end of the scale, the researcher


who is using Twitter as a platform for sharing live data from various experiments may find they have no followers at all. After all, those making use of the data are unlikely to be reading it in a traditional manner, but rather will be automatically downloading the data for


www.researchinformation.info


later analysis and may be leaving no discernable traces on the web. In reality, a user’s number of followers


would, hopefully, be quickly recognised as far too simplistic an indicator of someone’s Twitter impact to be of any use. There is little uniformity in the judgements that are made when people decide whether or not someone is worth following; some people seemingly follow anyone they come across, whilst others will try to restrict themselves to following a limited number of users, even if they may engage with members of the wider community when prompted.


FEATURE


about a researcher’s online impact. Nonetheless the popularity of tools like Klout are likely to encourage a tendency to think that there is only one right way of using social media. This risks increasing people’s focus on the metric rather than the underlying aim, whether this is the successful engagement with colleagues, the public, or something else. The difficulty in establishing new metrics


does not mean that attempts shouldn’t be made to establish them, but rather emphasises the need to proceed with care. The research community isn’t unique in looking for social- media metrics, although its requirements are


‘A user’s number of followers is far more to do with the number of people they follow and the number of updates they have made than quality’ As such, a user’s number of followers is far


more to do with the number of people they follow and the number of updates they have made than any objective notion of quality; the young researcher following a thousand accounts and sending out a thousand updates will quickly find themselves with more followers than the thoughtful professor engaging in meaningful conversation. It is therefore necessary to consider more subtle metrics that take an increasing number of factors into account.


Twitter metrics Unsurprisingly, the popularity of Twitter has meant that there are a number of online services that have emerged with the promise of helping with social media analysis and making the process simpler. These social media analysis services vary in both the metrics that they offer and the openness of the metrics. Overall, however, the tendency is to simplify. For example, in the case of the increasingly- popular Klout (http://klout.com) a person’s online presence can be reduced to a single Klout Score, in theory an overall indicator of social media influence. Such a score is obviously meaningless if we are


to accept that there is no single right way to use these technologies. Klout does supplement this score with three other indicators representing different aspects of a user’s social media presence: True Reach, which is the number of people a user influences; Amplication, or how much a user influences people; and Network Score, which is the influence of people in your network. However, as the exact way such metrics are calculated is not explicit, they are of little value in any meaningful conversation


likely to be significantly different from those of the general public, often having more interest in an impact and information flows within a sub-network rather than out to the whole community. In comparison to other social network sites such as Facebook, the relative openness of Twitter offers the potential for wider network-based indicators. Rather than focusing on the number of followers, or even ratios between the number of followers and the number of streams a user is following, it is possible to take into consideration the position of a user within the wider network. Such metrics are not simple, or obvious, and


can only be established through discussion in the open rather than as proprietary metrics behind closed doors. Even then we cannot expect the establishment of a single metric to represent a researcher’s usage of social media. Rather, what is required is a barrage of metrics, each of which measures different aspects of potential online usage that can then be the start of a conversation about a researcher’s impact. The creation of such metrics obviously presents a challenge, especially as we move from looking at a single open social-media site like Twitter, and take into consideration the wide range of content on sites that are far less open. Nonetheless it is a challenge that the research community needs to face if they are to successfully take advantage of the new technologies and successfully demonstrate the transfer of knowledge and the impact of research on the wider community.


David Stuart is a research associate at the Centre for e-Research, King’s College London, and recent author of ‘Facilitating access to the web of data: a guide for librarians’


OCT/NOV 2011 Research Information 19


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36