Writing that counts or writing that is counted?

At the beginning of December, I went to Leiden in the Netherlands to present a paper on behalf of the Academics Writing project. The conference was the Accelerated Academy, which brings together scholars from across Europe, mainly from STS (Science and Technology Studies) but also from other disciplines, to share research about the acceleration of higher education and the increased use of metrics for research evaluation. Clips of some sessions from the Prague conference in 2015 can be found here.

Our paper focused on the tensions that have emerged from the project around the REF. The UK’s national research evaluation exercise pushes academics in competing directions in their writing. The REF and institutional policies around it drive academics to write more and better journal articles in each REF period, and this has become an overarching goal for academics’ writing practices: to publish enough and to target the right journals is to keep their job, to get promoted and to be considered successful. But not all academics really want to publish this sort of journal article, or at least not at the expense of other genres. Historians value monographs, but these take a long time to write and because four research outputs can be submitted to the REF, four has become something of a magic number. Four monographs in a single REF period would be more or less impossible, but departments expect at least four publications, so historians are pushed towards writing journal articles. For Marketing academics, four translates as four-star; their ultimate target for publishing. They are encouraged to publish in four-star journals, which they see as a.) unrealistically difficult and b.) pushing them out of their discipline. If it’s too hard to get into a four-star marketing journal, one can always aim for a four-star management journal.

Finally, although since 2014 the REF rewards writing for impact beyond academia, writing for non-academic audiences was widely perceived by our respondents to be of secondary importance to writing for peer-reviewed, high-impact journals. Many were interested in writing for the media, writing blog posts, tweeting and the like, but didn’t feel that their institutions valued these genres. This view is unsurprising given the relentless counting and evaluating by universities of writing that is aimed primarily at other academics.

The project goes to summer school

I gave a lecture last week at an international doctoral summer school in Vienna, run by Klagenfurt University. The theme of the summer school was quality in scholarly practice in social sciences and humanities, and my lecture discussed findings from the Academics’ Writing project on how academics’ writing practices are influenced by measures of research quality such as the REF.

I shared extracts from interviews the project team has done with academics who talked about the pressures of having to publish certain numbers of papers in specified venues, often based on journal impact factors, and about feeling forced to publish in journals outside their discipline.

I also ran a workshop on getting published, and Klagenfurt University were generous enough to let me stick around for the whole week, listen to the other invited speakers and get to know the students, who came from Austria, Germany, France, Turkey, Italy, Bulgaria, Lithuania, Uganda, Nigeria, Brazil, Costa Rica and the USA. They were a passionate, articulate and multilingual group and I scribbled down many of their nuggets of wisdom, which sometimes bordered on the poetic. Here’s a selection:

“I write at night, like a witch”

“I feel oppressed by writing in English”

“You need to read texts against the grain”

“There are ghosts in the archive”

“The impact agenda makes bad autoethnographers of us”

I could pick any one of these and write about it all day, but I’ll leave you to ponder your own interpretation of these statements and how they might apply to you.

Cultures of counting: Metrics through a critical lens

Tuesday 24th May sees the Academics’ Writing project’s fourth and final workshop on the role of metrics in academic life. This time, we have invited two expert speakers to talk about what responsible metrics might look like in the context of both REF and TEF. The speakers are Professor Paul Ashwin of Lancaster University and Professor James Wilsdon of the University of Sheffield.

Metrics evoke a mixed reaction across the higher education community. A commitment to using data to inform decisions makes some enthusiastic about the prospect of granular, real-time analysis of our activities. Yet we only have to look at the blunt use of metrics such as
journal impact factors, h-indices and grant income targets, to be reminded 
of the pitfalls. Some of the most precious qualities of academic culture
 resist simple quantification, and individual indicators often struggle to do justice to the richness and plurality of our work.

Across both research and teaching, metrics are receiving greater emphasis from policymakers and managers. The November 2015 HE green paper outlines a new regulatory architecture, including the replacement of HEFCE with a new Office for Students, and the introduction of a Teaching Excellence Framework (TEF). Metrics are portrayed as crucial to the TEF, albeit with some scope for expert judgement alongside, and there are now fierce arguments raging across the sector about whether we need a TEF at all, and if so, how it should be designed, and what mix of quantitative indicators it should employ.

Metrics hold real power: they are constitutive of values, identities and livelihoods. How to exercise that power to more positive ends was the focus of The Metric Tide, a recent UK review of the role of metrics in research management and assessment. The Metric Tide sets out a framework for responsible metrics, and makes a series of recommendations for researchers, university managers, funders, policymakers and publishers.

In this seminar, James Wilsdon, who chaired The Metric Tide, will outline its main findings, and reflect on ongoing efforts to influence debates about UK research policy and funding, including over the design of the next Research Excellence Framework (REF), which is currently the focus of a further review by Lord Stern.

Paul Ashwin, in the context of the proposed TEF, will examine the challenges of developing measures of teaching quality that do not simply reflect institutional prestige.

Both speakers will consider what a culture of ‘responsible metrics’ might look like for research and teaching, and the opportunities and obstacles to achieving this.

James Wilsdon is professor of research policy at the University of Sheffield, and was chair of the UK’s Independent Review of the Role of Metrics in Research Assessment and Management. He is now chairing a European Commission expert group on the future of altmetrics.

Paul Ashwin is professor of higher education at the Lancaster University and a co-investigator in the ESRC/HEFCE funded Centre for Global Higher Education.

This talk is part of the Designing the Academic Self series, sponsored by the Academics Writing project, which is part of the Literacy research Centre at Lancaster and the Northwest Doctoral Training Center and is open to early career researchers and doctoral students from Lancaster, Manchester and Liverpool universities.  To book your place, visit Eventbrite. 

 

What can and can’t metrics tell us?

Tuesday 26th April sees the third of four interactive workshops on the role of metrics in academic life, run by the Academics Writing  project, alongside  Masud Khokhar and Tanya Williamson of Lancaster University library, called What can and can’t metrics tell us?

Higher education institutions in the UK have to justify how they spend their research funding and demonstrate that this spending resulted in high quality research outcomes. This is achieved mainly through the REF (research excellence framework) and bibliometric measures associated with it. The research activity of individual academics, departments and institutions is measured in various ways and the resulting data used for a range of purposes. But are we asking the right questions? And are we using the data in an appropriate way?

This interactive workshop is part of the Designing the Academic Self series, and explores the ways in which quantitative indicators of research output are used, by both institutions and individuals. We begin by exploring what exactly are we trying to measure, and consider how well are the metrics we generate can do this. We will talk about case studies of how metrics are currently being used in different UK institutions, and will consider the potential positive and negative effects of these on academics.

To book a place, visit Eventbrite 

How are metrics affecting academics’ writing practices?

Academics are increasingly subject to a range of ‘metrics’, attempts to produce a quantitative measure of the reach and impact of research (and, increasingly, teaching). Research metrics include measures at the level of the individual, such as the h-index (a measure of an individuals’ research output and the extent to which it is cited), at the level of individual papers such as numbers of downloads or numbers of citations, and at the level of journals such as the ISI ‘journal impact factor’, worked out on the basis of citation rates across a journal’s outputs. Newer metrics – ‘altmetrics’ – try, in addition, to capture the wider range of ways in which research publications can have an impact in the virtual world, producing measures based on numbers of views, downloads, saves, shares and recommendations online. All these metrics are becoming increasingly important in recruitment to academic positions, promotion, and of course assessment of academic departments via research assessment exercises such as the REF.

The existence and influence of such metrics has a range of effects on academics’ writing practices. They impact, for instance, on the selection of journals to write for, genres to write in and topics to focus on. Academics are increasingly expected to engage in self-promotional virtual practices including maintaining professional webpages and academic social networking, and these activities can have a direct effect on the newer altmetric quantitative indicators.

The second of four interactive workshops takes place this week, exploring the ways in which quantitative indicators informed by digital technologies are influencing academics’ writing practices.We will report on preliminary findings from the Academics’ Writing project and attendees will have the opportunity to discuss their own experiences of using and being measured via metrics.

Workshop title: Designing the academic self: How are metrics affecting academics’ writing practices at UK universities?

Date: Tuesday 16 Feb 2016, 1.00 – 3.00 pm

Venue: Lancaster University, Charles Carter A15

This series of workshops is run with support from the Northwest Doctoral Training Centre, and is free and open to doctoral students, staff and researchers from Lancaster, Liverpool, and Manchester Universities. Please register your attendance via Eventbrite. For dates and themes of the whole series of workshops, click here.