Cultures of counting: Metrics through a critical lens

Tuesday 24th May sees the Academics’ Writing project’s fourth and final workshop on the role of metrics in academic life. This time, we have invited two expert speakers to talk about what responsible metrics might look like in the context of both REF and TEF. The speakers are Professor Paul Ashwin of Lancaster University and Professor James Wilsdon of the University of Sheffield.

Metrics evoke a mixed reaction across the higher education community. A commitment to using data to inform decisions makes some enthusiastic about the prospect of granular, real-time analysis of our activities. Yet we only have to look at the blunt use of metrics such as
journal impact factors, h-indices and grant income targets, to be reminded 
of the pitfalls. Some of the most precious qualities of academic culture
 resist simple quantification, and individual indicators often struggle to do justice to the richness and plurality of our work.

Across both research and teaching, metrics are receiving greater emphasis from policymakers and managers. The November 2015 HE green paper outlines a new regulatory architecture, including the replacement of HEFCE with a new Office for Students, and the introduction of a Teaching Excellence Framework (TEF). Metrics are portrayed as crucial to the TEF, albeit with some scope for expert judgement alongside, and there are now fierce arguments raging across the sector about whether we need a TEF at all, and if so, how it should be designed, and what mix of quantitative indicators it should employ.

Metrics hold real power: they are constitutive of values, identities and livelihoods. How to exercise that power to more positive ends was the focus of The Metric Tide, a recent UK review of the role of metrics in research management and assessment. The Metric Tide sets out a framework for responsible metrics, and makes a series of recommendations for researchers, university managers, funders, policymakers and publishers.

In this seminar, James Wilsdon, who chaired The Metric Tide, will outline its main findings, and reflect on ongoing efforts to influence debates about UK research policy and funding, including over the design of the next Research Excellence Framework (REF), which is currently the focus of a further review by Lord Stern.

Paul Ashwin, in the context of the proposed TEF, will examine the challenges of developing measures of teaching quality that do not simply reflect institutional prestige.

Both speakers will consider what a culture of ‘responsible metrics’ might look like for research and teaching, and the opportunities and obstacles to achieving this.

James Wilsdon is professor of research policy at the University of Sheffield, and was chair of the UK’s Independent Review of the Role of Metrics in Research Assessment and Management. He is now chairing a European Commission expert group on the future of altmetrics.

Paul Ashwin is professor of higher education at the Lancaster University and a co-investigator in the ESRC/HEFCE funded Centre for Global Higher Education.

This talk is part of the Designing the Academic Self series, sponsored by the Academics Writing project, which is part of the Literacy research Centre at Lancaster and the Northwest Doctoral Training Center and is open to early career researchers and doctoral students from Lancaster, Manchester and Liverpool universities.  To book your place, visit Eventbrite. 

 

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

Academics’ experiences of professional networked learning

The project team have been presenting our findings on academics’ experiences of professional networked learning at the 2016 Networked Learning Conference. The slides can be found here: NLC10may2016_slides

It was a thought-provoking conference overall, with many sessions, including Helen Beetham‘s and Magda Bober‘s talk, echoing some of the themes that have emerged from our research. Helen talked about the disaggregation of professional roles in academia, including the blurring of boundaries between personal and professional time and identities. This is a major concern for our own participants – almost without exception, people talked about the pressure / drive to check email using portable digital devices, even when when not ‘at work’, and about concerns around the ways their identity could be constructed online. Magda talked about the ways in which students and staff in HE use mobile devices, and the symbolic meanings these held for these two groups.

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

What can and can’t metrics tell us?

Tuesday 26th April sees the third of four interactive workshops on the role of metrics in academic life, run by the Academics Writing  project, alongside  Masud Khokhar and Tanya Williamson of Lancaster University library, called What can and can’t metrics tell us?

Higher education institutions in the UK have to justify how they spend their research funding and demonstrate that this spending resulted in high quality research outcomes. This is achieved mainly through the REF (research excellence framework) and bibliometric measures associated with it. The research activity of individual academics, departments and institutions is measured in various ways and the resulting data used for a range of purposes. But are we asking the right questions? And are we using the data in an appropriate way?

This interactive workshop is part of the Designing the Academic Self series, and explores the ways in which quantitative indicators of research output are used, by both institutions and individuals. We begin by exploring what exactly are we trying to measure, and consider how well are the metrics we generate can do this. We will talk about case studies of how metrics are currently being used in different UK institutions, and will consider the potential positive and negative effects of these on academics.

To book a place, visit Eventbrite 

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

To blog or not to blog, that is the question

Our PI Karin participated in a public lecture in Lancaster last week on the theme of Teaching, Tweeting, and Trolling – Our Online Worlds.  In addition, I had the pleasure of going to Sweden to give talk at Stockholm University’s department of English on behalf of the Academics Writing project. In both of these talks we shared findings on how academics’ writing practices have been affected by technological changes.

We asked our participants if they did any writing on digital platforms such as Twitter, blogs, and Facebook, and found a bit of a trend across the disciplines (bearing in mind that our sample is too small to draw generalisations). Not many of the mathematicians used these platforms, while historians tended to speak rather more positively about them, even if they didn’t actually use them much.

The reasons our participants have given for not engaging with these new genres of writing include a perception that they were trivial or inconsistent with their professional identity. For example, one participant (a statistician) said, “I don’t necessarily approve of Twitter and Facebook so I tend to avoid them”. But these feelings of disapproval were not shared by everyone, and others were keen to use these platforms.

One History professor said, “I lay awake sometime last week thinking maybe I should have a blog. Haven’t got around to it. But the blogs I do read, some of them are terrific.” Another historian expressed similarly positive views: “I am really interested in the idea of blogs and sometime, maybe when I retire, I might get into blogs. I think they are really fun. I don’t do Twitter either. All those modes of communication seem quite interesting.”

The language these historians use speaks of the potential pleasures and creativity of these forms of writing, yet their take-up is constrained by the need to produce other, more privileged genres. This was particularly clear in David’s comment, “A lot of the work is grey literature where people have written blog pieces. I think that’s opened my eyes to what’s possible in that area but yes, if there’s time – I think it’s always a question of time. Again, that work is not valued by the university as far as I can see.” Although he saw potential in these forms of writing, particularly in terms of communicating to audiences beyond the academy, he acknowledged that peer-reviewed, scholarly publications take priority, partly driven by institutional demands to produce REF-able research outputs.

Where do you stand on these hybrid genres? Are they appropriate for academics? Should institutions value them more or would this simply add to already heavy workloads?

 

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

Dissemination events

As we reach the end of the first phase of the Academics Writing project, we are holding a series of dissemination meetings to share interim findings from the research and to invite responses and comments from those whose working lives the research may impact upon.

Because we have been asking our participants not only about scholarly writing, but also about teaching- and admin-related writing, there are interesting data about how academics decide what to prioritise among their workloads, the range of digital platforms and devices they use for writing, and how they create a space for themselves, both physically and psychologically, to get to the writing that really matters to them.

All of which has implications for how we prepare new researchers for their role, and for how their working hours and spaces are organised. Thus, the dissemination meetings are targeted not only at academics, but also at those involved in the professional development or training of early career researchers and doctoral students; those working in research management and support; those responsible for academics’ work spaces, and anyone with an interest in academic writing.

The meetings are on the 18th of March and the 19th and 22nd of April at universities around the North of England. If you’d like to attend, get in touch with Sharon on s.mcculloch@lancaster.ac.uk for more information. and we’ll send you details.

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

How are metrics affecting academics’ writing practices?

Academics are increasingly subject to a range of ‘metrics’, attempts to produce a quantitative measure of the reach and impact of research (and, increasingly, teaching). Research metrics include measures at the level of the individual, such as the h-index (a measure of an individuals’ research output and the extent to which it is cited), at the level of individual papers such as numbers of downloads or numbers of citations, and at the level of journals such as the ISI ‘journal impact factor’, worked out on the basis of citation rates across a journal’s outputs. Newer metrics – ‘altmetrics’ – try, in addition, to capture the wider range of ways in which research publications can have an impact in the virtual world, producing measures based on numbers of views, downloads, saves, shares and recommendations online. All these metrics are becoming increasingly important in recruitment to academic positions, promotion, and of course assessment of academic departments via research assessment exercises such as the REF.

The existence and influence of such metrics has a range of effects on academics’ writing practices. They impact, for instance, on the selection of journals to write for, genres to write in and topics to focus on. Academics are increasingly expected to engage in self-promotional virtual practices including maintaining professional webpages and academic social networking, and these activities can have a direct effect on the newer altmetric quantitative indicators.

The second of four interactive workshops takes place this week, exploring the ways in which quantitative indicators informed by digital technologies are influencing academics’ writing practices.We will report on preliminary findings from the Academics’ Writing project and attendees will have the opportunity to discuss their own experiences of using and being measured via metrics.

Workshop title: Designing the academic self: How are metrics affecting academics’ writing practices at UK universities?

Date: Tuesday 16 Feb 2016, 1.00 – 3.00 pm

Venue: Lancaster University, Charles Carter A15

This series of workshops is run with support from the Northwest Doctoral Training Centre, and is free and open to doctoral students, staff and researchers from Lancaster, Liverpool, and Manchester Universities. Please register your attendance via Eventbrite. For dates and themes of the whole series of workshops, click here.

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

The long word club

One of my pet peeves is conference speakers who sit down and put on their reading glasses. This is a sure sign that they’re about to eschew slides and read their paper aloud. Moreover, the paper will be crowded with abstract concepts described entirely through words of 8 syllables or more.

It may be called “a conference paper“, but this does not make it acceptable to inflict 30 minutes of unsupported listening on a tired audience. The projector is there for a reason.

I’d been moaning about this when my colleague Mary sent me a link to a blog post by Mark Carrigan,  in which he discusses  sociologists’ habit of writing in unsociably dense, turgid prose.  Carrigan quotes Les Back in the Art of Listening,  comparing academics to “bookish limpets” (2007, p. 163).  So if we recognise our own weaknesses in this regard, what should we do about them?

James Mulholland  has argued that rather than attempting to make complex research more accessible to general audiences, we should simply embrace esoteric knowledge and technical language as intrinsic aspects of dealing with complex ideas. Stay in your ivory towers, he urges, and write books that few people will read.

Carrigan suggests that blogging and tweeting offer possibilities for making academics’ writing more engaging and opening it up to a wider readership. While I’m not completely convinced that that academics who blog are not already preaching to the converted, I do think that Mulholland is missing the point somewhat. The debate is not about whether we should change the books we write, but how we might persuade more people to read those we do write.

The title of this post was inspired by / stolen from:

Gardener, S. (1992). The long word club: The development of written language within adult fresh start and return to learning programmes. Brighton: RaPAL.

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

SRHE conference paper on disciplines in Higher Education

Last month (December 2015) I delivered a couple of papers at the annual conference of the Society for Research into Higher Education. One of them was on this project’s findings so far. The Powerpoint is below:

One of the things I discussed with my audience was the strange things happening to disciplines in Higher Education, and the new disciplinary identities emerging. Since identities permeate academics’ writing practices for research, teaching, and even admin work, the paper generated a lot of interest and discussion. Some of these were also tweeted about:

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

Designing the Academic Self

Social media, metrics, preprints, websites – much of our academic identity is now tied up in how we mediate ourselves online. So what should we be thinking about and what should we do?

The Academics Writing Project, in conjunction with Lancaster’s Literacy Research Centre and the North West Doctoral Training Centre  is running a series of interactive workshops for PhD students and early career researchers on the use of social media and metrics, called Designing the Academic Self. The workshops are open to doctoral students, staff and researchers from Lancaster and Manchester Universities.

Friday 29 January 2016, 13.00-15.00

Bowland North

SR10

Session 1: Who does the Internet think you are?

Sharon McCulloch, Diane Potts, and Tanya Williamson, Lancaster University

Rescheduled from Dec 2015 – please note that this session is on a Friday.

You must register to ensure a place.

Tuesday 16 Feb 2016, 13.00-15.00

Charles Carter

A15

Session 2: How are metrics affecting academics’ writing practices in UK universities?

Sharon McCulloch and Karin Tusting, Lancaster University

You must register to ensure a place.

Tuesday 26 April 2016, 13.00-15.00

Charles Carter

A15

Session 3: What can and can’t metrics tell us?

Masud Khokhar and Sharon McCulloch, Lancaster University

Tuesday 24 May 2016, 13.00-15.00

Charles Carter

A15

Session 4: Metrics through a critical lens

TBC

The first workshop, Who does the Internet think you are? is a discussion of if and how emerging academics can be pro-active in creating or ‘designing’ a coherent online presence, including practical matters such as selecting your online name and keywords, as well as the pros and cons of commonly used sites for showcasing academic work. Click here to register for the first workshop.

By the end of the four Designing the academic self sessions, attendees will have a deeper understanding of the underlying mechanisms of metrics and how such tools can extend the audience for one’s work; of evolving academic writing practices in the face of new modes of dissemination and measurement; and of the critical questions such changes are provoking.

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest

Affect and academics’ writing

As part of the ‘autoethnography’ component of our research project we have been documenting the team’s views associated with the affective and emotional aspects of their professional writing. We decided to explore this particular aspect of academic professional life, as the affective experiences of academic working life (its joys and pains) are closely bound up with successful and unsuccessful experiences of writing (acceptances of papers, completion of monographs, proposals, feedback, etc.).
Here are some of the responses to the questions we asked in the autoethnographic survey:
1) Tell me about a piece of writing you did which brought positive feelings or joy?

I am quite a synthetic thinker and I enjoy the creativity involved in researching ideas and putting them together in an argument in a new way, or developing a storyline from disparate data and viewpoints.

If I am satisfied with a piece of writing (and this might just be an email that I feel I have successfully crafted), I sometimes go back and read it over several times, enjoying the achievement of it.

I don’t feel the same level of satisfaction when I prepare and deliver an oral presentation which is too unpredictable. I feel more in control of my ideas and self-presentation when I write.

I get satisfaction and pleasure from writing, but wouldn’t say it has ever made me “really happy”. To me, writing is mainly an intellectual endeavour rather than an emotional one.

My writing process goes something like this: initial enthusiasm, leading to a first burst of work on something; getting stuck, accompanied by a conviction that this time, it will all be impossible; relief when something clicks and I see how it will be possible (this is usually when I get from ‘idea’ to ‘an actual argument’).

2) What specifically was it that caused you to be happy?

In terms of process, the satisfaction came from seeing the book as a project come together from the original seminar, through negotiating publication with a good publisher, working with contributors and co-editors, designing the book cover etc. In terms of the topic, helping a new field to come into focus.

With pen and pencil writing, I don’t usually enjoy the act as it tends to be messy and untidy.

Content gives me anger, regret and sadness, but not the act of writing itself.

Online I like writing when I do it unnoticed. I hate being rushed, and I try never to circulate drafts if I am not happy with them.

I was satisfied to recently get a chapter from my PhD published – because it had always nagged at me that I hadn’t got journal articles from it and should have done.

3) Tell me about a piece of writing that you did, the memory of which you associate with negative feelings

A piece of professional criticism which I was commissioned to write. Though I stand by the content of the piece, I don’t think I fully understood at the time how my writing was being used politically by the commissioning body and the whole thing felt very uncomfortable – but perhaps uncompromising criticism is always difficult and confrontational – not my normal style.

I wrote an article for a public media outlet. I did it with some edits from their editor who kind of ‘sexed it up’ a bit. There was a really positive response on social media about the issues I raised, with very few negative comments. Some of the negative comments however got to me. It hasn’t changed the way I write. It’s just toughened me up a bit.

I tried to write a book based on my previous big research project, but didn’t, despite spending a lot of time on bits of it – I think mostly because of other life and work stuff happening which got in the way.  I was disappointed in myself – I see myself as someone who gets things done but there were a couple of years when I didn’t.  I also feel that although I didn’t have an actual contract, I let down the people who had said they might publish it.

4) What irritates you most about the professional writing that you do (teaching, admin or research)?

I hate doing boring routine stuff. I think this is why assessing repeated exam answers, for example, is such an effort for me.

I found it hard being required to annotate student scripts online with feedback comments but this was really about a technology that wasn’t fit for purpose. I tend to mark when travelling or at other times when I’m not near a large screen computer and mobile technologies are fiddly for this kind of writing.

I loathe filling in templates for things like annual reviews or references etc. Wherever I can, I take the headings as guidelines for writing in a narrative way and avoid filling the boxes.

Writing abstracts is demanding. You have to know what you want to say, or have to discover it by the act of writing. I was doing one this morning for a plenary in a few months, and I don’t actually know what I want to say. I tried staring at an empty, fresh, new Word document and that didn’t work. I moved away from my computer and tried with a pen and paper. That gave a set of bullet points, but not a well-argued abstract. So a lot of wasted time – it wasn’t the right time for writing something new.

I honestly don’t think there is anything that irritates me about the writing I do.

I don’t know if anything irritates me. Technology can be annoying sometimes I suppose.

5) What do you like most about your writing?

The feeling of “flow” of being absorbed in the moment of writing so that you don’t keep track of time.

The internet has also brought a new dimension to my writing as I frequently search online while I am writing to find references, citations, perhaps read a related article, watch a video or follow a trail that informs me about a concept or aspect of my topic more deeply.

The recognition from peers if it gets it. But the best is when my writing reaches people I never expected it to reach and they value it.

What I like most about my writing is when it has an effect. For example if I review an article for a journal and my writing is useful and has an effect.

I like the process of doing it, and it’s nice for my ego when people cite the resulting text. I don’t pay that much attention now, but the first time I discovered that someone famous had cited my work, I could scarcely believe it.

I like the feeling of thoughts clunking together. I like the neatness of a well-structured piece of writing.  I like finding just the right word to make a sentence work.

6) What do you think about this quote from the writer Dorothy Parker? “I hate writing, I love having written”Do you feel the same? Try to explain your answer.

Yes, I agree. Those horrible hours chained to a computer and not going out for fun, contrasted with the satisfaction of seeing something I’ve written appearing in print. But the opposite can be true too. Enjoyable mornings can pass working and making progress on an article or a chapter.

I understand what Parker means. Writers of all types often say that the process is torturous. There must therefore be all the more satisfaction in having done it, and being rewarded for all that hard work.  I love having written, and in that sense I agree with Parker, but I also like writing. It seems rather indulgent not to.

I know what she means.  I often hate the idea of writing, but when I’m not writing I feel edgy.  When I’m writing regularly I feel much saner.  Though I’m not sure I love having written either – I am often reluctant to look at things I have published.

We are finding that affect and its associations with academics’ writing is emerging quite a bit in our research data, chiming with other research that focuses on the social aspects of writing and the importance of motivation and supportive culture in which academic work can emerge (e.g. Cloutier 2015; Murray 2015).

Do the responses above correlate with your experience? If so, how? And if not, how would you respond to the questions we posed?

References:

Cloutier, C. (2015) How I Write: An Inquiry Into the Writing Practices of Academics, Journal of Management Inquiry, DOI:

Murray, R. (2015) Writing in Social Spaces: A social processes approach to academic writing, Abingdon: Routledge.

Email this to someoneTweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on Pinterest