Saturday, May 18, 2013

The Role of Blogging in the Academic Feedback Cycle

Feedback Diversity is Good
Last year I delivered a couple of research papers on the history of crime. The first was in October at the Institute of Historical Research or the IHR as it’s known, here in London. The second was in January, on a beach in Belize. I thought I'd talk a little bit today about how those two experiences were different, how they were the same, and what place I think each holds in the future of scholarship.

Now before you start looking for tropical conferences on 18th century crime, I should qualify that the first paper was delivered to a room full of people. The second was posted on my blog while I was on vacation – and yes, sadly, I DID write about 18th century crime while gazing out over the Caribbean Sea. For some people speaking to a room and blogging are probably significantly different activities. But for me they aren’t all that dissimilar. Let me explain why by talking about what I got out of both experiences as well as what went into them.

At the IHR, I presented an hour-long paper based on three chapters from my PhD thesis. It was about two years worth of work that I had condensed down and tried to make engaging for a room full of people. For about two months before I gave the talk I didn’t do much other than scramble to get the research done, create the graphs build the powerpoint presentation, and craft the 8,000 words that I was to deliver. It was an incredible amount of work. I wore a jacket and tie, and I think I might have even gotten a haircut. Good thing because some of the most eminent crime historians in the world happened to be in town and decided to come to my talk. In all, I think there were about 50 historians in the room, most of whom knew far more about crime and the eighteenth century than I do.

The talk was followed by a really engaging discussion – at least from my perspective. I had a number of people offer suggestions for improving my argument, or on sources and archives I should visit. A couple of scholars who also write on similar topics challenged my findings – though were collegial and offered their own suggestions. Afterwards we continued onto the pub and to dinner as a group and over the course of the evening I must have heard ideas, criticisms, and praise from about 25 individuals on what I was doing.

The beach was a very different experience. The paper itself was just shy of 3,000 words, so somewhere in the 20-25 minute range if I had delivered it orally. This time my paper was based on some quick research I’d done just before Christmas. In total I’d invested a little more than a week analyzing the use of language in the Old Bailey Proceedings over a two hundred year period. It was really nothing more than an idea I'd wanted to test out, based on a conversation I'd had at the pub concerning the size of the lexicon over time.

The results I came up with were what you might call half-baked. Not that I’d been lazy, or that I didn’t know what I was talking about, or that the results were wrong. Just that I hadn’t spent weeks or months revising my methodology and my prose as I had at the IHR. Nor did do an in depth literature review. Instead it was more an activity in play. I had some sources, I had an idea, I tried it out, I wrote it up – with a reasonable amount of care – and I posted it to the world, curious to see what it thought.

The world doesn’t scare me, although many postgraduate conferences suggest it should. It’s not uncommon for these postgraduate affairs to advertise the fact that they are collegial, and a safe place to try out ideas. No senior academics are going to be on hand to put you in your place and tell you how wrong you are. I’ve never been one for intellectual safety, so I don’t see putting a half-baked idea before the world as one of risk. Rather it’s one of potential. But it’s also one of uncertainty and often loneliness.

When I posted my paper on the blog, there was no beer and pizza afterwards – though I did have a nice swim. And in the end I got one comment on the post from Ben Schmidt at Harvard who offered a suggestion for improving the methodology and the results.

On the surface it looks like the blog post was significantly less successful, since the number of comments I got were 25 at the physical presentation, and only one on the blog. But I don’t think that’s quite fair, for a couple of reasons.

Firstly, the talk at the IHR was a formal affair presenting years of research, with a moderator that gazes around the room encouraging more questions from the audience. The blog post was a way to test an idea, which is shouted into the great wilderness. That level of anonymity readers of blogs enjoy means there isn’t the same pressure to respond. But just because they don’t respond doesn’t mean they didn’t engage with the content. It’s difficult to know how many people engage with content on the Internet. I know 50 people were in the room for my seminar paper at the IHR, and I didn’t notice anyone sleeping, but even then I can’t be sure who disappeared into the recesses of their mind as I talked away.

My blog however offers statistics, and though I know not everyone who visits a blog post reads it, I do know about 600 people came to take a look. That’s about 12 times more than showed up to hear my seminar, and because a blog post is printed on the Internet rather than delivered orally, vanishing on the wind as it’s spoken, my blog readers could be anywhere in the world, and could even have been sleeping when I delivered it.

But what I think is important is not how many people read the blog post. Rather, it’s the diversity of the people who did so. The seminar at the IHR was attended almost exclusively by specialists in 18th century British history. The blog reaches a much more diverse audience who typically come through one of two channels:

• Twitter
• Digital Humanities Now.

When I post a new blog post I then post a notice on Twitter letting my followers know. If I’m lucky a few people will notice and share it with their followers on Twitter, or will write a response on their own blogs. And if I’m really lucky a group of scholars in Virginia who run a blog called Digital Humanities Now, which post the best blog posts of the day related to digital humanities, will tell their audience about my post, sending even more people. That’s basically what happened in the case of my Belizean blog post. I published it to the blog, told Twitter, was re-tweeted by a few people, and was showcased on the Digital Humanities Now blog.

That meant my audience included a large number of digital humanists who work in a wide range of academic disciplines including linguistics, computational analysis, literary studies, and history. I think it’s fair to say most of that audience doesn’t care about 18th century British history. However, they do share an interest in the methodology I used to work with the sources. One of those digital humanists, Ben Schmidt, posted the comment that helped me refine my methodology and come up with even stronger results.

No one in the IHR seminar was going to give me that type of feedback because that’s not the type of expertise they have. Instead they focused on the details related to the history of crime or on the records they knew of in the archives. So by seeking out a different audience through the blog, I was able to get interdisciplinary feedback on my work.

History seminars are extraordinarily valuable, particularly for early career scholars like myself. The level of intimacy you get in that type of environment is unparalleled. But they’re a bit like poorly designed focus groups. If you want to take the pulse of the nation on welfare reform or Euroskepticism, you don’t want a room full of Horse and Hound subscribers. You need the diversity of a few Daily Mail readers thrown in the mix, who see the world from a slightly different angle.

And I think the blog and twitter provide that diversity for me. In my case, my blog attracts a lot of digital humanists, but blogs aren’t just a way to get feedback from digital humanists. I posted another blog post a few weeks later on the same research material, this time focused on using criminal records to measure immigration. I again got one comment, but this time it was from Tim Hitchcock, a historian of 18th century Britain, who offered a historical interpretation that might explain what I had found. Tim’s expertise with the provenance of the records meant he knew things about the sources I didn’t.

I posted a third blog post again on a slightly different topic, and received different types of comments again from linguists, computer scientists, and Sharon Howard, the project manager from the Old Bailey Online project. With three blog posts and roughly the same number of words as my seminar paper, I’d engaged a number of different types of people from all over the world with very different sets of expertise, and different types of feedback than I could ever expect to get from a room full of crime historians.

Which experience was more valuable? The seminar or the blog posts? For me, I don’t think there’s much that can compare with a room full of world experts devoting their combined experience to listening and critiquing years of your hard work. I also don’t think you can beat the type of connections that can only be made in a face-to-face meeting at the pub, or over pizza with people who share your interests. But I also don’t think we should sniff at a model that allowed me to test 3 ideas in an informal setting, get a broad range of feedback from interdisciplinary experts all over the world, and all without costing anyone a penny.

I’ve taken on board all of the feedback I’ve received from these two papers. My PhD thesis is stronger for having delivered the seminar paper, and I’ve decided to pursue the ideas expressed in my blog more formally as a future research project. So these papers were both valuable in their own right, and I think I’m a better historian for having delivered them.

This is the text of my talk at 'Our Criminal Past: Digitisation, Social Media, and Crime History' held at the London Metropolitan Archives, 17 May 2013. With thanks to Heather Shore for inviting me to speak.

4 comments:

James Baker said...

With apologies for the slightly off topic comment, but 8,000 words for an IHR paper. You must talk fast.
On which (so slightly closer to the topic) you'd never get away with a 8,000 word blog post. So perhaps a value of bums on seats is the desire to listen and contribute 'stated' by their having turned up and put their bums on the seats?

Adam Crymble said...

James, you're right. I don't think you'd get too many readers on an 8,000 word blog post, unless you prefaced it with a claim that it was a full academic paper.
I'm sure you're right about the audience too. Making a deliberate decision to come listen is significantly different than making a deliberate decision to click on a link in one's Twitter stream.

Andrew Prescott said...

Fascinating. I am very intrigued by the way in which some blog posts are generating many comments, and the comments in themselves become substantial pieces of scholarly writing. A good example is the recent open thread on Postcolonial Digital Humanities discussing how far DH has served as a way of escaping from consideration of issues of ethnicity, gender, disability: http://bit.ly/11cYqru. The initial post is very short, but the ensuing discussion is very rich, multifaceted and involves some of the leading people in the field. In this case, the discussion is a rich as an an IHR seminar. What intrigues me is how such discussion gets sparked off - it doesn't seem to be the blog entry itself, but rather the number of followers of the blog which provoke the discussion.

Adam Crymble said...

You raise a good point about the value of networks, and that's something that came up when I delivered this talk as well. How do you get people to comment? The person asking the question was concerned that she knew of 'good' scholarship posted on blogs that was being ignored.
I sometimes take it for granted that at least some people will read things I post on my blog. But they didn't always. It took years of hard work, and I've never experienced anything on the scale of the 159 comments you point to in that other post. But part of speaking from a soapbox is taking the time to make sure people know where it is, and that what you're going to say from atop it is worthwhile. We tell our colleagues to build soapboxes, but hardly ever tell them how to do it well.