The numbers are people

Educational research has two cultures, but unlike the two in C.P. Snow’s famous talk, they do overlap and they do talk to each other. One is fiercely qualitative, concerned directly and immediately with the lived human experience of learners and teachers, in all its ethnographic complexity, subtlety, and sophistication. The other is determinedly quantitative, concerned with what can be counted and known in more regular, repeatable, transferable ways.

My sympathies lie with both, but my inclination is definitely towards the empirical, quantitative, and generalisable. But that mustn’t be at the cost of losing sight of the human perspective. With the rise of learning analytics, and more and more quantification of learning – of which I’m a small part – it’s easy to be sucked in to watching the numbers and forgetting the people.

Two things have come to haunt me recently. Both are about predictive analytics: using all the data we have about learners, and previous learner’s success or failure, and using that to predict the success (or otherwise) of learners who haven’t finished yet – or even signed up. The OU is doing a lot of work in this area at the moment. The ethical issues are complex and difficult. I think both the quantitative and the qualitative perspectives are needed to guide our policy development here.

cc licensed ( BY ) flickr photo shared by Photo Mery

Continue reading “The numbers are people”

Password change for the better

Nine months ago, I posted my rather involved process for mandatory work password change day.

Today, it was password change day again, and the whole thing was done in 20 minutes, including 10 minutes spent on a completely different task. That’s the third time in a row that the simpler procedure (turn everything off, change the password, bring things back up one at a time and enter new passwords as required) has worked without a hitch.

So I’m going to mark this as resolved, and give a public thankyou to whichever nameless person in our IT department made the change that fixed whatever issue made the complex workaround malarkey necessary. Heaven knows “nameless person in our IT department who made a change I don’t fully understand” gets a lot of stick, so I’d like to give them a thumbs-up for once.

While I’m thanking IT infrastructure people, I’d like to thank all those responsible for Eduroam, from the high-level policy people to the back-room technical people who make it work and troubleshoot it. It’s gone from ‘this is a cool idea, hope it works’ to ‘this just works and I can take it for granted a lot of the time’, which is great and makes my work life so much easier.

While I’m talking about passwords, I have finally and belatedly made the switch to using a password manager. It’s such a relief. I had a pretty good system before, which worked with how my memory works, but it was fraying around the edges, and it didn’t cope well when passwords had to be changed. (That meant I had to memorise an exception to the system, which was a lot of extra work, and tended to disrupt the whole thing.)


It was a bit of work to set up the password manager, but mainly because it turned out I had over 150 username/password combinations to enter. I’m impressed at how well my old system worked – I hadn’t realised quite how many unique passwords it let me remember – but then again, if it wasn’t so good, I’d have gone for a password manager much sooner, and would’ve been better off as a result.

It’s fantastic. It’s like having a new superpower.

Faced with a demand to create yet another unique, strong password for some new online service or other, I can click a couple of buttons, paste in &QtjhQWFIkgr/(! and be confident I’ll be able to remember it later. When a site has idiosyncratic requirements (e.g. must have non-alpha characters, but only a small subset of them that doesn’t include the ones my system requires, or must not exceed 12 characters, must not have more than half lower-case letters, etc), I can do that and I don’t have to memorise another exception to the system. When a site I don’t trust demands I set up password recovery questions, I used to worry about divulging my mother’s maiden name, and struggled to think of what answer I could give to questions like “Favourite sports team” that I’d be able to remember later. Now I can simply say that my mother was born Miss 4^mSKZFI9@PNoa8 and that I’m a lifelong supporter of those paragons of sporting prowess, G3loF!aQynSR?Z%.  When I get yet another “this site has been hacked and all passwords stolen, please change your password”, I can go “Ok” and not worry about it.

For those who care about the details, I use PasswordSafe on PCs, Password Gorilla on Macs, and pwSafe on my iPhone, all synced via Dropbox. The runner up was KeePass, but 1Password and LastPass looked Ok too, although my paranoia doesn’t like security software where you can’t see the source.

I’m pretty sure that which password manager you choose matters a lot less whether you use one, and wholeheartedly encourage everyone to use one.

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.

Looking harder at Course Signals

“Prediction is very difficult, especially about the future.” – Niels Bohr, Mark Twain, Yogi Berra, or perhaps someone else.

I think the nascent field of learning analytics is facing its first serious, defining issue.

It seems the claims for the effectiveness of Course Signals, Purdue University’s trailblazing learning analytics program, are not quite as robust as it seemed. You can get a quick journalistic overview from Inside Higher Education’s story (which has good links to the sources) or the Times Higher Education’s version (which appears identical, except the useful links have been stripped).

Traffic light sculpture, Isle of Dogs, Tower Hamlets

This post is a bit long. As Voltaire, Proust, Pliny or (again) Mark Twain may have said, my apologies, but I didn’t have time to write a short one. It’s important, though.

Continue reading “Looking harder at Course Signals”

Password change log

So, it’s mandatory password change day for me today. In line with many organisations, the OU requires its users to change their password at regular intervals – at the moment, every three months/90 days. Also, in pursuit of improved security, they’ve recently reduced the number of unsuccessful login attempts you can make on your account before it locks you out. This has the unfortunate side effect of meaning that when you change your password on your desktop machine,  your phone or tablet – sitting quietly in your pocket or handbag – will keep trying to connect using your old password, which will lock you out of your account.

I had a very bad encounter with this issue six months ago when I lost most of a day to it. The problem was made particularly hard to resolve by the fact that we’d just gone over to IP phones connected to our PCs, so when I was locked out of my account, and therefore my PC, I couldn’t phone IT to get them to unlock my account, and had to go to another office some way away to ring them.  Which meant the ‘Try now – is it working?’ bits of the conversation had big gaps in the middle while I hung up and scurried back to my desk, then went back to room with the phone, redialled, waited to get through, and then updated the new person at the other end of the line with what’s going on. (Things are better now – I have a direct IP phone on my desk, and these days my office usually has people in it, which means I can borrow a phone from someone if that one’s not working.)

Three months ago – the last time I had to update my passwords – I was aware of the problem, and did it carefully and systematically in just under an hour, with no accidental lockouts! I also kept a sketchy list of what I did. Today I did it again, tweaking the list a bit as I went so I can follow it quickly next time.

Here’s my procedure. I’m posting it mainly to make it really easy for me to find in three months’ time (and I’ll print off a copy and leave it in my desk drawer). It’s obviously only directly relevant to me and my devices, but it might be useful to other OU people or people with a similar setups.

Bombe Machine, Bletchley Park
(cc) mendhak on Flickr

Continue reading “Password change log”


I’ve been reading Clay Shirky’s latest essay. He takes the usual Clay Shirky line: technological change, in the transformatory shape of the internet, is evidently about to profoundly disrupt a large sector, and the current incumbents are not going to be able to do much about it. But the good news is that more people are going to get more of what they want than before, and for a lot less money. He’s ridden this line over the last ten years, with classic essays like “Help, the price of information has fallen and it can’t get up” which analyse – usually presciently – the disruption that’s happening to industries like music, books, newspapers, TV, video, film.

In this latest essay, the industry facing change that is almost literally inconceivable to the incumbents is higher education, and the current shape of change is the MOOC. His prognosis?

“We’re probably going to screw this up as badly as the music people did.”

He’s a better essayist than me, so really, in all honesty you’d be better stopping now and reading his writing instead.

Android invasion

Continue reading “MOOCs, OER and Wikipedia FOR GREAT JUSTICE”

Fat pigs and assessment reform

One hoary old saw in education says that “you don’t fatten a pig by weighing it”, which is true enough, and I was riffing on this when I tweeted this morning that “Trying to raise standards by changing the exam is like trying to fatten a pig by buying new scales.”

This was in response to some not-very-good soundbites I’d heard around the proposal – yet to be unveiled – to change GCSEs (the exams most 16-year-olds take in the UK England and Wales). But it’s also relevant in the wider, international debate around high-stakes and standardised testing.

(CC) Chesi – Fotos CC on Flickr. No resemblance between any of the participants in this debate and a pig is intended.

Everyone (well, nearly everyone) wants educational change, and everyone (well, nearly everyone) agrees that assessment really matters. Alas, there is less agreement on how education should be changed, and still less on what should happen to assessment. There’s even less agreement on what “raising standards” means in an assessment context: Increased average marks? Mean or median? More students passing? More accurate exams? More precise discrimination between scores? More reliable or consistent exams? Fewer students failing? More students getting top marks? Fewer students getting top marks? More students learning more and better? More valid exams?

Buying new scales, or weighing more often, isn’t going to fatten the pig on its own. I seem to recall that the “you don’t fatten a pig by weighing it more often” argument was deployed when the UK system was changed in favour of more continuous assessment. But what you measure and how is crucially important in what the educational system as a whole ends up achieving. If your scale is broken, or measures the wrong thing, or measures it badly, you’re going to struggle to improve.

This is one of the themes I harp on about in a learning analytics context (including in my LAK12 paper The Learning Analytics Cycle, now available open access form!). If you make a system that optimises for a particular metric (say, exam results, or completion rates, or whatever), but that metric doesn’t reflect what you actually value, you’ll be making things worse, not better.

If what you actually want is more lean meat (or, heaven forfend, happy pigs), then optimising your farming system for the pig’s weight alone is not going to help you: you might end up with bigger pigs, but more fat and less muscle. Maybe you don’t actually want to eat pork anyway.

What’s on the exam, and how things are examined, is crucially important. Teachers and learners are not daft, and generally want to do as well as they can. They’re also under increasing external scrutiny and pressure to do so. Changing what’s assessed, and how, will not on its own improve learning. But assessing what you don’t value is likely to make things worse, not better.

I’m really not convinced that the most important thing we want sixteen-year-olds to have learned is the ability to get good marks in a single-shot closed-book hand-written three-hour exam with no consulting of references, sources, notes or colleagues. But I think I might have to wait a while before that view is reflected in the formal assessment system – though hopefully not until pigs fly.

This work by Doug Clow is licensed under a Creative Commons Attribution 3.0 Unported License.
No further permission needed to reuse/remix (with attribution), but it’s nice to be notified if you do.

Learning from dolphin learning

The conference dinner for LAK12 was at the Vancouver Aquarium, in Stanley Park. The highlight of the evening was a dolphin show.


For me, it raised lots of interesting issues linked to themes emerging in the conference. Dolphins are legendarily clever, and I was reminded powerfully of Douglas Adams’ Hitchhiker’s Guide to the Galaxy on relative intelligence:

For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much—the wheel, New York, wars and so on—whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man—for precisely the same reasons.

(Which also reminds me of how far we’ve come with inclusive language.)

Continue reading “Learning from dolphin learning”

The Sinister Sausage Machine

There’s a longstanding argument – currently very live in the UK – about whether higher education is or is not a market, or should or should not be.

If it is one at all, it’s not a straightforward market by a long chalk, no matter how hard you might try (if, for example, you are a Government trying to make it in to a market).

In a market, you have sellers, products, and buyers. But in HE these are all unclear.

In fact, I think, the confusion between all these, implicit in the market view of higher education, makes a university look like nothing less than a Sinister Sausage Machine.

Summer = Grill = Hotdogs!
photo (CC) Mike Johnson -

Continue reading “The Sinister Sausage Machine”

There is no bubble in UK Higher Education

There’s increasing concern that there is a higher education bubble.

Is there a higher education bubble in the UK?

This is an easy question, with an easy answer: No. There is no higher education bubble in the UK.

(Whether there is one in the US is a very different and much harder question, to which I may return later.)

Free Baby Blowing an Enormous Bubble Creative Commons

(photo (CC) by D Sharon Pruitt) Continue reading “There is no bubble in UK Higher Education”

Marx and Learning Analytics: Towards a praxis of educational improvement

There’s a serious danger, I think, in Learning Analytics, of producing and processing ever more data in ever more exciting and attractive ways, but which aren’t connected up to making things any different for learners.

As Marx famously put it in his Theses on Feuerbach,

Die Philosophen haben die Welt nur verschieden interpretiert; es kommt aber darauf an, sie zu verändern.

(Philosophers have only sought to interpret the world in various ways; the point is to change it.)

This is nowhere (!) more true than in learning analytics. Analysis without action is wasted effort – which we can ill afford now. Those of us with a penchant for data, sums and code (I know, I know, but we have our uses) can get terribly excited about new sources of data and new analyses. It’s great to come up with new ways of understanding and interpreting what’s going on. But if that effort doesn’t translate in to better action – it’s pointless.

On the other hand, action without analysis may be making things worse – you just don’t know. (A point sometimes lost on misinterpreters of Marx who focus only on the ‘change it’ bit. You can say many rude things about Marx – and I sometimes do – but not that he was short on quantity of analysis of capitalism.) We’re certainly going to be seeing a lot of dramatic change in universities in the coming years. If we don’t have the right evidence supporting decision-makers in that process, not only will the decisions be made in the dark, we won’t even know how wrong they are.

That’s why there needs to be a learning analytics cycle:

The process from learners to data to metrics/analytics is the core of what learning analytics is. But those outputs need to feedback to learners if the process is to be of any real value. (As Tony Hirst mentioned in his Learning Analytics Pre-Workshop, in technical terms, we’re talking a closed-loop control system.)

For me, this means engaging with all the frustrations of internal systems, processes and decision-making. If I was just interested in research outputs, I could just dive in to the systems I can get access to, grab some data, do some analysis, and go. It’s much harder to arrange and endure the endless meetings, papers, persuasion, and frustration that is what happens when you try to make a substantial change to a large bureaucratic organisation like a university. But that’s where the real value and potential lie.

Learning analytics starts with the learner, and their ‘data exhaust’. We need to make sure it gets back there again.


NB For clarity (thanks anonymous tipster!) that last sentence is meant entirely figuratively and I am emphatically not advocating gassing students. That would be Evil and Wrong. I’m trying to make things better for them.

(This post brought to you by my desire to bolster my resolve and enthusiasm for helping to work up an internal project along these lines. Will say more if and when there’s things to say.)

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.