Data science, research, analytics, technology development, and people
Author: dougclow
Data scientist, tutxor, project leader, researcher, analyst, teacher, developer, educational technologist, online learning expert, and manager. I particularly enjoy rapidly appraising new-to-me contexts, and mediating between highly technical specialisms and others, from ordinary users to senior management.
After 20 years at the OU as an academic, I am now a self-employed consultant, building on my skills and experience in working with people, technology, data science, and artificial intelligence, in a wide range of contexts and industries.
I’ve been reading Clay Shirky’s latest essay. He takes the usual Clay Shirky line: technological change, in the transformatory shape of the internet, is evidently about to profoundly disrupt a large sector, and the current incumbents are not going to be able to do much about it. But the good news is that more people are going to get more of what they want than before, and for a lot less money. He’s ridden this line over the last ten years, with classic essays like “Help, the price of information has fallen and it can’t get up” which analyse – usually presciently – the disruption that’s happening to industries like music, books, newspapers, TV, video, film.
In this latest essay, the industry facing change that is almost literally inconceivable to the incumbents is higher education, and the current shape of change is the MOOC. His prognosis?
“We’re probably going to screw this up as badly as the music people did.”
He’s a better essayist than me, so really, in all honesty you’d be better stopping now and reading his writing instead.
One hoary old saw in education says that “you don’t fatten a pig by weighing it”, which is true enough, and I was riffing on this when I tweeted this morning that “Trying to raise standards by changing the exam is like trying to fatten a pig by buying new scales.”
This was in response to some not-very-good soundbites I’d heard around the proposal – yet to be unveiled – to change GCSEs (the exams most 16-year-olds take in the UK England and Wales). But it’s also relevant in the wider, international debate around high-stakes and standardised testing.
(CC) Chesi – Fotos CC on Flickr. No resemblance between any of the participants in this debate and a pig is intended.
Everyone (well, nearly everyone) wants educational change, and everyone (well, nearly everyone) agrees that assessment really matters. Alas, there is less agreement on how education should be changed, and still less on what should happen to assessment. There’s even less agreement on what “raising standards” means in an assessment context: Increased average marks? Mean or median? More students passing? More accurate exams? More precise discrimination between scores? More reliable or consistent exams? Fewer students failing? More students getting top marks? Fewer students getting top marks? More students learning more and better? More valid exams?
Buying new scales, or weighing more often, isn’t going to fatten the pig on its own. I seem to recall that the “you don’t fatten a pig by weighing it more often” argument was deployed when the UK system was changed in favour of more continuous assessment. But what you measure and how is crucially important in what the educational system as a whole ends up achieving. If your scale is broken, or measures the wrong thing, or measures it badly, you’re going to struggle to improve.
This is one of the themes I harp on about in a learning analytics context (including in my LAK12 paper The Learning Analytics Cycle, now available open access form!). If you make a system that optimises for a particular metric (say, exam results, or completion rates, or whatever), but that metric doesn’t reflect what you actually value, you’ll be making things worse, not better.
If what you actually want is more lean meat (or, heaven forfend, happy pigs), then optimising your farming system for the pig’s weight alone is not going to help you: you might end up with bigger pigs, but more fat and less muscle. Maybe you don’t actually want to eat pork anyway.
What’s on the exam, and how things are examined, is crucially important. Teachers and learners are not daft, and generally want to do as well as they can. They’re also under increasing external scrutiny and pressure to do so. Changing what’s assessed, and how, will not on its own improve learning. But assessing what you don’t value is likely to make things worse, not better.
I’m really not convinced that the most important thing we want sixteen-year-olds to have learned is the ability to get good marks in a single-shot closed-book hand-written three-hour exam with no consulting of references, sources, notes or colleagues. But I think I might have to wait a while before that view is reflected in the formal assessment system – though hopefully not until pigs fly.
I’m Doug Clow, and this is my introduction for Ed Startup 101. Hello everybody! I often say that video is rubbish, so this is not my usual medium. But it’s a much more personal and direct one for lots of people, so here’s me. If you want a longer, text-heavy intro, I have one of those.
This is the light version.
I’m an academic (a member of faculty, for our American and Canadian Cousins), and I’m proud to work at the Open University in the UK. I really like fiddling around with new technology and seeing what I can do, and I really like feeling that what I’m doing is helping people – helping them learn more and becoming better able to make a difference in the world. So being an educational technologist at the OU is great.
It’s a step away from frontline teaching, though. At the OU, the academics design and develop the courses, and our associate lecturers do the direct teaching. And I don’t even do that much direct course development.
There’s an old (and horrible) saying that those who can, do, and those who can’t, teach. There’s an extension that says that those who really don’t know what they’re doing teach teachers how to teach. I’m the one who’s helping those guys.
As you might be able to tell, in presentations, I like to use the humorous technique of self-deprecation, but I’m not very good at it.
This week, I’m officially on holiday – or vacation in International English – so I’m doing the opposite of what lots of people do in a MOOC: I’m only dropping in from time to time for the first week, but will be more present later.
It is the time of MOOCs. Massively Online Open Courses – of various sorts – are officially the buzzword in the field at the moment. Most recently, a bunch of UK universities have signed up with Coursera, including the University of Edinburgh. The Principal of Edinburgh, Sir Tim O’Shea, is a very accomplished senior manager, and knows his stuff in the ed tech field, so at this point one can’t write it off as misguided foolishness.
I’ve signed up* to Entrepreneurship in Education – Ed Startup 101 for short, #EdStartup for Tweeting. The course is:
designed to acquaint educators, educational researchers, and others to the world of entrepreneurship and intrapeneurship, help you decide which one is right for you, and support you in the first steps of your journey.
The course is an old-school connectivist-style MOOC which ‘focuses more on building community and learning together socially than on watching video clips and answering multiple choice questions’. Which is my preference by a long chalk. In that spirit, here’s some positioning thoughts from me just ahead of the course start.
Here’s an organising summary of stuff from the Second International Conference on Learning Analytics and Knowledge, LAK12, held 29th April to 2nd May 2012 in Vancouver, BC.
–
This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.
Looking for reviewers, conference. We are still a bit biased towards the male gender, looking to improve that balance.
For 2013, have set up a few goals, to knit together a wide, fragmented community. Informal discussions so far.
One to host monthly session to showcase learning analytics activities. Maybe bringing in Knewton, or interesting researcher in a related field. Build regular contact.
Formalising the legal structure, creating a transparent process that has a democratic model for who leads, and what, how decisions are made.
Regional events – SoLAR Storm. Contact through the Google Group, maybe a listserv instead.
Research lab, some students interested. Looking at about 5-6 students, maybe more. If you as researchers have interests in working with them, is an option.
Executive Committee is the people who are supposed to be doing things. Steering Committee is the people we connect with to get a sense of direction, advisory board; guidance around activities. Need to improve our relationship, clarify. Better quality information flows. Making it clearer – currently can’t trace decision-making process. Fine while small group; not scalable.
Liveblog notes from Wednesday morning plenary session at LAK12 – George Siemens’ keynote.
(This photo is taken from more-or-less where the conference centre/hotel is, and I have a view of this from my room’s window. Except this morning it was drizzly and almost totally grey. Welcome, as they say, to Vancouver.)
Shane Dawson thanks Al Essa and Desire2Learn for sponsoring the conference dinner at the Aquarium. Invites everyone to the SoLAR AGM. Gives out SoLAR T-shirts as prizes for helping, tweeting, and so on. Showed visualisation of reference map from the papers at the conference – it’s interactive so you can explore it; will be on the LAK12 website presently.
The conference dinner for LAK12 was at the Vancouver Aquarium, in Stanley Park. The highlight of the evening was a dolphin show.
For me, it raised lots of interesting issues linked to themes emerging in the conference. Dolphins are legendarily clever, and I was reminded powerfully of Douglas Adams’ Hitchhiker’s Guide to the Galaxy on relative intelligence:
For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much—the wheel, New York, wars and so on—whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man—for precisely the same reasons.
(Which also reminds me of how far we’ve come with inclusive language.)
Doug Clow's Imaginatively-Titled Blog 