CALRG Conference 2013 (1)

Liveblog notes from the 2013 Computers and Learning Research Group (CALRG) conference #calrg2013 – Tuesday morning, 11 June 2013.

Stumpled Upon
Stumpled Upon (cc) by Ian Sane, on Flickr

Eileen Scanlon: Opening Remarks

The new Regius Professor of Open Education, Eileen Scanlon, welcomes everyone to this, the 34th CALRG meeting. Thanks Canan for organising. To celebrate the award of the Regius Chair, to the OU, to IET, and to herself, the programme focuses on open education.

Tim O’Shea: Flourishing after Forty Three years in the wilderness

Professor Sir Timothy O’Shea, Principal & Vice Chancellor, The University of Edinburgh. Founder of the CAL group in the early 70s.

Everyone’s thrilled about the Regius Chair. Started PhD 43 years ago, in computer-assisted learning. Educated in Essex – Royal Liberty, Sussex (AI), Leeds (PhD Comp Sci), Texas (practical work), Edinburgh (4y postdoc), OU, PARC, OU, Birkbeck, Edinburgh; 19y at OU; Edinburgh 15y.

Machine code – 1964 – if everybody was forced to understand machine code, we would have order and understanding. For A level (experimental) wrote a maze learning program in Algol 60. CBLU at Leeds. Self-improving systems 1970 – discovery method for learning quadratic equations. Learning with Logo 1974. 1976 modelling subtraction errors. 1978 OU CAL Group. Advises (tongue in cheek) people to do as he did and listen carefully to the director of IET and then do otherwise. 1979 Micros for Managers. 1982 Micros for Schools. 1986 Smalltalk at PARC. Series of failures – We’ve failed to get everyone to learn to program in Logo, in Smalltalk, in BASIC. 1989 Shared Alternate Reality kit. 1997 Crystallography at Birkbeck. Good stuff there in geography and others, and crystallography -open source, RasMol. 2002 Edinburgh eLearning. 2007 Online Masters. 2012 MOOCs at Stanford.

False starts – intelligent tutoring systems, student models, natural language understanding, programming for all (maybe we’ll go there with Scratch – Uruguay is the only country that’s implemented one laptop per child – exploring all of them how to learn Scratch in Spanish, currently looking simpler than thought.), CYCLOPS (OU system – remote screen drawing, annotation), video tunnels (nicer than Skype! – but big wooden box; less high quality but ubiquitous now), touch screens (in 1974, had big boxes with touch screens). All prescient, but within universities we didn’t have the capability to get them out. Speech input, UK eUniversity, NCET (then BECTA) – good idea but abolished, NDPCAL, UK Subject Centres, International eUniversities (many started but failed), 50 IETs (originally every university would have one – Surrey, OU, King’s, CBLU at Leeds, sort-of at Heriot-Watt; now just the one left. At the minimum you can be the museum (!)).

Reasons for failure: too early, tech not tobust, to hard to ‘solve’ AI, platform dependent, person dependent, assessment unresolved (big permanent issue), funding inadequate. Ministers get excited, say here’s some millions of pounds, usually a factor of 10 too little for the goal – same applies for MOOCs.

Three case studies – innovation is technology driven’ austerity is a poor driver. Learner enhancement good. Tech and scale my focus. MOOCs aren’t going to save us a lot of money.

Micros for Managers

It was nice, Intel 8049 simulating itself. 25,000 users. First OU home computer kit, went out 1979.

Micros for Schools

OU 1982, 10,000s users. 5 microcomputer platforms, simple languages which turned out ephemeral. OU Home Computing Policy started in 1987 – students weren’t required to have them but could use them to study.

Making journals available – JISC

Helping SOAS to student enrolment.

Edinburgh –

Principal’s eLearning Fund – FY 2003-8, $6.3m, 64 projects from £20k to £150k, selection. Then investment in online Masters – building on the capability. £4.5m investment over 5y; all 22 Schools having 100 person online masters. Most successful one in surgery. Also equine health, law, elearning. Currently 31k conventional; aim 10k students online PG.

Now in to MOOCs.

Arrived in Stanford the day after MOOCs hit the news, signed up with Coursera. Went for very different courses, six. Stopped people who wanted to build their own platform – some not pleased. Edinburgh has the capability to build one but we didn’t.

Why? Reputation as early adopter of educational technology, has been fabulous. Exploration of new space to inform practice. Wish to reach as widely as we can with our courses. 100k started on philosophy course! Many sixth-formers, found ourselves doing widening participation work. Sharing experiences with peer universities. Fun! Very positive at Edinburgh Senate meeting about MOOCs.

Who? Limited data. Mainly 18-35yo, most >83% (?) have a degree qual already. 2/3 not US/UK.

Next steps – offer credit, accept credit, partnerships. Integrating MOOCs in to core first degree curriculum. We’ll be doing a lot more, looking at conversion. Equine Nutrition – that’s the baby MOOC, only 23.5k started, but has best retention and completion of any Coursera MOOC. They’re serious. If 1% go on to PG course, that’s lots, even 0.5% would double the enrolment.

For us reputation enhancement strong. Not so for Georgia Tech’s one. High risk game. Few times in my career as a manager I went leading edge; normally I say let 25 others do it first. Not free to produce (academic staff costs mainly) but not expensive. Spent £200,000 to get 308,000 hot sales leads, that’s a good deal.

Trans-national education in the UK – why not overseas campuses? 3% is done by overseas campuses. Online or distance 28% (mostly OU), validation of local programmes 51%, other (joint degrees, flying academics (18%). Only growing one is online or distance.

Cheery bit: New dawn. Online postgrad courses, partnerships. MOOC momentum, tech is cheap, ICT convergence complete, hybrid normal (online/f2f), platform independence (if you don’t, you’re nuts), statistical AI. When I did machine learning it was all symbolics, now all Bayes’ Law – Bayes was an Edinburgh alumnus. We can do it now because we have the numbers. More recently saw learning analytics using Bayes’ Law on MOOCs. Listening to Roy Pea talking about the big project – quote from my PhD thesis, which was impossible in the 70s. With 10,000s of learners it’s possible, can have statistical student models.

Technology is a driver, Moore’s Law still true. Off to CERN tomorrow for next big collider launch. Good for another 8-9 years. Metcalfe’s Law still untrue – group-forming networks scale 2^N; important that it’s not the case or the Internet would fall over. Cloud 40 years late but works. OU, Jisc and MOOCs scale. OU and JIsc sustainable; perhaps MOOCs are sustainable. Radical change is possible.

Acknowledges Jeff Haywood and Sian Bayne.


John: Tell us more about credits and MOOCs – is it Edinburgh M-level credit?

That’s what we’re thinking about. Our MOOCs are heterogeneous. If you look at it from our point of view, we offer 600 different undergraduate degrees. We are the most comprehensive university in teaching terms apart from Birmingham. Scottish 4y degree, online, with routes, is complex. An online masters in AI, equine health, it’s easier. There are aggregators – people taking stuff from our MOOC, others, aggregating for larger course targeted at African universities. My supposition is we’ll start doing it in small amounts at the PG level. We do a ladder – Cert, Dip, M – so do e.g. 1/4 of a certificate. They’ll still pay for the other 11/12ths. We will beef up the assessment side. Coursera doing good stuff on authenticating learners. We’ll do it piecemeal. For Edinburgh, professionally oriented PG courses. Online masters in Forensic Science, Palliative Care, with international partners. So used to give credit, accept credit. Assessment is the big issue.

Patrick: The world is 43 y behind you. What that you were working on, do you wish would still happen?

It hasn’t all happened. Have real emotional bond for young people programming. Was part of my own development, at 14, it was just so exciting. Woke up at 14 about thinking about setting registers. Incredibly exciting, most exciting thing between 14 and 18. Firmly believe that with IT/comms convergence, to understand modern world, and don’t program, it’s very hard. The Seymour Papert project of learning programming … We failed in Logo. Smalltalk. Basic. Don’t know if there’s enough in Scratch. I’d see that as helping young people. Gives more control to young people. Program or be programmed. That’s the big one.

Leigh-Anne: Noticed that bridge building charity launched a MOOC with EdX. Saw any future in MOOCs as a partnership between universities and NGOs?

It’s happening. Difficulty is philanthropist says everyone needs to understand this issue, here’s half a million pounds, produce the MOOC that enables them to understand. It’s risky. It is happening. The MOOC providers are banging their head on the wall. Issue with funders. Real issue – who do you take the money from. People say here’s 20,000 pounds, get everyone to understand ?diet. Will cost more than that. Easy to say give us the money and we’ll sort you out. Nervous about history on people working to get computers to ameliorate autism, dyslexia – but didn’t. A moral issue there.

Ann Jones: Open digital resources for informal language learners: a case study of Welsh learners’ practices

I’ve taken Tim’s suggestion – I’ve ignored issue about logos and attributions. Tim has been talking big, I’m talking about work in progress that’s very small.

Partly inspired by talk in CREET, Clare Crouch, founding director of Berkeley language centre. Heritage language learning – spoken by previous generations of your family, but less so now. Motivation different, a connectedness, part of your roots. Project focused on heritage/minority language learning in the UK.

Another line – personal history. First language in English, grew up in most Welsh-speaking town in Wales; mother didn’t speak Welsh, didn’t speak it at home. Went to school at 4 with no Welsh – and it was a Welsh-speaking school! Negotiated talking in the street. Never taught grammar. In secondary school, got red marks on Welsh work, never got on with teacher. When I realised I’d lost my Welsh, one reason was to get back at that teacher to show her. After this talk I’m off to do an exam in Welsh, just to show her.

Clip of minority language in action – Tura Arutura – appropriating tools. Met his father at a seminar in SOAS, who said ‘my son raps in Irish’. And indeed he does.

Welsh – spoken by about 20% of population in Wales (562k speakers); shock in 2012 Census was that it’s going down, not up, as was thought. Emerged from Brythoneg in C6th, common precursor of Celtic languages. Minority language by 1911. 8% in Cardiff, 42% in Aberystwyth, 88% in Caernarfon. Small speakers, uneven distribution – ideal for web2.

Intersection of literatures – work on minority and heritage languages, language learning and OER, Web 2 in language learning, mobile devices. Relatively little on learner’s informal practices. ‘cultural nerve gas’, Krauss 1992. David Crystal – An endangered language will progress if its speakers can [use online]. Cunliffe reckons it can help.

Crowdsourced Welsh Facebook, now in use. Study exploring extent to which it was normalised. Looked at profiles, social spaces, groups. Found bilingual, Welsh-only. 400 groups (some small, some 13k members). Pretty much in use.

Mobile devices increasingly important role in the learner’s toolkit of resources. Use what they have.

Focus on use of digital techs to support learning Welsh informally – interview small N (10 so far) by phone, short survey. 8 live in England, about half have a connection with Wales.

Early ethnographic study – Prosser 1986 – ethnographic study learning Welsh to see process and community, acquiring a new identity. Motivation very different than for non-minority language. There were some monolingual people in 1970s, but not now.

Some surprising motivations. One went to the Eisteddfod, spoken to in Welsh, wanted to be able to respond. Another, I did not want to be a monoglot English speaker and lived close to Wales. ‘I wanted to learn another language, so picked Welsh, because I was close’ – now lives in Derbyshire and drives to where they can pick up Welsh radio. More often – Welsh fading out in family history, stops being spoken in the household. In South Wales, parents would say, we’re not speaking Welsh, English is better for your education – sometimes a sudden, instant stop.  Another had many Welsh links, started look around, and then found there were Welsh lessons available in Basingstoke (!) so went for it.

Google translate – Gwgl (Welsh search engine, doesn’t appear to be official Google property). Email, text, Facebook, Twitter, blogs, wikis. Many other resources – Memrise, SSIW (Say Something In Welsh – first lessons free, then paid) – meetings, S4C, CDs, Big Welsh Challenge, BBC Wales, new one Y Bont (the bridge).

TV programmes on S4C and Clic (=iPlayer). Youtube, Radio Cymru. Facebook used a lot to connect learners in England – ‘adopt a learner’. Others. Blogs.


Rebecca: Struck at mention of crowdsourced. I worked with a group doing listings for S4C and Radio Cymru. Excitement in 80s when Welsh atlas came out, had been no reference source for that before. Is there scope for big crowdsourcing projects?

Only one I’ve bumped in to is looking at what happened with Facebook site. Active group of techie Welsh people. Bit of a barrier if learned as a second language, terminology is recent. Worth having a look at.

Martin: Interesting analogy with tensions in Wales. Interviews with people in villages – people move in, do we make them speak Welsh? Welsh-only makes a silo. Or speak Welsh in the ordinary Facebook.

Both. Some people choose English because they want to reach out. A tension.

Anne Adams: Different use of video. Creatively used.

Tim: OU Catalonia, teaches only in medium of Catalan. Could imagine an OU of Wales that’s only in Welsh?

In some, it’s smaller, the inner bit, that will be your experience if you do a Welsh degree. Can I imagine that? Probably not. Catalan is different, quite a lot of speakers, though is still minority – minority doesn’t necessarily mean small.

Mike Sharples: Massive open social learning: a Pedagogy for FutureLearn

Rocket Man (I Think It's Going To Be a Long, Long Time)
(cc) peasap, on Flickr

When Tim was talking about notable CAL failures – Cyclops, Logo – my past flashed before my eyes. You wait 30y for a major elearning success, and then suddenly they all come at once, which is great. We have the Open Science Lab, and we have FutureLearn.

The reason for the pedagogy – hope you all know about Futurelearn, MOOC platform set up by OU with 20 othter universities, BL, British Museum. Decided couldn’t compete on technology, but possibly we could on pedagogy. With people from OU, BBC, partners. Pedagogy very important.

Thanks Rebecca Ferguson, Russell Beale, Matt Walton for ideas and slides.

FutureLearn vision statement – which gives you pedagogy – open to all, social learning-through-doing. How do you enable social constructivist learning for the Futurelearn platform?

Long list of different ways that could happen – direct learning from others; knowledge sharing, vicarious learning, implicit learning, conversational learning, orchestrated collaboration, shared knowledge building, Zone of Proximal Development (ZPD). From this menu – which do we choose, how do we enable that at a distance, at scale (currently on iTunesU at 100,000s) – and all done in two weeks. That’s the great thing – in past, have 3y project, final report proposing some innovation. But here have to do it in 2 weeks because there’s a bunch of people in a basement in Camden who are waiting to implement what you say. New iteration of platform every two weeks. Has to embed excellent pedagogy. Exciting opportunity! Having said pedagogy needs to inform design for years.


First challenge is the challenge of massive – the OU still largest user of Moodle. FL development team considered taking Moodle as platform. After weeks of debate, decided no. 1, it’s too monolithic, around centralised course provision module, doesn’t fit with eclective, modular approach. Also, 2, it just doesn’t scale – at 200,000 it’s badly struggling. If you have a forum, new learner comes in, too much flashing in front of their eyes.

One approach – link conversations to content. All notes and questions are directly linked to content. Content includes a time-sequenced part of a video – e.g. ask a question at a specific point in a video. So don’t see a generic forum, see notes and questions directly linked to the content.

Another approach – personal activity feed. Take that contextualised information. Issue there in seeing in general what’s happening, want to keep up to date. Each learner has their own feed of content. Don’t want it flashing, scrolling every half second, need to prioritise it.

So then organising the social learning – three ways: Following (other learners; Twitter model), cohorts (large groups, assisted by mentor, doing course together), activity groups (small groups in shared activity for a short time).

Following – new learner follows pre-existing friends, people who seem interesting. Activity stream prioritises posts from people you follow. Learner sees people who are following, can follow back. Links persist after and across courses.

Example – see someone, decided to follow them. Then can see either everyone stream, or people-you’re-following stream. Possible relationships – inspiration, study buddy, tutor, mentor. Can be paid e.g. tutor.

(All the ‘person’ pictures have dots for eyes and a round O for a mouth – look surprised, or vacant/stupid!)

[NB This next bit about cohorts of 200 is what they decided not to do!]

Within a course, learners allocated to groups of about 200, last for the duration. But in cohorts where activity has dropped off (e.g. to <80), they are collapsed together.  Can associate mentors with courses. Mentor can see top-rated activity from all cohorts, copy posts across cohorts. Can see everyone in your cohort. Can follow people, see selected posts from other cohorts.

Problems – this relies on mentors to spread ideas across cohorts. Possibly complicated ‘collapsing’ mechanism. Some cohorts may have more activity than others.

So, for next iteration – cohorts 2.0 – changed the idea. Take social learning and apply to large groups. Within a course, tag learners by interest, or by mentor, to form ‘cohorts 2.0’. Or perhaps tag automatically (?). Learners and Mentor see prioritised activities of their cohort. Learners see prioritised messages from their Mentor. Everyone sees other messages but lower down the activity stream.

(Note to self: Mockups using Lorem ipsum text can be pretty misleading when the point is to direct your attention to the content! Hard to make sense of it, very hard to understand how the experience will be for the real situation. Very very hard to show someone having a different experience of the same interface thanks to filtering etc.)

Third strategy – activity groups. Short-lived, form together to do something. E.g. general discussion, or e.g. a jigsaw learning activity, or a peer assessment. Form in to small groups for particular activities. Based on who’s active at the time – e.g. at 10pm on a Tuesday night, form them in to a small group to do that activity. Learners allocated to different groups for the next activity. Randomly, or by ability.

Three mechanisms – bottom up following, top-down cohorts, activity groups – basic approach to doing different social learning and collaboration.

In the first version at least, everything is public.

That’s the first basis for it. We think it’s going to scale. That’s what the people in the basement in Camden are beavering away at. They’re used to building large scale applications. As long as the mechanism is scalable, we have people who can implement it.

And one more thing … (shows some screenshots of the platform as it was last night)

Liz: Why 200 for the cohort? Dunbar’s number? (Actually 150.)

That was our first thought. About the right sort of people you could identify with. But we decided not to go with that. If spread across students, some will be less active, some more, and only see inside that group unless a mentor decides to percolate material across. Want benefits of grouping but without rigid segmentation

Karen Kear: Sounds complicated but interesting way of dealing with people. Highly dependent on behaviour – e.g. rating things – IME students not that great at doing it. And assigning people to small groups, if some don’t participate. Lots of potential problems.

Rating – assumption is that people will learn vicariously, see things associated with content. If have large numbers of students, there will be some activity with all content. No assumption you’ll be an active contributor, but see it, but then become more confident and make contributions, aligned to bits of content. Helps people who are shy initially.

Karen: Not being shy, it’s rating.

If you’ve got massive, you only need 1%, or 0.1%, to get useful ratings. We’re getting value from the massive. The small groups, organising people – it’s still a problem, we have to solve. We’re wondering if there’s something different to do there. If organise on the fly, they’re online at the same time, currently engaged in learning – not allocated at the start of the week. But then have to ensure they’re doing something, or you’ll sit there rather folornly.

Tim: Part of the folklore of the OU was that it was specially suited to the shy or misanthropic. Making strong assumption that all will turn in to social, happy bunnies. Or long term space for misanthropes?

Absolutely space for misanthropes. It’ll depend on the course. Astronomy and Moons, maybe not heavy social engagement. But will see things happening. If on creative writing, expectation you’ll be a bit more creative and that you’ll write. More expectation of learner contribution. That’s what we’ve got to organise. Not assumption that all learners on all MOOCs will be engaged – but make it possible.

Tony: Fascinating. OU also has history of abandoning relationships. DO a 9 month module, all those relationships chucked out. Incredibly irritating, against social pedagogy. Are you assuming only one unit on FutureLearn, or will relationships continue?

Absolutely will continue. Want relationship with Futurelearn. Will have a persistent, long-term FL identity. Following will persist across courses. All material searchable from outside (!!!). Relationship with the concept, rather than a particular course.

Someone: (I couldn’t hear)

Very good question, don’t have a good answer. A lot of these things, want to build in the opportunity – e.g. for tagging, can say I’m interested in this, this and this. And use that to prioritise the activities. How that works in practice for a particular course and particular learners is to be determined. What happens if they shift interests, or change, add new, want to support that. Once have mechanism in the platform, that’s where the learning design comes  in. When have the possibilities, need to think about how you design the learning to make use of these opportunities.

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.

Author: dougclow

Experienced project leader, data scientist, researcher, analyst, teacher, developer, educational technologist and manager. I particularly enjoy rapidly appraising new-to-me contexts, and mediating between highly technical specialisms and others, from ordinary users to senior management. After 20 years at the OU as an academic, I am now a self-employed consultant, building on my skills and experience in working with people, technology, data science, and artificial intelligence, in the education field and beyond.