CALRG 30th Anniversary – Session 3

[Crossposted to Cloudworks]

Adrian Kirkwood

Evaluating the OU Home Computing policy. First courses in 1988. A meta-project, an organisational activity.

Previously, provided students with computing facilities since 1970s – remote access and at study centres etc.  Desktop computers entered the mass market.  New Home Computing Policy required students – on a few, specific courses – to arrange their own access to a PC.  Huge change in practice, not just for students.

The Home Computer required: “an MS-DOS machine with 512K memory, disk storage, mouse, and capable of supporting graphics”, “the technical strategy does depend on having an MS-DOS capability for under £500”.

Courses: M205 Fundamentals of Computing – ‘foundation’ computing course. DT200 Intro to IT. Sent them a modem! M353 Computational Mathematics – modelling tool.

Very high priority. Practical arrangements, additional costs, course completion impact?

Evaluation team within IET – Tony Kate, Ann Jones, Gill Kirkup, Adrian Kirkwood, Robin Mason, short-term assistants. Interested in longer-term educational and social issues associated with the change, not (just) the logistical and practical ones. Different ways of working all round.

Issues:  Implications for course design. How it could enhance T&L and support.  CMC – very important for a distance education institution, big shift for OU. Many questions about access and equal opps, especially wrt gender and age – a ‘yuppie’ effect on recruitment patterns? Social and physical context – loss of control and knowledge of the setup by the organisation. Institutional change.

Example – DT200 student read “when you receive your materials, copy your materials as a backup”. Student took a photocopy.

What happened?  It wasn’t a disaster in the first year, “we got away with it”, senior management lost interest in those aspects. More course teams added, wealth of information collected and alanysed for internal reports and external publication. Was it institutional research or academic research, or both? It varied across a spectrum.

New, current, project – “English in Action” in Bangladesh – DfID funding over 9 years.  Developing communicative English – spoken particularly – through technology-enhanced interventions.  Access there is still a big issue.

Mike Sharples

Was only here for two years “but it seems like a lot longer”; partly because keeps coming back but partly because it was a very formative experience.  First proper job after PhD. Partly because job interview on 8 Dec 1980 and heard that John Lennon had died, important transition time.  Partly because first person met was Liz Beattie, became partner.

CYCLOPS – in 1980- a telewriting system.  30 years ahead of its time. Had great help – a personal PA, and resources of BT to redevelop it to his requirements.

It was to support OU tutoring – students in Regions – either had telephone tutorials or had to drive to the regional centre.  CYCLOPS meant they could go to a nearby study centres – a few miles rather than fifty or more.

Shared screen telewriting plus phone conference – like an OHP at a distance. Could write, pre-prepared slides, overlay, multiple interaction.  True WYSIWIS. Up to 10 centres connected in a live meeting.  Students preferred it to the other options.

So why not used now?  Framework for evaluation – look micro (HCI), meso, macro (organsitional) levels at each of usability, usefulness, efficiency, etc.

It worked!  Familiar system image (OHP), students operated it with no training.  Opened a cupboard door, connect it up, get it working … and it was Ok. BT conferencing centres started off – BT conference operators weren’t used to managing data connections, so had to set up their own.  Suited lots of interaction.

Worked at meso level too – tutors adapted it to their teqaching style. Adopted conventions – e.g. signing in with your handwriting at the start, identity.  Cyclops studio for pre-prepared illustrations – early Photoshop facility.

At the macro level … it worked for students, matched their needs.  Wrong business model – saved student travel costs but increased OU costs, for facilitator and line charges.  Unacceptable transfer (and increase) of costs.

Fast forward … to Smart Meeting Pro.  By Canadian company that developed SMARTboard.  Meeting room and conferencing system with telewriting system. “See how to write over applications”

Will it work? Probably not.  Micro – over-complex, is an add-on.  Meso – integration and purpose (vs smart boards).  Macro – connections (critical mass required) and meeting support.  Which is a bit sad.

(Mike’s lab do a lot of work with tech companies comparing/evaluating their tools like this.)

For technology to really take off, it has to: appeal to the youth market, and fit in to their social life.  Mini car in the 1960s – part of the 60s social life of London.  The CD-ROM – when marketed as serious CD-I as educational tool got nowhere, took off when part of computer games.  SMS and texting – small business market until teenagers discovered social uses.

What would happen for telewriting with young people and social networking?  Perhaps the new Nokia 5800 – Facebook, touchscreen – ‘tap here to write something’.  Combine Facebook (social) with telewriting.

Andrew Ravenscroft

Digital dialogues for thikning and learning.

Ideas came from conceptual change in science: collaborative argumentation key in realising stable conceptual change and development.  So developed dialogue modelling work-benge (CoLLeGE), then dialogue games (CSCL), then more flexible, powerful and easily-deployable digital dialogue game tools (InterLoc).

Learners in the ‘social web’ makes this even more crucial.  Worries about ‘The Thinker’, and Vygotsky. Greater emphasis on ‘learning dialogue’ but internalising what?  Home brew vs brewed by experts – quick and inexpert vs long-run.  Homebrew intellect vs Grolsch intellect.

What are we designing, predominantly?  New spaces for learning. Socio-cognative tools.  Improved semantic back-ends and knowledge networks.  Ambient pedagogies and ‘experience design’.  And ‘deep’ learning design.

Need to manage – or constrain – complexit.  Intelligent ‘anti-social’ software – from semantic web to the intentional web?  Sensible computing?  Bouncers on the door of courses.

Patrick McAndrew

Found his interview presentation from when he came to the OU.  Found a picture on his current website taken well before the slides were written.  Reanalysed it as a Wordle – tasks, framework, learning, course.  ‘Open’ doesn’t appear at all.

“Walter Perry told his new staff … .to design the teaching system to suit an individual working in a lighthouse off the coast of Scotland” – Sir John Daniels (no evidence found of whether Walter Perry said precisely that, but it was an idea in circulation)

Open then meant: contained, controlled, costed (course in a box) BUT ALSO available, accessible, all-inclusive, supported.  But that lighthouse keeper audience is shrinking.  Checked the quote a while ago, found a lighthouse keeper doing an OU course … and keeping a blog!  So the audience is changing.  People’s bags contain ‘too much technology’, world is becoming much more connected.

There is still a digital divide, but it’s not for us to solve.  If we assume the problems people have, we’ll get it wrong.  We should reach to the world out there, other initiatives address the digital divide.

We have gone open with our materials – OpenLearn.  Have learning that people are interested in the content, and the social connectivity.

Did a more current Wordle on last paper (with Grainne, Doug, et al) – OER, Learning, design, process, use, resource.  Getting Grolsch for free!

OLnet is about being open to the world in all sorts of ways, including our research approach.  Openness is at the bottom of communicate, share, learn.

Need to move to a more open version of open-ness, free up the control we have of the students. Accept that there is a free route.

Open now = unlimited, freed, free BUT ALSO available, accessible, connected, empowered.

CALRG 30th Anniversary – Session 2

[Crossposted to Cloudworks]

John Cook

Slides available in Slideshare.

Snapshot 1 – Cooperative Problem-Seeking Dialogues in Learning. (2000) to Snapshot 2 – Going for a Local Walkabout: Putting Urban Planning Education in Context with Mobile Phones. (2009)

Music a key feature throughout.  MetaMuse designed to adaptively structure interactions between pairs of cooperating learners – decisions made about traversing State Transition Networks (STNs). AI basis.  Lisp/Mac based.  Generated musical ideas fast so they could get verbalisation/externalisation leading to self-regulation/self-diagnosing – problem-seeking.

Picking up models of how pairs of cooperating learners.

Now at London Met, strange news lately, Learning Technology Research Institute. Prof of TEL, half-time helping university with e-learning. A pocket of excellence in the RAE.  RLO CETL, FP7 project CONTSENS, mobile learning, work with Agnes Kukulska-Hulme.  Urban area study, capturing pictures/VR as they go around. GPS-triggered events, show you old photographs/newsreels of the same area. Students work in pairs to solve tasks.  Schools started looking like prisons, then flatter.  High-end phones (HTC Diamond/N95), builtin voice recorder for capture of notes.

Continuity – the song remains the same?

User data still at the centre, and adaptively structuring interactions.

Important research issues: equity of access to cultural resources for education; learner generated context; appropriation; mobility and learning pathways; informal learning.

Informal learning has taken him to being an Investigative DJ on

Rick Holliman

Diverse media in here, multiple streams of information, affects how we use and produce information.  Particularly interested in science communication.

Abstract done as tweets – key events.

Followed Martian invasion – meteorite harbouring fossilised remains of ancient bacteria (?). Very controversial – was it an artifact or a real microfossil?  Much tabloid interest; interested in how science communicated in the media.

Then Dolly the sheep, 1997. Key questions – why is there only one sheep? Because the scientists doing it didn’t expect it to work, so used genetic material from their freezer … and then it did. So some controversy in the scientific – but not public – media about whether she was an actual clone because the background testing not done.

Another thing at the same time … shift in to online word in terms of news, around the UK general election. Guardian Unlimited, Electronic Telegraph.

Finger-length ratio: established in the womb, dependent on hormone balance at that time.  That’s fairly clear, but what that means in later life is much less clear.

Broadsheets changed from broad to tabloid , or compact, or Berliner. Categorisation becomes difficult – and newspapers exist in multiple formats too.  ‘Elite and popular’ almost works for printed media, but not for broadcast or online.

Language is changing, the way we describe things is also changing: abuse of vowels and pronouns is rife. The result of txting?

Many complexities of consumption and production, and data collection and analysis.

Claire O’Malley

Her new boss was on the Dolly the sheep team … and he has finished where she’s finishing.  Twenty years from NATO Advanced Research Workshop 1989, to CSCL 2009.

Conference proceedings in 1989 used a cartoon of ‘Computer-Supported Co-operative Learning’ showing a teacher standing on a computer (Mac SE) as a podium, pointing at a blackboard with ‘E=mc^2’ (shared representation), computer supporting interaction (!) but not getting in the way of teacher-student interaction (looking at each other).

Shared representations – several projects. Conceptual Change in Science. Ros Driver. 1980s, Ideas still here in latest project. More recently: Ambient Wood (Yvonne Rogers) – same thing but the technology is different. Get students to investigate real things, unmediated, but script the investigation (scripting is CSCL current buzzword) – give them representations of those.  Now Personal Inquiry (PI) with Eileen Scanlon et al.  Again, new technology but idea the same: unmediated science, mediation to help learners talk about it.

Another strand – communication. Shared ARK – Josie Taylor, Simon Buckingham Shum. Video-mediated communication with shared science simulation. Real-world question about whether to run or walk in the rain. (Answer is a brisk walk.) High-quality analogue video, real time, even enabled eye contact. (Cool!)  Video-Mediated Communication – link to superfast Janet ATM connect, very high-bandwidth digital video early/mid 90s – two video streams at once! Focus on talk that was produced. Task – same map, other instructs on a route using talking-heads video.

Interesting snippets of findings from all this video:

Despite the quality of connection – bandwidth, latency, eye contact – people don’t talk the same way as if they were face-to-face.  They just don’t.  Whether in next room or across continents.  The task can be differentially affected by that.

So if you want a bargain and you’re on dodgy ground, use the telephone not the video. If your case is strong, use video because you can persuade more.

People think that if they’re on a video, they’ll somehow leak the truth when they’re trying to deceive.  Likewise, they think they can pick up lies from others.  But people are awful at spotting lies on video, and if they do leak the truth when trying to deceive, it’s by voice, not by what they show.

People who can see each other tend to say less than on audio-only channels; gestures – nodding etc – are crucial to maintaining smoothness of interaction.

LEAD project – EU-funded – mediating f2f communication with computers using text chat … like we’re doing now in this conference with the Twitter backstream.  Good route for more interactive lectures.

Digital Replay System – these contexts produce great streams of data that take ages to analyse and make sense of.  National Centre for e-Social Science, to help people make sense of large datasets like this.  Digital ethnographyThings like auto-analysis of head-nodding.

On the ‘Horizon’ – new EPSRC Digital Economy Hub – at Nottingham – research on ubiquitous computing, big building.  Cloud computing, specks etc … very many people you don’t know will have a lot of data about you that you don’t know. How do we make it acceptable for people that they do? How do we deal with issues of privacy, identity, security?

CALRG Conference – day 1

Notes from the 29th Computers And Learning Research Group annual conference. I’m discussant for the first day.

[Added later: This was liveblogging but due to my stupidity with the Save instead of the Publish button it’s 24h later than what one could with entirely reasonably call live..]

Designing Learning activities: processes, tools and research issues (Andrew Brasher, Simon Cross, Grainne Conole)

“We’re now starting to expand”

Shift of focus from content to activity (so I’m producing more content)

How can we design learning activities which make effective use of tools and pedagogy? How can we scaffold and support the design process?

Been going for 1y, funding till Dec 09; focus shifting from developing tools to rolling out and embedding in Faculties from Autumn.

Gathering empirical evidence to understand the design process. Case studies, workshops, interviews, course observation & evaluations.

IMS Learning Design needs you to be able to think like a computer programmer; it produces executable code. LAMS is easier to use but still is executable. But neither support the design development process, or offline activities.

So CompendiumLD – and specialised notation (loose). Generally, time flows downwards (like UML swim lanes but less formal). Actors perform actions making use of tools and resources to reach a specific outcome. Has context-sensitive help – gives you tools based on the label you type (keyword lookup – discuss, organise); also ‘more about instant messaging’ – Google Custom Search Engine.

Schneiderman 2007 – tools to support creativity – functionality that a tool should offer:

Low thresholds, high ceilings and wide walls.

Next developments to tool: time estimates. Also visual mapping of activities to learning outcomes.

Discussion about evaluating learning design. Consensus that it’s really complex, especially separating influences. (Interesting conversation with David Hawkridge about contrast between being didactic and being learner-led – Betty Collis example about oil rig engineers Masters. Students agree the assessment criteria – and perhaps the assessment itself. And curriculum. But tension with defined/fix curricula and hence accreditation, which is Universities’ USP (?).)

Simon’s tall so can’t read notes if they’re on a table, so he brings along a stand for his notes to sit on (a document stand usually used beside a monitor).

Case studies – feedback that it’s difficult to abstract from specific examples – but the converse is also hard.

Evaluation is hard, complex, evidence tricky.

Feedback in formative e-assessment: closing the loop in distance learning (Stylianos Hatzipanagos)

Project mostly at King’s Learning Institute; funded by Centre for Dist Ed of Uni of London. Also includes Bob McCormick. Three ODL environments (King’s, UoL external programmes, OU); investigation of relationship between formative assessment and learning technologies. Models of formative assessment. Technologies work has mostly been about objective tests rather than effective assessment (depending on what you mean by effective, of course!) vs Bob McCormick (2004) saying promotes innovation and deeper thinking.

Research-led institutions focus on examining/summative. UoL external programmes are very diverse. OU emphasis on periodic assessment and systematic provision of feedback.

Some tutors equated ‘formative assessment’ with ‘continuous assessment’.

‘Closing the loop’ formative feedback rarely takes place in courses with emphasis on end of year (summative) assessment.

Cross-fertilisation between distance learning and f2f learning would be good.

Improving spoken English with ICT? ‘English in Action’ in Bangladesh (Adrian Kirkwood)

Light relief break – just a few ramblings (!). Big motivator of break between Pakistan and Bangladesh (independence 1971) was Urdu vs Bangla. So teaching English is an interesting project. “Very flat” – like Norfolk.

DfID funding for 9y (!) from May 2008, launched from Downing St, aims to have impact on 25m people. (Out of 150m population.) Request came from Bangladesh – tool for participation in global economy, empowerment and development.

Partners: BBC World Service Trust, OLSET (South Africa), OU. TV and radio programmes, mobile devices stuff. Builds on experience of OU’s DEEP Project in southern Africa … also TESSA? OU bit focusing on developing teachers.

English already major timetabled subject in school; aim to improve communicative skill, rather than written exams which you can pass by learning the grammar rules. (Why not change the assessment? Hard to do.) Problem with low skills level in (communicative) English in the teachers too.

Parallels with situation in Southern Europe (Spain, Greece, Portugal etc) and Japan right now too – very little oral work in English assessment.

Another big project where it’s hard to pick out a big-R Research angle from the general this-is-a-good-thing development one.

Navigate through materials: redesigning a language course with a modular flexible approach (Fernando Rosell-Aquilar)

Issues on existing Level 1 courses feeding in to remake: Poor retention. Too much content, inaccurate timings, repetition, linear/directive structure with little flexibility.

So took ‘learning objects’, modular approach. First on L140, Spanish.

Students approached from different places and all thought they’d started from the ‘right’ place. One found the approach ‘random’. Free to make choices.

Blended tutor support – f2f, Lyceum, telephone, forum, student support.

Have a blog directory of all students – using the Glossary functionality in Moodle! So students can put their own entry in. But can also change your entry when you post to blog so can see who’s updated via Browse by Date function in Glossary.

Chris and Chetz in the Datacapture Suite with wikis, blogs and learning objects (Chris Pegler and Chetz Colwell)

Accessibility in the Web 2.0 environment. Work in context of PROWE (with Leicester) and Stor Curam – now Learning Exchange. Repository projects. Expert accessibility.

Parallel with Web 2.0 with the introduction of Windows, when whole load of new accessibility issues emerged and a lot of catch-up needed.

Big problem with accessibility features being present but not ‘visible’. Classic errors with tab order wrong, labels for textareas wrong, and so on. There are techniques and ways of doing this right but often don’t.

Tagcloud vs JAWS … reads it with ‘link …’ after each word. And no indication of the size of the tag. Information in the popup, also hidden from JAWS (doesn’t read TITLE elements by default – which you want). Could simply include the number in the tag itself, which is probably better than a separate list (and is better for all, too!).

Video records useful for mediating in disputes about what you’ve found. (“The users are using it wrong!”)

Equability and dominance in online forums (Jon Rosewell and Tony Hirst)

Analysis of dominance of forums by small number of posters – only looking at number of posts (for now, because it’s easy). Could lead to dashboard approach to monitor health of forums. (Link to Student Support Review?)

Mostly large, moderated without tutors (peer support). 10pt/10wk courses, 100-800 students. 4 courses, 36 forums, 3000 posters, 27000 posts. Forums with ostensibly different purposes.

Robin Mason rule of thirds – 1/3 post many, 1/3 post few, 1/3 lurk. Bit more subtle. Also do-not-attend at all.

Pattern of posts made by posters – oh wow, it looks like a power law. Small number making a lot of posts, long tail making very little. Jon calls it a J-shaped distribution. Showed neat animation of patterns from loads of course/presentation and they look very, very similar.

Ecological analogy – communities often composed of dominant species plus long tail of rare species – biodiversity indexes. Indexes need to capture both richness (species count) and equitability (relative abundance of species). Shannon’s index widely used.

The distributions in the model go straight when plotted as log … but not for real data in biology (get S-shaped curve). His forum data aren’t straight or sigmoid, it’s a curve.

Diversity vs richness data cluster very closely on a curve. Equitability doesn’t vary with richness (number of participants). Startlingly little variability by different conferences apart from bare number of people.

No. of posters versus number of posts – seems to tail off: as the number of posts increases, the proportion of people prepared to contribute tails off. (Causality direction though.) End up with one person writing 10% of the posts … pretty much all the time, over a certain size of forum.

Patterns of participation seem remarkably consistent – so doesn’t tell us much about what makes for good forums! (Although these are all open, general forums with no direction.)

But the patterns are only in terms of the maths – the experience would be very different in many different forums. Whaddaya know, qual and quant give you different structures.

Link to Clay Shirky’s work on power laws and blogs. And my playing about with the KN – docs by user, downloads by doc – all look the same.

Using computer tools to scaffold argumentation: A case study in a postgraduate science community (Canan Blake and Eileen Scanlon)

InterLoc tool – built by Digital Dialog Games JISC project and many others. Rules and roles for participants: highly structured/reasoned dialogue, based around ‘locution openers’.

Latest version – InterLoc3 – is web-based.

Trial sessions, based around discussions of controversial topics (creationism in schools, DNA database). Does keep tone civil, and collaborative questioning, clarifying and justifying positions takes place.

Have to look at content of the discussion, not mechanical features, to see the value. So content analysis is next step.

The (In)visible WItness project: working with children and young people to explore gendered representations of science, technology, engineering and mathematics (Jenni Carr, Liz Whitelegg, Rick Holliman, Eileen Scanlon and Barbara Hodgson)

Research with children & young people, not on. Dave Gauntlett (1996) quote about children being ‘researched on’ rather than ‘worked with’. So conceptualise children as active interpreters of media messages. (Children constructed as incompetent media users, compared to competent adults. Ha! Often the other way round.)

Media literacy very high; good appreciation of broadcasters and audience. And creative, with interactive/multi-platform programming. Expectation of interaction. Had a sense of what’s objectionable, used stereotypes – but complex and nuanced phenomenon. (Also, I’d say use of stereotypical images is very media-savvy – in the sense that that’s what media does.)

They consistently understood difference between presenters and scientists; understood that presenters didn’t really know science.

Alas, the proposed prize of a day out with scientists wasn’t seen as a big enough draw, so they went for Paris Hilton and Orlando Bloom instead.

Recent e-learning research and development at the University of Leicester (David Hawkridge)

Vote: who wants PowerPoint, who wants paper, who wants me to talk? (Only one asks for Powerpoint … and gets the USB stick.)

OU gave out 78 gramophone records – on A100! It had music, poetry and some prose. Surveys showed not everyone had the tech to use it. How else to give audio? Oooh, shiny new audiotape technology.

Podcasts … a modern version of deploying audio. Lots of ways of using them.

All of Gilly Salmon’s projects have animal names (IMPALA, MOOSE, WoLF …). Cool naming scheme.

Also stuff on SecondLife (MOOSE) and Pocket PCs (WoLF). Wrong equipment problems; steep learning curve – lot of time needed from very time-poor people.

Issue with 40% of students not wanting work-related podcasts mixed up with their own entertainment/music stuff on their MP3 player. General perceived barrier – also with texts, emails, etc.

Evaluating the Accessibility of a Virtual Learning Environment within ALPE (Chris Douce and Wendy Porch)

Accessible eLearning Platform for Europe: is there a market to support a company providing consultation and resources around the production of accessible e-learning material?

Platform is dotLRN with UNED enhancements for accessibility.

Got participants in to do tasks; struggle to recruit (multiple projects at the same time). Content is SCORM-generated. Hard to locate accessibility technology. (Lunar, Window Eyes, Guide, JAWS … etc. Software is expensive and hard to find, especially at short notice.) Video feed interrupted by OU PC security features.

Lots and lots of very rich data – 20h of video.

‘Virtual learning environment’ not commonly understood. Inconsistent terminology bad. Complexity bad. Inconsistency between content viewing controls and other system not an issue. Time to complete tasks a lot longer for users with disabilities. Users with disabilities are very, very diverse.

Developers put a lot of effort in to custom access keys … and nobody used them. (They often conflict with the other assistive tech.)