CALRG Conference – day 1

Notes from the 29th Computers And Learning Research Group annual conference. I’m discussant for the first day.

[Added later: This was liveblogging but due to my stupidity with the Save instead of the Publish button it’s 24h later than what one could with entirely reasonably call live..]

Designing Learning activities: processes, tools and research issues (Andrew Brasher, Simon Cross, Grainne Conole)

“We’re now starting to expand”

Shift of focus from content to activity (so I’m producing more content)

How can we design learning activities which make effective use of tools and pedagogy? How can we scaffold and support the design process?

Been going for 1y, funding till Dec 09; focus shifting from developing tools to rolling out and embedding in Faculties from Autumn.

Gathering empirical evidence to understand the design process. Case studies, workshops, interviews, course observation & evaluations.

IMS Learning Design needs you to be able to think like a computer programmer; it produces executable code. LAMS is easier to use but still is executable. But neither support the design development process, or offline activities.

So CompendiumLD – and specialised notation (loose). Generally, time flows downwards (like UML swim lanes but less formal). Actors perform actions making use of tools and resources to reach a specific outcome. Has context-sensitive help – gives you tools based on the label you type (keyword lookup – discuss, organise); also ‘more about instant messaging’ – Google Custom Search Engine.

Schneiderman 2007 – tools to support creativity – functionality that a tool should offer:

Low thresholds, high ceilings and wide walls.

Next developments to tool: time estimates. Also visual mapping of activities to learning outcomes.

Discussion about evaluating learning design. Consensus that it’s really complex, especially separating influences. (Interesting conversation with David Hawkridge about contrast between being didactic and being learner-led – Betty Collis example about oil rig engineers Masters. Students agree the assessment criteria – and perhaps the assessment itself. And curriculum. But tension with defined/fix curricula and hence accreditation, which is Universities’ USP (?).)

Simon’s tall so can’t read notes if they’re on a table, so he brings along a stand for his notes to sit on (a document stand usually used beside a monitor).

Case studies – feedback that it’s difficult to abstract from specific examples – but the converse is also hard.

Evaluation is hard, complex, evidence tricky.

Feedback in formative e-assessment: closing the loop in distance learning (Stylianos Hatzipanagos)

Project mostly at King’s Learning Institute; funded by Centre for Dist Ed of Uni of London. Also includes Bob McCormick. Three ODL environments (King’s, UoL external programmes, OU); investigation of relationship between formative assessment and learning technologies. Models of formative assessment. Technologies work has mostly been about objective tests rather than effective assessment (depending on what you mean by effective, of course!) vs Bob McCormick (2004) saying promotes innovation and deeper thinking.

Research-led institutions focus on examining/summative. UoL external programmes are very diverse. OU emphasis on periodic assessment and systematic provision of feedback.

Some tutors equated ‘formative assessment’ with ‘continuous assessment’.

‘Closing the loop’ formative feedback rarely takes place in courses with emphasis on end of year (summative) assessment.

Cross-fertilisation between distance learning and f2f learning would be good.

Improving spoken English with ICT? ‘English in Action’ in Bangladesh (Adrian Kirkwood)

Light relief break – just a few ramblings (!). Big motivator of break between Pakistan and Bangladesh (independence 1971) was Urdu vs Bangla. So teaching English is an interesting project. “Very flat” – like Norfolk.

DfID funding for 9y (!) from May 2008, launched from Downing St, aims to have impact on 25m people. (Out of 150m population.) Request came from Bangladesh – tool for participation in global economy, empowerment and development.

Partners: BBC World Service Trust, OLSET (South Africa), OU. TV and radio programmes, mobile devices stuff. Builds on experience of OU’s DEEP Project in southern Africa … also TESSA? OU bit focusing on developing teachers.

English already major timetabled subject in school; aim to improve communicative skill, rather than written exams which you can pass by learning the grammar rules. (Why not change the assessment? Hard to do.) Problem with low skills level in (communicative) English in the teachers too.

Parallels with situation in Southern Europe (Spain, Greece, Portugal etc) and Japan right now too – very little oral work in English assessment.

Another big project where it’s hard to pick out a big-R Research angle from the general this-is-a-good-thing development one.

Navigate through materials: redesigning a language course with a modular flexible approach (Fernando Rosell-Aquilar)

Issues on existing Level 1 courses feeding in to remake: Poor retention. Too much content, inaccurate timings, repetition, linear/directive structure with little flexibility.

So took ‘learning objects’, modular approach. First on L140, Spanish.

Students approached from different places and all thought they’d started from the ‘right’ place. One found the approach ‘random’. Free to make choices.

Blended tutor support – f2f, Lyceum, telephone, forum, student support.

Have a blog directory of all students – using the Glossary functionality in Moodle! So students can put their own entry in. But can also change your entry when you post to blog so can see who’s updated via Browse by Date function in Glossary.

Chris and Chetz in the Datacapture Suite with wikis, blogs and learning objects (Chris Pegler and Chetz Colwell)

Accessibility in the Web 2.0 environment. Work in context of PROWE (with Leicester) and Stor Curam – now Learning Exchange. Repository projects. Expert accessibility.

Parallel with Web 2.0 with the introduction of Windows, when whole load of new accessibility issues emerged and a lot of catch-up needed.

Big problem with accessibility features being present but not ‘visible’. Classic errors with tab order wrong, labels for textareas wrong, and so on. There are techniques and ways of doing this right but often don’t.

Tagcloud vs JAWS … reads it with ‘link …’ after each word. And no indication of the size of the tag. Information in the popup, also hidden from JAWS (doesn’t read TITLE elements by default – which you want). Could simply include the number in the tag itself, which is probably better than a separate list (and is better for all, too!).

Video records useful for mediating in disputes about what you’ve found. (“The users are using it wrong!”)

Equability and dominance in online forums (Jon Rosewell and Tony Hirst)

Analysis of dominance of forums by small number of posters – only looking at number of posts (for now, because it’s easy). Could lead to dashboard approach to monitor health of forums. (Link to Student Support Review?)

Mostly large, moderated without tutors (peer support). 10pt/10wk courses, 100-800 students. 4 courses, 36 forums, 3000 posters, 27000 posts. Forums with ostensibly different purposes.

Robin Mason rule of thirds – 1/3 post many, 1/3 post few, 1/3 lurk. Bit more subtle. Also do-not-attend at all.

Pattern of posts made by posters – oh wow, it looks like a power law. Small number making a lot of posts, long tail making very little. Jon calls it a J-shaped distribution. Showed neat animation of patterns from loads of course/presentation and they look very, very similar.

Ecological analogy – communities often composed of dominant species plus long tail of rare species – biodiversity indexes. Indexes need to capture both richness (species count) and equitability (relative abundance of species). Shannon’s index widely used.

The distributions in the model go straight when plotted as log … but not for real data in biology (get S-shaped curve). His forum data aren’t straight or sigmoid, it’s a curve.

Diversity vs richness data cluster very closely on a curve. Equitability doesn’t vary with richness (number of participants). Startlingly little variability by different conferences apart from bare number of people.

No. of posters versus number of posts – seems to tail off: as the number of posts increases, the proportion of people prepared to contribute tails off. (Causality direction though.) End up with one person writing 10% of the posts … pretty much all the time, over a certain size of forum.

Patterns of participation seem remarkably consistent – so doesn’t tell us much about what makes for good forums! (Although these are all open, general forums with no direction.)

But the patterns are only in terms of the maths – the experience would be very different in many different forums. Whaddaya know, qual and quant give you different structures.

Link to Clay Shirky’s work on power laws and blogs. And my playing about with the KN – docs by user, downloads by doc – all look the same.

Using computer tools to scaffold argumentation: A case study in a postgraduate science community (Canan Blake and Eileen Scanlon)

InterLoc tool – built by Digital Dialog Games JISC project and many others. Rules and roles for participants: highly structured/reasoned dialogue, based around ‘locution openers’.

Latest version – InterLoc3 – is web-based.

Trial sessions, based around discussions of controversial topics (creationism in schools, DNA database). Does keep tone civil, and collaborative questioning, clarifying and justifying positions takes place.

Have to look at content of the discussion, not mechanical features, to see the value. So content analysis is next step.

The (In)visible WItness project: working with children and young people to explore gendered representations of science, technology, engineering and mathematics (Jenni Carr, Liz Whitelegg, Rick Holliman, Eileen Scanlon and Barbara Hodgson)

Research with children & young people, not on. Dave Gauntlett (1996) quote about children being ‘researched on’ rather than ‘worked with’. So conceptualise children as active interpreters of media messages. (Children constructed as incompetent media users, compared to competent adults. Ha! Often the other way round.)

Media literacy very high; good appreciation of broadcasters and audience. And creative, with interactive/multi-platform programming. Expectation of interaction. Had a sense of what’s objectionable, used stereotypes – but complex and nuanced phenomenon. (Also, I’d say use of stereotypical images is very media-savvy – in the sense that that’s what media does.)

They consistently understood difference between presenters and scientists; understood that presenters didn’t really know science.

Alas, the proposed prize of a day out with scientists wasn’t seen as a big enough draw, so they went for Paris Hilton and Orlando Bloom instead.

Recent e-learning research and development at the University of Leicester (David Hawkridge)

Vote: who wants PowerPoint, who wants paper, who wants me to talk? (Only one asks for Powerpoint … and gets the USB stick.)

OU gave out 78 gramophone records – on A100! It had music, poetry and some prose. Surveys showed not everyone had the tech to use it. How else to give audio? Oooh, shiny new audiotape technology.

Podcasts … a modern version of deploying audio. Lots of ways of using them.

All of Gilly Salmon’s projects have animal names (IMPALA, MOOSE, WoLF …). Cool naming scheme.

Also stuff on SecondLife (MOOSE) and Pocket PCs (WoLF). Wrong equipment problems; steep learning curve – lot of time needed from very time-poor people.

Issue with 40% of students not wanting work-related podcasts mixed up with their own entertainment/music stuff on their MP3 player. General perceived barrier – also with texts, emails, etc.

Evaluating the Accessibility of a Virtual Learning Environment within ALPE (Chris Douce and Wendy Porch)

Accessible eLearning Platform for Europe: is there a market to support a company providing consultation and resources around the production of accessible e-learning material?

Platform is dotLRN with UNED enhancements for accessibility.

Got participants in to do tasks; struggle to recruit (multiple projects at the same time). Content is SCORM-generated. Hard to locate accessibility technology. (Lunar, Window Eyes, Guide, JAWS … etc. Software is expensive and hard to find, especially at short notice.) Video feed interrupted by OU PC security features.

Lots and lots of very rich data – 20h of video.

‘Virtual learning environment’ not commonly understood. Inconsistent terminology bad. Complexity bad. Inconsistency between content viewing controls and other system not an issue. Time to complete tasks a lot longer for users with disabilities. Users with disabilities are very, very diverse.

Developers put a lot of effort in to custom access keys … and nobody used them. (They often conflict with the other assistive tech.)