CALRG Conf: Challenges for Game-Based Learning

Nicola Whitton – challenges for game-based learning. She keeps a blog at

Background in computing, HCI and gaming interfaces, then interested in educational side.

Some people sometimes learn some things from some games sometimes!

Can motivate or engage people who wouldn’t otherwise be so. Problem-based learning environments have a lot in common with instructivist learning environments.

Good games do a lot of what we know makes for good learning: Swift, timely appropriate feedback. Safe, can try things out. Scaffolding, things build up in difficulty. Offers goals and rewards.

But! Three big challenges.

1. Public perception. 2. We’re only scratching the surface. 3. Barriers are too high in practice.

Continue reading “CALRG Conf: Challenges for Game-Based Learning”

CALRG Conf: What do players think?

Jo Iacovides on What do players think about engagement, motivations and informal learning through digital games?

Rationale – Malone et al on games as intrinsically motivating, trying to harness for learning. Gee on how games promote active/critical learning, throuhg participation in affinity groups, semiotic domains – link to CoPs.

But lack of empirical evidence – especially outside MMORPGs. Potential often unrealised – not successful, lack of integration, socio-cultural factors. So need to further our understanding about how engagement, motivation and learning come together in practice. Link breaks down in some contexts.

Audiences are changing – from stereotypical young, male, semi-evil player, to family young/old male/female cooperative play. Games are more socially acceptable and prevalent – but we don’t know much about that.

So RQs around: motivation to play games, what affects engagement, how they describe learning, and the links between motivation, engagement and learning.

Continue reading “CALRG Conf: What do players think?”

CALRG Conf: Games and virtual worlds

Diane Carr, Martin Oliver and Caroline Pelletier on What are we actually studying?

Diane Carr

starts with a bit of background – they’ve worked in games research for a long time. Recently thinking about research design and how concepts might be developed and theorised, and their ramifications.

Latest work – Eduserv-funded project Learning from online worlds, teaching in Second Life – e.g. WoW. LPP, and so on. Couples who play together. Also teaching courses that are related – clinical ed and simulations, computer games studies, cmc, education and technology, literacy, affect, games.

Digra – Digital Games Research Association – has excellent online archive/library.

Continue reading “CALRG Conf: Games and virtual worlds”

CALRG Conf: xDelia

Gill Clough, Gráinne Conole, Eileen Scanlon on xDelia’s Design and Evaluation framework.

Not going to talk too much about the framework, more about the link to games.

xDelia is a pan-European project, €3.2m, looking at effects of emotional bias on financial decision-making of: traders; investors; individuals. Three year project, using bio-feedback (sensors) and serious games. Active workpackages on Traders and investors (OU – OUBS), Financial capability (Bristol), Games development (Sweden), Sensor development (Germany), Evaluation (OU – IET).

Continue reading “CALRG Conf: xDelia”

CALRG Conf: Digital Games, Gender and Learning in Engineering

Richard Joiner, Jo et al – Digital Games, Gender, and Learning in Engineering – do females benefits as much as males?

Work on Racing Academy.

UK context – STEM focus, interest in using video games to support learning of STEM, especially in the US (NSF committees). Reasons: video games part of kids’ lives; can be powerful learning; simulations give authentic, open-ended challenges;  research confirms benefits for situated learning. But some students don’t like or have access to games, concern that this particularly means girls. Early research (1980s) supported that boys more likely to have access, played for longer, preferred them more. But more recent research suggests female game playing is increasing. Lenhart et al (2008) found boys 99% played vs 94% girls. Evidence that females play casual games more than males. However, there are differences in amount of experience and type of playing – Heeter & Winn (2008) found male student logged thousands more hours of play than female student – so concern around video games in STEM education.

So question: are there differences?

Racing Academy – racing simulator. Players design cars to go as fast as they can and race against an AI car. Design process is intended to teach engineering – apply it to make it work better.

Study: Undergraduate engineering course, 143 males 15 females, 18.5-year-old. Pre/post test. Worked in teams of 3 and 4, project lasted 2 weeks. Grand prix at the end with a race-off.

Findings: Gender differences found – males more likely to play games, for longer, and different games. No differences in self-reported use of Racing Academy (no logging, alas); no gender differences in learning outcome; both improved. No gender differences in engineering identification – motivation towards study engineering (no improvement, but a ceiling effect). Only gender difference found: Female students thought Racing Academy was more useful to studies than male.

Conclusion: No evidence that video games detrimental to female students; appear to benefit as much as males; found it more motivating.

Big caveat: this is a group who’ve already chosen to do engineering. Hope to do an Open Day study – with females & males not yet committed to study engineering.

Didn’t find a big link between time spent playing and learning.


Jeff: Has electrical engineering background, engineers are fundamentally weird. Don’t know if this would transfer to other majors. Women engineers at Georgia Tech are very small percentage. Engineering students less social, those who stay work that way. Others who didn’t like the atmosphere have left – and those women (and perhaps some men) are a group who may be missing out.

Richard: Agree. This was a group of self-selecting engineering students. Would be interesting to explore:

Jo: Was done in the first year, as part of the induction exercise, so may have had people who were considering dropping out – might have included some ‘normal’ people.

Richard: Is an interesting question. Games are not universally seen as positive.

Anne: Maths, gaming have been connected. Interesting to bring in the design aspect, more creativity. Thought about comparing results between engineering and more creative design – e.g. fashion. Discipline difference might be stronger than gender?

Richard: Yes, would be interesting. Could do it in developmental psychology, designing children (!). Could use principles in other disciplines. Hard to do direct comparison since the games would be so different. Have done same task on different versions of a game, get very strong gender effects. So would expect strong difference.

Martin: Analogy to Pokemon, wouldn’t expect it to be racist, but racial inequality in terms of outcome. Biological model of gender, categories of difference. There are certain things it’s Ok for women to be seen doing; it’s dangerous to assume that a certain category of people can’t do it. But to choose to do it, and be seen to do it, is a different thing. So this group, where they opted-in, is different. Outside that to keep structural model of gender separate. Could find women capable of playing games but not saying they are, talk, justification might be gendered in a way that performance on the game might not be. Not saying biographies have no impact.

Someone: When ask people how much time they spend on games, it’s a self-description, and that’s gendered, so it’s not pure experience, it’s reports of experience. Sense of time might be highly gendered. If you ask game players what games they play they answer in generic categories. But new-to-games people or girls tend to mention titles. May be exactly the same games but represent the experience differently.

Richard: Literature says boys play more than girls:

Someone: A question of how you find out.

Richard: All show the same finding.

Someone: Self-reports are problematic.

Anne: Could be Ok, need to compare log analysis and self-reports.

Richard: Think log analysis would show the effect. Consistently shown in responses from his students. Not sure he’s asking in a way whereby they’d pretend not to be. Can you trust self-reports? Log record might show something different.

Someone: Solution isn’t logging, it’s interesting to see ..

Richard: Is interesting to explore why it’s Ok for males to say they play games in ways it isn’t for females.

Daisy: Had more male participants, wonder why? Isn’t the sample biased?

Richard: Was undergraduate engineering course composition. There is a gender bias. Arguable whether 15 women was enough. Is unbalanced. Had to work with evaluating a real course, but is a methodological problem.

Patrick: Has a daughter and a son, a balanced sample. Changes interesting – son skilled Xbox murderer, but daughter good at games in on the Wii, Club Penguin/online games that she’s spending time on. The marketing/producer people have realised they’ve been under-serving a market. So situation is changing. May have to do it all again.

Richard: Agree. Games important in learning roles. Girls and boys if playing different games will have impact on development, opportunities different. Is interesting.

Someone: Perceived game-playing experience compared with actual game-playing experience. Can collect data on student spending certain number of hours. Players may spend 8h/day, but don’t think so. Need to clarify. May think 8h/day not long because someone else plays much more. Showing off as a factor.

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.

CALRG Conf: Do intrinsically integrated games help or hinder learning?

CALRG conference switches over to a Games-Based Learning Symposium.

First up is Shaaron Ainsworth (and Jake Habgood) from University of Nottingham – Exploring the effectiveness of intrinsic integration in serious games.

Looking at Intrinsic Integration (Kafai 2001) – integrating core mechanics and learning – reject significance of role of fantasy, and in favour of core-mechanics (Salen & Zimmerman 2004) – mechanism by which they make meaningful choice and get meaningful results.

Design guidelines – Habgood et al 2005 – built on this notion. Two things – (1) deliver learning material through the stuff that’s the most fun to play and don’t interrupt ‘flow’; (2) embody learning material within the structure of the world and the interactions – it’s an external repreentation of the learning content, explore it through the core mechanics of game play. Integration of a more motivational/cognitive account of effectiveness of games.

Is Intrinsic Integration better or worse? Evidence hard to come by. Argument for: Flow is in service of learning; we know motivation encourages task persistence (and hence learning). Cognitively, interacting with representational structure helps learning. But against: would it stay in the game? Flow may prevent reflection. Low transfer – because low reflection, mismatch between world and real world, salience on irrelevant-to-learning features (zombies!). Simulations (Goldstone) shows knowledge transfer less for concrete experience. Encouraging learners to play with representations can make it harder to see them as representations, rather than objects.

Ran studies with Zombie Division – national curriculum, to understand that multiplication and division are inverse operations. Compared intrinsic, extrinsic and control versions. Also time-on-task studies comparing intrinsic and extrinsic. Interviews, data mining of game logs.

Iterative design process, paper prototypes, trials.

Games are videoed on YouTube.

IYou are a Greek Hero, and a matrix.

In intrinsic version, you have weapons e.g. sword = 3, gauntlet=2, etc. Zombie skeletons have numbers on their chest – if you have the right numbers you can fight them (if your weapon is a divisor of the zombie’s number), if not you need to run away. Larger ones that ‘divide’ in to smaller ones. Choice of three attacks on each level. Intrinsic game – maths and fun integrated.

In extrinsic version, fight skeletons, but just have symbols on chest which match the weapon – pattern-matching only. With end-of-level boss maths quiz. Control version has no end-of-level maths.

First study – n=59 and 7/8 year-olds. Pre-test, game, reflection (teacher led), game, ost-test, game, delayed test (+3weeks). Two challenge levels – 20 items from the test turned in to a game level as a test of transfer.

Learning outcomes – everyone gets better, and intrinsic significantly better than extrinsic and control, especially on delayed post-test. Gender analysis – no effects. Both conditions did better in the game than the test, even when the questions were the same. Importantly for transfer argument, both groups were about the same level.

Data mining – children reached same level regardless of condition, but in the extrinsic condition they were more accurate. Again no gender effect. Also no relation between game performance and what children learned – all progressed.

Second study

In an after-school club (9-11 year olds) – showed them the two games, let them choose. Only one group. Free switching, 2.5h limit (could drop out). Then group interview.

Spent significantly longer on the intrinsic (p<0.001, r=.89) 61% vs 8%. Girls spent longer than boys 84% vs 50%.

Interviews showed very sophisticated understanding of games – ‘it’s like subliminal advertising with maths’. One student preferred the extrinsic ‘because the teacher would like it’!


Modest but significant advantage for learning outcomes for intrinsic with fixed time on task; huge increase in task persistence.

But don’t know why – could be attention, arousal, affect, better strategies.

Was better, but very costly to develop. Extrinsic can be reapplied in many situations. Teacher-led reflection (1h in the middle) was felt to be crucial to success of Zombie Division, and helped control group.


Richard Joiner: Worked individually in after school club?

Shaaron: Yes. Final interview was whole-group, but they did various things.

Jo: Could they play other games? What else, and how did it compare in terms of time?

Shaaron: Yes. Girls spent 84% of time available on intrinsic game. 80-90% of time was spent on Zombie Division – though was an artificial condition. Probably wouldn’t last for, say, a whole year. But did last for three club sessions.

James: In study 2, had choice between two games, but in study 1 they only saw their own game. Did any in study 1 notice maths being smuggled in to the intrinsic?

Shaaron: Didn’t do interviews so don’t know for sure.

Anne: Is this available? Evaluation, linking gaming to exams, thinking to link to iCMAs?

Shaaron: Background is AI/Ed, never separated learning from assessment, too much time on assessment for its own sake, these are assessment, no need to separate. It’s not just the maths they’re learning, lots of other stuff. Did heavy educational data mining, produced learning curves, to factor out to see how speed-up on maths learning interacts with other learning (e.g. motor skills). Not available online, but will give it out if you ask nicely. Hoping to find a platform to move it on.

Jeff: Are there multiple types of extrinsic?

Shaaron: Yes.

Jeff: Intrinsic has math going on visibly, and extrinsic only visible afterwards. Done in simulations too – e.g. velocity/acceleration adjustments not learned as such. Third type – unrelated but solely as motivator

Shaaron: Don’t like binaries, more a continuum between the two, most things somewhere in between. Intrinsic games/simulations – e.g. football management – not that intrinsic, learning before rather than in – but many more than that. Don’t need to be told you’re learning maths to learn maths.

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.

CALRG Conf: Learning in Public

More liveblogging from CALRG Conference – Tony Hirst on ‘Learning in Public – from Uncourse to Short Course’.

Short course – T151 – started life in an unusual way. Originally mooted 3 years ago. Several false starts, not clear whether it’d be a real course, but started to write it anyway, in a blog environment.

Continue reading “CALRG Conf: Learning in Public”

CALRG Conf: Interactive whiteboards, cameras, action

Alison Twiner et al on Focusing on multimodality to observe meaning-making trajectories, through a LearnPhysical Interactive approach to subject teaching and learning.

Chris Thompson from dance group – with Katie Vernon-Smith – from ‘the place‘ to discover dance. International centre for contemporary, creative dance. Primary schools – KS1 and KS2. Exploratory work in language, literacy – funding from DfES, then other Departments and projects. Using movement to teach across the curriculum, and shift to working with the teachers. Now ended up as LearnPhysical interactive.

Looking inside a multi-stranded teaching model, exploring new ways of teaching with digital equipment and theoretical framework. Children making physical metaphors in a class about materials.

Alison – case study of Year 2 class, 20 pupils, one term, four-week topic on Great Fire of London (history), two lessons a week, one in classroom, on in hall (with dance specialist on the LearnPhysical interactive). Many students with English as Additional Language – to helping build communication.

Sociocultural approach to analysis – teacher’s toolkit (Wertsch 1991.) Multimodal, multisensory.

Interactive whiteboard used to display and work with text, images, graphics, video, simulations. PSPs to capture and review pupils’ physical explorations.

The resources were improvable objects; re-used the timeline – prepared slides, added to during teaching. Also used images from PSPs.

Explored teacher’s planned meaning-making trajectory. Now analysing this and the learner’s trajectories.


Someone: Focus on talk, but about dance?

Alison: Was analysis focus, not the teacher’s. As a means to review the data.

Chris: That was the trajectory of the analysis. This is about the metaphor; it’s a collaborative activity, negotiating meanings and space. They negotiate how they’ll do it, and how to represent it, and then have to explain it afterwards. Talk is intertwined; the physical metaphor – linguistic metaphors grounded in the experience of the body. How we conceptualise space is based in our physical experience, and is embedded in our language. Is very complex – socially, psychologically, cognitively.

Ruslan: Electronic whiteboards – is there something that teachers can’t do on a conventional board?

Alison: Yes. One slide among many, can follow on many, couldn’t mark them all up in advance, don’t have to draw them out every time, can do them in advance. Having the photographs from the students on display, visuals, resizing images and videos. Could do some on a regular whiteboard but most of these easier or more fluid on interactive – and many just not possible otherwise.

Kim: Dance and older students – these are young, fairly uninhibited – harder with older learners, especially boys?

Chris: Fine up to Year 6; not much of a problem in Primary.

Someone: They get used to working with each other, get closer, work well together. Sometimes team teach male/female teachers, which seems to help. Experimenting to Secondary is harder.

Chris: We have short 3 minute video of Y6 boys dancing who’ve worked on this since Y2, very uninhibited.

Jon Rosewell: Robocup Junior, includes dance competition, good at getting boys in up to Y6/7, program robot to dance, dance alongside it.

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.

CALRG Conf: Face to face vs online tuition

John Richardson on Preference, performance and pass rates in white and ethnic minority students.

Has a handout, but only on request. This topic keeps coming round – face to face versus online tuition. Especially around the attainment gap between white students and others.

Asian and Black students are less likely to obtain good degrees (1st, 2:1) than White students – across the HE sector.  At the Open University specifically … over 2002-5, awarded about 21k honours degrees, only 3.7% were from non-White ethnic groups. And trend for non-White students to do worse is very similar to the sector one. Source of effect is the variations in the attainment at the course level. in 2003 133k students taking courses, 6.8% non-white – same pattern in per-course results.

Considerable concern about this, across the sector, and here.

What about face-to-face versus online tuition? Is possible the gap arises from interactions with tutors and other students. Move to online tuition – are there consequences for the attainment gap? Two possibilities: online environment impoverished so less support for underachieving students; or, ethnicity less salient in online communication, so might reduce attainment gap.

Sample: five pairs of courses in Arts and Management – face-to-face and online tuition, but identical curriculum & assessment. 4620 f2f, 1164 online. Original survey about student experience, remarkably little difference found.

Reanalysis of the data to explore the attainment gap.

Why did they choose the mode? Quasi experimental, but no control over which tuition model chosen by student. 85% white, 6% ethnic minorities, 9% unknown (usually expect 1% refuse). No significant difference between ethnicity profiles of students choosing f2f or online tuition. Preference for f2f was mainly (71-75%) preferred it; didn’t know about online (10% ish), don’t have reliable Internet access. Other reason includes need for personal contact or confidence. For online, more like 55% preferred it, >50% was because other commitments prevent attending tutorials; Other (20%+) was around flexibility or disability/chronic illness. Statistically no significant differences – seem to have similar reasons for choosing.

How did they perform? The students’ marks differed significantly across ethnicity – White students outperform others, apart perhaps from Asian students. Variation in marks was broadly similar with f2f and online tuition.

The attainment gap appears to be independent of both discipline and mode of study.

Pass rates – Arts f2f was similar to online; Management pass rate was consistently lower for online. Possibly because of lack of experience of tutors, or priority for study, and so on. Independently, pass rate varied across ethnicity (Black students worst). Variation in pass rates with ethnicity was similar in Arts and Management.

In the end: students from different ethnic groups are qually likely to choose f2f vs online; give similar reasons for choosing; achieve similar marks; and overall pass rates.

Introduction online tuition doesn’t make the gap any worse. (Phew!) But doesn’t make it any better. The mode of tuition does not seem to be involved – but we do need to find out what those factors actually are.


Ruslan: Geographical location of Management students – all UK-based, or some in Europe?

John: Thorny issue. Was all students who took the courses. Students who apply online to take our courses (regardless of mode) are asked the same question about ethnicity as UK-only … categories derived from the 2001 census – and so include ‘White British’ etc. That would be a place to look had he found differences but didn’t.

Kim: Did they come in at the same level of achievement, prior attainment?

John: No. Some had not prior educational qualifications, some had PhDs. Is a big predictor of OU performance, but ethnicity gap exists independent of that. Controlling for prior attainment doesn’t remove the attainment gap. In f2f UK HE, it explains about 50% of the gap. But we don’t collect data in the same way as the other universities, since they get A-level points scores.

Canan: Ethnicity question – White British, White Other?

John: ‘BME’ is often the phrase used, but there are White minority ethnic groups, discourse leaves those out. Are these categories ones that students regard as sensible? Yes for most UK students, No for students from outside the UK.

John: We lost ethnicity data for students when they applied on paper – about 25% just not typed in. Online they have to tick something, only about 1% refuse to tick anything.

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.