CALRG Conf: Do intrinsically integrated games help or hinder learning?

CALRG conference switches over to a Games-Based Learning Symposium.

First up is Shaaron Ainsworth (and Jake Habgood) from University of Nottingham – Exploring the effectiveness of intrinsic integration in serious games.

Looking at Intrinsic Integration (Kafai 2001) – integrating core mechanics and learning – reject significance of role of fantasy, and in favour of core-mechanics (Salen & Zimmerman 2004) – mechanism by which they make meaningful choice and get meaningful results.

Design guidelines – Habgood et al 2005 – built on this notion. Two things – (1) deliver learning material through the stuff that’s the most fun to play and don’t interrupt ‘flow’; (2) embody learning material within the structure of the world and the interactions – it’s an external repreentation of the learning content, explore it through the core mechanics of game play. Integration of a more motivational/cognitive account of effectiveness of games.

Is Intrinsic Integration better or worse? Evidence hard to come by. Argument for: Flow is in service of learning; we know motivation encourages task persistence (and hence learning). Cognitively, interacting with representational structure helps learning. But against: would it stay in the game? Flow may prevent reflection. Low transfer – because low reflection, mismatch between world and real world, salience on irrelevant-to-learning features (zombies!). Simulations (Goldstone) shows knowledge transfer less for concrete experience. Encouraging learners to play with representations can make it harder to see them as representations, rather than objects.

Ran studies with Zombie Division – national curriculum, to understand that multiplication and division are inverse operations. Compared intrinsic, extrinsic and control versions. Also time-on-task studies comparing intrinsic and extrinsic. Interviews, data mining of game logs.

Iterative design process, paper prototypes, trials.

Games are videoed on YouTube.

IYou are a Greek Hero, and a matrix.

In intrinsic version, you have weapons e.g. sword = 3, gauntlet=2, etc. Zombie skeletons have numbers on their chest – if you have the right numbers you can fight them (if your weapon is a divisor of the zombie’s number), if not you need to run away. Larger ones that ‘divide’ in to smaller ones. Choice of three attacks on each level. Intrinsic game – maths and fun integrated.

In extrinsic version, fight skeletons, but just have symbols on chest which match the weapon – pattern-matching only. With end-of-level boss maths quiz. Control version has no end-of-level maths.

First study – n=59 and 7/8 year-olds. Pre-test, game, reflection (teacher led), game, ost-test, game, delayed test (+3weeks). Two challenge levels – 20 items from the test turned in to a game level as a test of transfer.

Learning outcomes – everyone gets better, and intrinsic significantly better than extrinsic and control, especially on delayed post-test. Gender analysis – no effects. Both conditions did better in the game than the test, even when the questions were the same. Importantly for transfer argument, both groups were about the same level.

Data mining – children reached same level regardless of condition, but in the extrinsic condition they were more accurate. Again no gender effect. Also no relation between game performance and what children learned – all progressed.

Second study

In an after-school club (9-11 year olds) – showed them the two games, let them choose. Only one group. Free switching, 2.5h limit (could drop out). Then group interview.

Spent significantly longer on the intrinsic (p<0.001, r=.89) 61% vs 8%. Girls spent longer than boys 84% vs 50%.

Interviews showed very sophisticated understanding of games – ‘it’s like subliminal advertising with maths’. One student preferred the extrinsic ‘because the teacher would like it’!

Overall

Modest but significant advantage for learning outcomes for intrinsic with fixed time on task; huge increase in task persistence.

But don’t know why – could be attention, arousal, affect, better strategies.

Was better, but very costly to develop. Extrinsic can be reapplied in many situations. Teacher-led reflection (1h in the middle) was felt to be crucial to success of Zombie Division, and helped control group.

Questions

Richard Joiner: Worked individually in after school club?

Shaaron: Yes. Final interview was whole-group, but they did various things.

Jo: Could they play other games? What else, and how did it compare in terms of time?

Shaaron: Yes. Girls spent 84% of time available on intrinsic game. 80-90% of time was spent on Zombie Division – though was an artificial condition. Probably wouldn’t last for, say, a whole year. But did last for three club sessions.

James: In study 2, had choice between two games, but in study 1 they only saw their own game. Did any in study 1 notice maths being smuggled in to the intrinsic?

Shaaron: Didn’t do interviews so don’t know for sure.

Anne: Is this available? Evaluation, linking gaming to exams, thinking to link to iCMAs?

Shaaron: Background is AI/Ed, never separated learning from assessment, too much time on assessment for its own sake, these are assessment, no need to separate. It’s not just the maths they’re learning, lots of other stuff. Did heavy educational data mining, produced learning curves, to factor out to see how speed-up on maths learning interacts with other learning (e.g. motor skills). Not available online, but will give it out if you ask nicely. Hoping to find a platform to move it on.

Jeff: Are there multiple types of extrinsic?

Shaaron: Yes.

Jeff: Intrinsic has math going on visibly, and extrinsic only visible afterwards. Done in simulations too – e.g. velocity/acceleration adjustments not learned as such. Third type – unrelated but solely as motivator

Shaaron: Don’t like binaries, more a continuum between the two, most things somewhere in between. Intrinsic games/simulations – e.g. football management – not that intrinsic, learning before rather than in – but many more than that. Don’t need to be told you’re learning maths to learn maths.


This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.

Advertisements

Author: dougclow

Academic in the Institute of Educational Technology, the Open University, UK. Interested in technology-enhanced learning and learning analytics.