The Open University’s Computers and Learning Research Group (CALRG) is 40 years old! To celebrate, we’re having a one-day conference, on Friday 19 October 2018, in the Berrill Lecture Theatre, The Open University, Walton Hall, Milton Keynes. These are my liveblog notes.

Welcome & Introductions

Eileen welcomes everyone and says she’s acting as air hostess, pointing out the exits and so on. She’s had many nice messages. When you look at the collection of old photographs outside, if you want to contribute to the yellow notebook, please do. The goody bags include 10p off coffee!

Mary Kellett – Acting Vice Chancellor, OU

Mary appears in a recorded video. She welcomes everyone. The CALRG has a special place in the history of the OU. In 1978, the OU was still young, and only in its 8th year. From the outset, the potential of the computer to enrich our students’ experience was recognised. [And then there was a technical hitch that made the video stutter and freeze.] The OU is the C20th’s greatest educational experiment, so an ideal testbed for the judicious and imaginative use of computer to help our students. A strong international reputation for international excellence has persisted and flourished. A large cohort of PhD students is crucial, so a special welcome to current and past students who’ve come back today. The group has an enviable record of external funding, well-cited publications, and contributions to the success of the university in research assessment exercises. Their unique position comes from being in a university whose raison d’etre is around using technology in education. Congratulations and have a wonderful day!

[There was some informed discussion in the crowd about the nature of the problem, and the irony of this group being disrupted.]

Hazel Rymer – Pro-Vice-Chancellor, OU

Welcome everyone to this 40th year of CALRG activity! In the room and online. We’re celebrating the story of comimtment, sheer bloody-mindedness, to developing the reputation of the OU for world-leading educational technology. It rests on current and former members who have spread the word nationally and internationally. Welcome particularly our keynotes – Prof Diana Laurillard and Prof Neil Mercer. The group was started a cross-university group of staff, led by IET. First director and founder of the group was Prof Sir Tim O’Shea. Fun stats: over 100 PhDs awarded, at least 10 former students hold professorships, including the OU’s only Regius Professor (and the UK’s only one in education). Excellent results in five research exercises. Not (just) about looking back, but forward. The programme is about visions that underlay the group’s work. Let’s get on with it!

Eileen Scanlon – Regius Professor of Open Education

Thanks for the kind words. Forty seemed quite special, so we were a little more ambitious. We’re looking at work that has persisted over 40 years. We have time for four visions today, but there are a lot more than that. I do love the quote from Voltaire – history is a pack of tricks you play on the dead. We’re playing a trick in trying to retrofit some group visions. You don’t have to agree with us about them, but it’s an attempt to link the past, present and future of the group. We’re hoping it’ll be interactive, and where you’ll disagree and be quite forceful. Tim O’Shea and Ann Jones are the only two from the founding group who are here. Tim, in his interview at the OU, was recruited to work with the Maths faculty, but was told not to do any of the educational computing he was famous for. Within a year we had CALRG. Hazel mentioned bloody-mindedness, it’s a great trait across the group. There’s also the sense of community, of sharing endeavours. Students are the key glue that helps a research group. One common acknowledgement in theses was thanking members of the research group, “past, present and future”.

Four visions, located within the OU.

• Vision 1 – Learning is accessible for everyone. A strong line here. Not just for people with disabilities. Openess, inclusion.
• Vision 2 – Adapting teaching.
• Vision 3 – Learning at scale. The biggest course the OU had was about 15,000 students. So this notion of scale and reach is one we can explore.
• Vision 4 – Science learning. Where Eileen herself started. “Teachers will bite your hand off, Eileen”.

What matters? Many varieties of openness. Transparency and collaboration are key indicators of the health of a group. This has been a cross-university collaboration, led by IET. We make a point of being open, transparent and collaborative.

It’s more than sheer bloody-mindedness. Background to the visions. In 2013, a cross-university group, worked on “Beyond Prototypes” (beyondprototypes.com), an in-depth examination of the processes of innovation in technology-enhanced learning. What makes a difference that lasts? Many case studies, looking at interviews with researchers and industry.

Found three things, key factors: persistent intent working through successive projects and and understanding of the complexity of the infrastructure round Technology Enhanced Learning (the TEL complex) and the process of bricolage. Bricolage is my favourite, it’s something like tinkering, the work you do is influenced by the tools available, it’s of its time – we push the cutting edge – but some of the choices are to do with timeliness in terms of the technology. It sounds a bit theoretical, which I’m not famous for. But in an in-flight magazine I found a definition of bricolage, so thought maybe it’s OK to use.

We’re going to try to produce a book, using persistent intent and bricolage as a frame on which to hang the description of our work.

It’s been exciting, but complex. Demands collaboration. Bricolage sounds like tinkering, creating tools that are no use in the future – but no, one of the key things in this area, even when the product you produce is of its time, the fact that you made your teaching ideas explicit, inspectable and testable – whether RCTs or more qualitatively – that’s what makes the area exciting. Make your ideas concrete, and people can inspect them. There isn’t a distinctive methodological approach in the group, which can be a problem, but so can sticking to one. Studying in teams that are interdisciplinary is part of the challenge.

Finally, a plug for OpenTEL. The CAL group is great, but I have to thank another group, which extends a little further, a priority research area, called openTEL. Has a lot of the features and people of CALRG, but draws even more people in to our work. A link between research and practice – the synergy between TEL research, and the mission and strategic interests of the University. There’s also our OpenAI group, and others. The openTEL group has funded the reception at the end of the day, so thanks to them for that.

Thank you.

Eileen then welcomes Diana. Her contribution to the OU and HE and ed tech policy is huge. We’re delighted she’s back. We used to pack CALRG in to a room and have tea and cake every day.

Keynote – Diana Laurillard

Reflecting on the CALRG vision and its diaspora

Chair of Learning with Digital Technologies, UCL Knowledge Lab.

Thank you for inviting me back. It was such fun working here. Wishing you a happy birthday. 40 years! That was 1978!

An image to reflect on that, and what it was like, and it was this. A British Council visit to Yugoslavia, a conference on ed tech, I was to do a presentation on interactive graphics for computer simulations to teach science. I knew Tim and Eileen’s work. The idea was to take this to Belgrade. The technology hadn’t caught up with our ambitions. I took with me a microcomputer consisting of three enormous boxes. One was the computer, one was the screen, a gigantic TV with a CRT, and another with keyboard and other guff. The plane landed in Belgrade, I watched the luggage trolley with my boxes on top, and I worried it would fall off. Customs were suspicious, even more when I said it was a computer. They got more suspicious and feel on it with knives until someone from the BC explained what was going. You couldn’t show this with an OHP. You had to be bloody-minded to get your point across. Shows how foresighted Tim and Eileen were to setting this up.

I’m not a good digital archaeologist, rooting through my old CDs, it was hard to find stuff to characterise CALRG. The papers for the CALRG conference, I can compare by looking at CALRG papers up to 1995. Two word clouds. 40 years ago, it was the subject areas that dominated – science, music, etc. But today they don’t appear at all. Learning and education dominate, students loom large, and MOOCs replace intelligent systems. They’ve become the focus of my research, although I was skeptical. There were stupid things said about MOOCs, cruel myths. But if we as researchers explore and challenge what we can do, we get something really interesting.

Reflecting on those four visions, and how they are worked on here, illustrated by some things from the UCL Knowledge Lab.

Learning is accessible to everyone.

This makes me think about the UNESCO vision of learning for all. This requires something different to what we’ve been doing. We have to use learning technology to develop high quality HE on the large scale. The sustainable development goals – SDG4 requires 68m teachers by 2030. But technology can help. Teaching skills for digital age – government panic about AI taking people’s jobs, asking what can we do – one of the first thing is to help the teachers who are helping the students, continual change and innovation.

More online models to reach more key audiences? Transnational education, that’s ok for a few hundred thousand students who are well off, but to get to millions it needs online learning. British Council sees that too. TNE demand is currently 4m, but not commensurate with the 200m global demand.

Need to understand relationships between open universities and MOOCs. OU is lamentably undercelebrated within government. Other countries have copied what we’ve done here. Proven for undergraduates. It’s not free, but you get personal tutor support, get tutor discussion, feedback, assessment, and summative assessment. But MOOCs are not suitable for undergraduates – it’s free so no personal tutor support, relies on peer discussion, feedback, assessment. But it is highly suitable for professional development. Teachers value this very highly. They trust their peers, the discussions are vibrant because they know what they’re talking about. You don’t expect and exam and qualification from professional development. Researchers value the pathway to impact on end-users. MOOCs have a role to play.

It’s a two-step cascade model. In the development context, there’s a cascade model – run work for experts and teachers at national level, then a long cascade where there ends up with a pale reflection at the end. But from a MOOC with 10,000 professional participants, if each of those trains 25 local students or adults, that’s 250,000, via blended learning. That’s how we can get to millions.

Teaching is adapted to meet learner’s needs

We developed MOOCs on FutureLearn on blended learning essentials, funding from UFI charitable trust. Running for 2y, 25,000 teachers engage (double that registered). 60% in the UK. The most successful prof dev in the FE sector ever. Has brought in HE teachers too. A way to get to teachers, they get a window on others’ practice. Access to tools and resources, talk to each other – they’re really cooperative and supportive. We learn a lot too. We can take this further e.g. with Padlet, sharing practice. We can explore the real contexts of our learners, which is incredibly important for the research.

Digital competence, technology-enhanced teaching for teachers, a different project. Progressing Technology-Enhanced Teaching – MENTEP. To enhance the self-assessment tool, we created this MOOC, with a form to submit their learning design and how that matched to the competence framework. Again, teachers share what they’re doing, and contribute back. Teachers are isolated in their innovation, but on these platforms they can share. Collaborative knowledge-building, the entire workforce needs to be engage.

Then in research contexts, we have a project looking at refugee settlements, in Lebanon. RELIEF project (Refugees, Education, Learning, IT and Enterprise for the Future). The project us using MOOCs to develop community researchers and teachers professional development. Work with them, what they really need. They said classroom management, so we’re starting with that. This looks like it will work in these challenging refugee camps. The wifi goes down for a few hours a day, but you learn to work around it.

The co-design approach means to engage, co-design, blend, embed it in how they already operate, and work it so it becomes self-sustaining, because we will fade away. That has to keep iterating.

So this is when teachers get support to develop the community knowledge of how best to use computers in learning.

Teams can successfully teach any number of students at a distance

Team of researchers create a MOOC with 10k local professionals, each of them works with 25, those 250,000 people engage – that enables working with them, not dropping stuff on them out of a helicopter.

Some tools to do this – one is the Course Resource Appraisal Modeller (CRAM). Helps people understand what they need to go online. The cost models are different – large initial outlay, but amortised over many years of teaching.

TPD@Scale Coalition – many moved. We hope there is much help for teachers out of this collaboration. Reports often say that teachers or teacher education should do this or that, but rarely much on how they will.

Learners engage enthusiastically with science learning

Yes they do! Not just the teachers bite your hand off, the learners do too. We’ll here more about that through the day.

The CALRG diaspora

It must be huge by now. I want to end with a rather self-aggrandising claim. Two incontrovertible statements, and a conclusion.

There is no clearer force for good in the world than education.

There is no way to provide education for all except through digital technologies.

Therefore, educational technologists are doing the most important job in the world.

Questions

Rose Luckin – Two questions. A slide at the beginning, and near the end. The outside world that we have to cope with. The slide that showed how things had changed, from disciplines to learning. The curriculum doesn’t change, it’s still disciplinary. We’ve changed but it hasn’t. The second is to do with funding. So many initiatives that are really good, do engage in participatory design, still flounder in that transition to scale. Struggle to see how we impact more. How can we do more of that? Second is not related. At what point do we as researchers admit that we’re wrong. It’s hard to publish things that haven’t worked. It’s difficult to accept when we haven’t got things right. Is there a contradiction as researchers?

Diana – that was at least three questions. [laughter] Curriculum – I don’t agree, I think it has agreed. [I was thinking of schools] In university it has changed. In schools politicians get in the way, we get inappropriate things demanded. How we affect those things is part of what we do in co-design, a stakeholder mapping. In change in education that’s always the education ministry. In Lebanon, we start with the guys at the top, and they are all guys, then we work down to the people who do things, we work with all of them. We need them in the MOOCs as well. The co-design works with all these different groups and sometimes mixes them up. When you hear the stories about what it takes to do anything in the camps, the people from the ministry learn something they don’t know. Sustainability is absolutely critical. Our blended learning essentials we started on day 1 working with government and agencies to take it over, wouldn’t cost much to keep it running. They came back with it’s not in alignment with our strategic plan. Unbelievable! What can you do? I spent 3y in the Dept for Education and Skills. No point using government to make viable change. To make things work, had to go back to academe and work with the teachers. They know stuff, they’re committed. That’s the best I can offer.

Ray Ison – On the viability model, I’d challenge you, the reason it’s working is the FL platform provides innovative space because it’s not settled. Why aren’t we as the OU doing what you’re doing ourselves? That’s taking a design turn, absolutely the way we have to do it. Are we not being blinkered by our own adherence to the orthodoxy of named degrees and education meeting prescribed means from others.

Diana – that’s what universities have to be, the market creators of new knowledge, that’s why research is so important. The OU has to take credit for the existence for the OU. How you use it is up to imagination. On co-design, the professionals know stuff we don’t. A perfect design cycle. With younger students it’s more difficult. Different with students than professionals.

Vision 1 – Learning is Accessible for Everyone

Martin Weller & Kate Lister

Kate starts. Four parts – Martin with a video on the Revenge of Open. Then Eileen on remote labs, me on inclusive practice, then all of you engaged. Four themese – making inaccessible.

Martin – Revenge of Open

Starts with a Star Wars crawl. We should embrace the woolliness in the ideas of open access – it enables us.

Different models of open access, illustrated with hats.

He starts with a horned helmet. The traditional model. Open entry, distance education, modular, part time. They’re radical. Ed tech startups seem to emulate it. Like the Vikings, hence the hat, it swept the world.

Next a white curly wig. Another interpretation is boosted by the Internet, Open Educational Researchers. People can do new things with it, as well as accessing it. It’s a legal (hence the wig) as much as anothe process.

Then a yellow builder’s hat. MOOCs. The cost is seen as a barrier, but limitations on rights we didn’t see with the previous. The cost of removing the cost barrier is a loss of support. Constructing barriers.

Then a face mask with respirator. (Chemist) This is mixing everything all together.

Finally, (with no mask) open access takes many different forms. We shouldn’t think of open access as one thing, but as a group of related practices. We should constantly review these, always refine what it means to be educators and an open university.

Eileen – Remote Laboratories and Remote Access

We actually haven’t moved on from thinking about science. Past work – Practical Experimentation by Access to Remote Learning (PEARL) project, Enabling Remote Access, OpenSTEM Lab.

Acknowledgements – small list – many people involved.

How does the introduction of technology change science learning?

There’s a practical problem with practical work in science. Home experiment kits – lasers in the post – or, as the Daily Mail had it, Death Rays Through The Post.

Martyn Cooper, PEARL project, EU Framework V 2000-2003. Providing access at a distance. How to turn some of our introductory science courses in to something people can experience at home or elsewhere. Other partners did manufacturing engineering and electronics. But at the OU, took the physics experiment from our programme – spectroscopy – previously had to build your own at summer school. If you think through how to deliver it at a distance, you improve it for everyone. Helps accessibility and inclusion. From a distance, use a robotic spectroscope set up here in a lab, could do work they would’ve done had they not been unable to attend summer school. This was a complex task.

That is a forerunner of a lot of the work very successfully developed by the Science Faculty, now the STEM Faculty, with a grant from the Wolfson Foundation. History – home experiment kits and broadcasts, residential lab-based classes, on-screen interactive experiments and instruments, livestream experiments, remotely operable analytical instruments.

Another project, led by Trevor Collins, trying to make geology fieldwork. ERA – Enabling remote geology fieldwork by transient wireless networking. Opening practical science for everyone. Persistent intent. Ancient Mountains course (SXR339), came from a course team query about setting up an alternative experience for learners who couldn’t attend. Started in what geologists need to see. Developed idea of the remote activities model. Students brought as close as they could be, but communicating with others. Currently this is working in Access Anglesey. In 2017 had an accessible field trip funded by the NSF, 2014, 3.5d field course on hydrology and meteorology in the field.

Out There and In Here – led by Anne Adams. Cross-university team. EPSRC funded project. Collaborative teams working in a mixed environment, some ‘in here’ working in a command centre, and others out in the field – ‘out there’ – who send and view images. Bringing lots of information together from different sources.

These are examples of the TEL Complex – you can look at what is going to be possible with near-future technology. Persistent intent, bricolage – but also interdisciplinarity. This was like that.

Kate Lister – Evolving inclusive practice

How we’re applying the lessons we’ve learned to our practice.

Another project – EU4All. Who’s heard of this? [most people] Large EU project, 13 partners. Wide stakeholder engagements. Two outputs are a model of professionalism in accessibility, and a learner-centred framework for personalisation f content and service. Three lessons: accessibility requires multi-faceted, multi-stakeholder approach; accessibility is continually evolved – persistent intent is required; understanding user needs, experience and preferences is key.

Three examples where we’re taking thees lessons and applying them.

SeGA – who’s heard of this? [almost everyone] A small team, in LTI, with a goal of embedding accessibility. We connect people to a wider community, multi-stakeholder approach. Has been around for 8y, has revolutionised how accessibility is operationalised at the OU, innovative in the sector.

Second example – the IncSTEM project, led by Trevor Collins. Aim to scale up examples of inclusive practice to the HE sector. 8 OU case studies/mini projects. It’s a huge project. Will take the example of the online practical work, the OpenSTEM labs. The virtual microscope is one of the examples. Can look at an object (e.g. a Martian meteorite), in various orientations, and in the microscope. Working up practical previews – a chance for students to try the tools off-module. A professional demo, then hands on with expert support, time to ID and address accessibility issues, and time to ID and address tech/skills gaps.

Third – understanding students, ‘Our Journey’. Students have diverse backgrounds and goals, but hard to evidence the barriers at scale. Collaborative development. Represent student journeys from the student’s perspective. Shows how various aspects impact on their learning. Can be used online or face to face. Data at scale or guided reflection.

Themes covered: making the inaccessible accessible; making things better for everyone; designing for diversity; breaking down barriers.

The future

Take 3-4 minutes, talk to those around you, pick one of these themes, ID issues you want to see on the research agenda.

[discussion]

Tim – Everyone talked about scalability. That ought to be on the agenda. Eileen’s thing with the rocks, brings to mind many games where people are in virtual environments, or literally running round the streets of Dundee. One of the places to learn from is the big successful multi-user computer games.

Someone – broadening term of accessibility, to include English learners, people from diverse backgrounds, making learning content accessible to them.

Do feel free to communicate more ideas to us!

Vision 2 – Teaching is Adapted to Meet Learner’s Needs

Ann Jones & Bart Rienties

Ann starts. As one of the original members of the CAL group, I thought I’d go back to seek a framework looking at the early days, in a report from the first annual conference in 1981. Then Bart will talk about more recent work.

I’ll talk about three themes in the report – Models of learning, method for studying learning, and institutional research: evaluation.

A photo from 1979, with the CALRG sitting on the grass, in Cambridge, in the garden of APU.

Models of learning

Some were interested in developing production systems – for psychological modelling, computer programs consisting of a set of rules, ways of deciding between different rules, and an interpreter to run the system. A focus on collecting student protocol data, trying to understand what’s going on – but some of us didn’t progress to production systems. There was a strong relationship to supporting student learning, and a range of domains – maths, physics problem solving, and novices learning to program – my [Ann’s] interest.

My thesis work focused on novices learning programming. Has her report on ‘SOLO’. Participants came to the university, worked through the materials, on their own, then recorded talking aloud about what they were doing.

She showed some transcript data from a student talking about how they had gone about a programming task. Some of the more interesting examples are when they have difficulty.

Feeding in to teaching – it fed in to how the design of instructional materials supported understanding. CAL was also used in courses. Tim developed a computer game, used in a mathematical course – developing mathematical thinking (EM235).

Some of our systems were pretty advanced. The CYCLOPs system developed at the OU could capture written input as well.

Institutional research – we evaluated 3 tutorial CAL systems in Science and Social Science courses. (in the early 80s). Some used at summer school. Using questionnaires, observations at summer schools, interviews, automatically recorded usage, and student-completed log books.

Case study – Evaluating tutorial CAL. MCQs answered at home, went to study centre to access a terminal, would get further questions to answer which were more interactive. Use was lower than hoped, and dropped over time. Questionnaires were built into the tutorial. Overall, there were barriers to negotiate. Students had little knowledge of computers – hard to get on system. Other students around, some found it an anxious experience. Tradeoff of perceived benefit and hassle.

Another case study – work metallurgist – Canan Blake et al 1999. Evaluating another tutorial, this on in a game format, interpretations of phase diagrams, used at residential school. It worked well, most students bought a copy. Designed for individual use, but tended to use it in pairs.

Reflections on then and now. The drivers – understanding student behaviour, improving instructional design – are very similar. But in our evaluations we were looking at one part of the course (CAL), because we could change that. Research students were crucial part of group’s work, as now.

Bart – back to the future

Technology portrayal can be positive or negative.

Bart does a quick poll of those in the room – “In comparison to 1979, teaching at the OU is now adapted to the needs of students”. Most are neutral, but a just under half agreed or entirely agreed.

Another one: “With the affordances of analytics and AI, within five years teaching at the OU will be adapted to the needs of students – 36% agree, more neutral, quite a few disagreeing.

We’ll do a pre-post test.

OU Learning Design framework. And how the way we teach has an impact on student satisfaction and retention. Communication is the best predictor for engagement, and for whether they pass the module. Since 2016, have done amazing work on identifying what teachers are actually designing. Quan Nguyen took this further, mapping what the students are actually doing (on the VLE) and how active they are, and how that links to the learning design. You see linkages, and disconnects. Also, how we teach courses at the OU are widely different. Some are peaks and troughs, some are steady all the way. 69% of what students are doing in a week is determined by us, teachers.

Can also see when our students are actually studying. Compare successful to failing students. The vast majority students do not follow the course structure! [In the sense of what activity is done at what time. Some very clear trajectories. Also analysing what students are saying, we can predict emotions – positive, negative, neutral, mixed – just on writing. Our tool is much better than the other tools available.

So, we can look at what effective teachers do, look at what the best paths for students may be, understand what students ‘think’ and feel, and provide personalised feedback what to do next. But are we ready as OU to do this?

Return to the questions. Who would change their answer to the 1979 to now – not much change, but not much change. Perhaps a broader spread. Then will it be in five years?

Final question – what are you hoping that CALRG/technology will bring in 5 years to the OU?

Some suggestions: Design for diversity, replicability, larger inclusion of student-centred design, happy and fulfilled students, robust meaninful analysis of big learning analytics data, get researdh into practice, understanding students’ needs at scale, better evidence bout our treasured theories of learning, personalised, more active learning for time-poor students, research informed new regulations.

Discussion:

Stylianos – I see we have a better way of collecting big data, making meaningful assumptions about students experience of learning. We have gained a lot, but have we lost anything? Ann’s work on close scrutiny, precise social science methods, compared to big data.

Bart – yes. That’s why I like CALRG, we need interdisciplinary researchers. If the learning analytics people were in charge, you wouldn’t want to live in that society, because our view is so limited. You need that triangulation of perspectives. We had 40y of research, but lots of the problems were known but not empirically proven. Now we can show 69% of what students done is determined by us as teachers. We should get our skates on.

Ann – We do still bring in students to try to get a fine-grained view of what they’re doing, interacting with systems, or students with disabilities. We haven’t lost that.

Tim – When Ann and I looked at students using computer system Cicero, we were surprised. BCE – Bad Computer Experiences. e.g. the porter wouldn’t let you in, or you couldn’t get it to work. Another was emotional – feeling others watching you, or the computer would inform your tutor you didn’t really understand. That was helpful for the academic computing service of the time, which explained why usage was dropping off. All of this is sunny uplands. Is there not a proportion of students for whom the modality is adverse. Assumption is this is great for everybody. Is it?

Bart – You’re right. We didn’t show OU Analyse. We couldn’t understand, some students never clicked on anything and still passed. How is that possible, they’re not doing what we expect? But turned out they were in secure environments and couldn’t click. Vast majority of students are not following the course structure, but we focus on it.

Tim – Methodological problem, still around. Students are instrumental. Ask them why did you do something, they’ll answer. If you ask them why did you not do something, they’ll say they were too busy. Won’t say because thought might make a fool of myself, etc. How do you get underneath that to the actual reason?

Ann – We have some students who do not want to engage online. Some are on machines all day, last thing they want is to do that in the evening. A challenge for us.

Andrew Ravenscroft – Following on from Tim. LA, the problem is there may be crucial factors you can’t capture. We measure what we can easily find. Would you capture a moment of insight? Probably not. A cautionary frame round this. A lot of work on LA is less advanced on how you intervene to make things better. Can see students who may fail, too much attention on that and less on what we do to intervene. I have students who are aware of the difficulties, but getting them to improve their performance is really difficult. We need to focus on those interventions.

Bart – Has to be actionable feedback. Giving the feedback to teachers helps them give feedback. Giving feedback to students can help them if – and a big if – they are able and willing. So how can we provide really well designed activities where they’re willing to engage with our feedback. Maybe we should think about how we collect feedback, so we know before the feedback. We can do so much better at the OU than a one size fits all solution. We have the power in the research community to help.

–
This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.