LAK13: Thursday afternoon (10) Panel on trends

More liveblogging from LAK13 conference – Thursday afternoon.

Panel: Recent and Desired Future Trends in Learning Analytics Research

Liina-Maria Munari (European Commission), Myles Danson & Sheila MacNeill (JISC), John Doove (SURF), Sander Latour (SURFSIG LA)

Erik Duval is chairing. Welcomes everyone. Aim to have dialogue between funders and researchers. Series of ten-minute presentations from the funders on the panel. Focus on LA research. Then discussion and questions. Better understanding is better for all sides.

Liina Munari (European Commission)

One of the project officers (=program manager in US terms). EU funding research in technology-enhanced learning TEL. EU slides traditionally full of text and boring. Have open call on LA closing on Tuesday – probably 90% of the Europeans in the audience are involved – but not talking about that.

Our current programme ends at the end of this year. Entire EU is in the process of drafting the next one – won’t be a Framework Programme but will be Horizon 2020.

European Commission funds FP7, CIP, think of in Brussels, but their unit is based in Luxembourg. Fund research on learning-related technologies, FP7, FP6, FP5 – 20 years. We’ve done it all, workplace, school, universities, networks of excellence. STELLAR, many you’ve heard of, we’re proud of them, we’ve pushed forward research in this field. There was a huge hype about what tech can do, practitioners not convinced, have kept it on the agenda to know and hope it’ll be included in the next 7y programme.

Funded 44 projects, EUR 185m total. Learning analytics is about EUR 6.6m, in the current call. The rest is something else. Went back to project file – Juxtalearn, WeSPOT, Go-LAB – which do have some learning analytics as a small part – from 2011-2012 call. Only calling it learning analytics very recently.

Commission is a large institution, other units fund other technology. What about neighbours who fund Big Data? One project – Linking web data for Education – a support action.

Data Value Chain vision – creating social and economic added value based on the intelligent use, management and re-use of data sources in Europe. This is LA! Big data ‘frenzy’. Focused services, drawing on big analytics, on fast data. All about speed and scale. What’s big data? When the size of the data is part of the problem, or large enough tat it cannot be processed using conventional methods. Do we have that in education? We have a boutique problem. If we want to convince the big data community, we have to show them the big data … but it doesn’t come out unless you use more technology for learning. Chicken and egg problem. Where do we position ourselves? Big data? Or very specific with links? Volume, Velocity, Variety.

Good if standards mature, and if there is sufficient interest and funding from national and state governments … need to define the location, and argue the value of this research.

Erik: My first job in 1989 was funded by the European Commission. We’re involved in some of those positions.

John Doove, SURF

I’m a project manager, responsible for activities including learning analytics.

SURF’s not a research funder. But a collaborative organisation for innovation. Only fund projects in HE in the NL, and they’re not research – more practical application. Want to improve quality of HE and research. Learning analytics important.

Their focus is transfer from research to practice, and policy in to practice. (Less interested in Policy to Research). And transferring good practice more widely, at national level. Special Interest Groups, people who are interested in a theme and want to advance it. The QR tags take you to their website about the LA SIG. They also inform policy, trying to influence directors and boards.

We like SoLAR. It’s a research community, but with an eye for putting it in to practice, and getting that relationship to policymakers.

Fund not just projects, but programmes. Find and ID gaps in knowledge they need.

Concrete example – one building block is experimenting, developing and encouraging use. Did that with 7 projects, only EUR 10k each. Interesting results. They were allowed to fail. They don’t have to get significant research results for a paper, or a major success story. Can fail, so long as it’s responsible and others can learn from it. There’s a video to show them.

Started with research expertise. So have an Expert Group: Wil van der Aalst, Hendrik Drachsler, Erik Duval, Paul Kirschner, Ton de Jong. Also reviewed results afterwards.

Research projects that would make them happy:

• Research driven by a question from educational practice – not a solution, or a dataset; practical applicability important
• Ongoing exchange between practical experience/validation and research focus – iterations, each iteration is valuable
• LA is multidisciplinary topic, strong proposals involve those multiple disciplines.
• Stakeholder involvement (service providers, users) throughout the process; vendors have picked up on at-risk identification, but little more comes out, not looking in to the learning process.
• Impact analysis – helps big time in reporting back to policy and decision makers
• Research that is transferrable, reproducible and broadly applicable as possible
• Research that makes use of related educational fields – not LA on an island. Digital assessment, adaptive learning, OER, MOOCs, etc.

Myles Danson (JISC)

Myles slides on Slideshare

JISC is similar to SURF, but in the UK. A bit bigger. We’ve funded about 1400 projects, all underpinned by technology. I’m a programme manager. Last year 40 projects in his purview.

JISC is a shared service across UK universities. We run the network that connects them together and to the internet – SuperJANET – fastest in the world. Couldn’t do that individually. Also content services – negotiate on behalf of the sector, cheaper. Then services helping with uptake and embedding, like John. Finally, the innovation group, where I am.

Within my 40 projects, looking at relationship management, from pre-admission to alumni. Had 4 projects looking at learning analytics without having namechecked them, finished last year. Had a piece on Business Intelligence. GBP 50k over 6 months. Two drifted in to LA.

Have also worked on activity data, engagement analytics. All similar fields; you get this. Got about 50k, Sheila did the JISC CETIS Analytics Series – big breadth, includes learning analytics. All this stuff available for free.

Early characteristics of the field:

• Newish field
• Beacons of excellence
• Narrow applications
• Promise of great things
• Little coordination of effort
• Little evidence of business cases (Myles special interest)
• Reliance on the implicit – articulate the benefits, bring them to the fore, quantifiable ones – useful for demonstrating need for funding
• More holes than net – Sheila’s work uncovered, much that could be done
• New terminology, be careful, can bamboozle & confuse people you want to influence
• New roles – data scientist vs put the right team together, more fluid
• Intra community excitement – lot of buzz, the SoLAR, LAK, good job
• Extra community confusion – publicity pieces, in the Times Higher, jargon-free but trying to lift the field so important folks get a grasp

Many opportunities. Early adopters, grassroots interventions, nurturing activity. Strong case for peer support – conference a good examples, workshops, SIGs, CoPs, Shared problem identification and solving. Educause, JISC and SURF have had videoconferences to coordinate internationally.

Challenge: Important part is the institutions and sector stakeholders. Requirements gathering/horizon scanning. Creating, updating and testing – lots of that, the research this conference is focusing on. Don’t see much synthesis and knowledge creation, could work on that. Which means not enough solutions, advice and guidance to go back to the institutions and sector stakeholders. JISC’s role is about embedding, changing practice, within institutions – uptake stuff. This is a cycle, it’s iterative.

Business Intelligence – roadmap for doing it, 1-6, from fragmented sources to reliable predictions. Use maturity frameworks, adapt, look at organisational maturity around analytics. Work through representative bodies. Shoot high – senior management team, policy, governance.

Need to show beneficiaries, and reality of benefit in the timescale, reality of sustaining the outputs, value to the sectore, and innovativeness.

Same benefits from BI can be demonstrated by LA projects. (Has a long list – see Slideshare above.)

Summary

Innovation/embedding cycle. Focus on benefits. Keen on co-design. Business case. Policy and governance; readiness issues (digital literacies). Keep up grassroots innovation!

Discussion

Chris: Interested in international partners that are working in synergy, through SoLAR. Would love to work with Simon, he could get JISC funding. How do we show our synergies could benefit?

Myles: That’d fall in to the evaluation criteria, track record and make up of the team. If Simon applying to JISC, benefits go to the UK, but if you’re adding the value through team makeup, expertise.

John: Easy way, I agree with Myles. It’s about the added value. If you deliver value, that’d be a good way to go. We are funders for the Dutch situation, but we do care about other places. If working in Canada, we’d evaluate that. Added value.

Liina: We’re a bit more generous. We can’t directly fund Americans, Australians, Japanese, unless you really justify why expertise is not found in Europe. But we have other means, can always be there as a formal partner without asking for funding. In the current call, asking for a coordination and support actions, organising events, networking things – counterparts can help, use the EU money in a more, larger context. We’re international, there to benefit the European Community, but EU research has links, not bound by continent. If in China or Russia we can fund you – ACP countries. But developed ones.

Sheila: There’s a role for community building here with project funding. Events important, maintain connection and open practice. Share our research, cite each other, project proposals cite relevant people we might work with or directly relates. While funding projects, we can’t fund people from the States, but can do community building. Moving the research community in to the practical application.

Sander: The SIG doesn’t get directly involved in funding. But do value collaboration, especially international. The LASI in Stanford, there’ll be in Amsterdam and England, try to get out of national borders and work together. Expertise exchange works through to next funding proposal.

Q: Compare with discussion after lunch, with the US community. One big difference was the venture capitalists, the money from industry in LA. What’s the position in Europe about this.

Liina: Everything we do, we try to boost growth and jobs, if can link it to that. We’re happy for industry partners. We have some instruments, the European Bank of Investment, that’s venture capital. Anything that would attract VCs, maybe the new programme, there’s be more instruments related to this. In TEL, looking for more VC to fund risky interventions. That’s not excluded. But institution sets a pace for a number of years. It should be highlighted if not happy with how things are funded now. EU is very democratic, but whoever shouts the loudest can lobby. Very pro any business initiatives. Even when a research programme.

Sheila: From JISC, we’re agnostic. It’s not part of our culture, we fund institution. There are small companies who’ve worked with our community. Tools developed in tandem, not vendors just coming in with a tool, but can say we need x and y. That could change over next couple of years.

Myles: Projects outlined strong cases to work with software vendors – not VC per se but certainly private sector. Consider benefits to the project, insist through licensing that outputs freely available to the JISC sector (UK HEI). Partnering with companies, suppliers. Obvious benefits – not developing software and then struggling to spin it out, but include in a commercial offer – with caveat to make available.

John: Get the stakeholders involved. Meeting for vendors on LA.

Erik: Great, we can still get funding if we have VC partners.

Myles: Wonder where the loop is from LA research community, going in to best of breed. Market share – Banner big for student record systems, developing their own tool. If influence those vendors, features IDed as valuable from research. Rather than vendors providing the wrong ones.

Erik: We are developing proposals to do that.

Myles: Standards are key, and interoperability.

Sheila: Moving in to a mixed economy. Cloud services, more challenges. Work with vendors differently. Get access to data. Manchester Met, using hosted services, small vendors built them some apps, testing with real users. The way we’re working is developing. Want to inform the development.

Q: Can we promote the work of researchers, on LA, in the CFP on educational practice, lifelong learning, to make innovation work. So we don’t start over again so all the researchers work is on the shelf.

Sander: Be a good community, share the knowledge. Large gap from where researchers stop and practice begins. We should move towards getting tools so teacher can use them. The tools are ready, but when try in practice,  have to read a lot and there’s no time for that. Finding research is one thing, SIG helps. But seeing where gap still lies, aiming at that is important.

Q: That gap is what you want to solve in your project. But lots of people don’t know what’s there. The researchers do not connect very well. How many people are here who are not researchers? What role could they have?

Sander: Role for SURF, JISC and my SIG. We’re trying to do, like SoLAR, have connection places. E.g. SoLAR Evidence Hub. Practical things – LASI in Amsterdam. Week before that, will organise a seminar to inform teachers and where that gap lies. You do that by communicating in the community.

Liina: Communicate among the researchers. Open access to research results. If you do something nice, that makes a difference, communicate, and in lay terms. If you have something nice – to capture the eye of the Commissioner, you need an article in say Times, this EU project, then we hear – did we fund this, can we fund more like this? Don’t forget the old means, press releases, talking to journalists. When in terms that are understandable.

Myles: When put your proposals together, look for a stakeholder map. JISC specifies will mark down no evidence of related work – often does come out in the evaluation process, stop reinvention. In JISC, have programme information management, can search all previous projects, outputs are all there. JISC repository with materials.

Sheila: We have a project db with tools and technologies in projects, outputs and links, completely open. Was Linked Data, may not be any more. Want you to do the research. People always want to reinvent the wheel.

Erik: Not reinventing the wheel. Does that mean you think LA is not old wine in new bags, or do you not think it is? Is it different? We’ve had close to 30y funding for TEL research, why do that?

Sheila: We’re refining the wheel, getting faster and quicker. There are elements of LA that are new and exciting. Common cultural problems in all TEL. Barriers to adoption, nothing to do with the technology, that’s communication. Same message in a different way. LA is a powerful set of tools to affect learning.

Liina: Same thing. Bit of a newbie in this field, but if you can instantly tell teacher or student this will help, we’re closer to closing that gap between adoption and research.

Sander: What makes LA slightly different, it’s useful much earlier in the research phase. Not develop entire theory in research environment. We have participatory design, set movement in the teacher community. A lot earlier than with many TEL movements. Vital to bridge that gap.

Myles: I also have a background in TEL, eAssessment systems, with a team of learning technologists. The cyclic nature of using things – instructors, academics, teachers would go to use the systems, then leave them for six months, a year – is a problem. Then the system had been updated, need further support to re-remember. Heavy lift done by the analytics, that lowers the bar, provide accessible visualisations, it’s a literacy people are picking up, embed analytics in practice, run these reports out, replacement for previous task, not an extra burden, few key differences there. [Doug’s prize for most metaphors linked in a single utterance there  – good points though.]

John: I do not have a background in TEL! Group psychology. Never waste a good hype to use for a good cause. LA called hype for a good reason. More RoI, more online data, more open education.

Erik: Don’t waste a good hype, nor a good dinner! See you at the dinner!

Liina: Promotion: Commission uses experts to evaluate programmes. Come to our website, might need people for the forthcoming call. Always gaps to be filled. A good way to get involved. International experts.

Erik: It’s also a good way to learn how to write good proposals.

–

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.