LAK12: Monday afternoon

This is liveblog notes from the Monday afternoon session at LAK12. It’ll be sketchy for the first session because I’m doing my talk then.

Session 2B: Institutional Perspectives

Tim McKay is chairing, introduces the four papers: one long, three short.

The Telus World of Science

Hendrik Drachsler and Wolfgang Greller: Confidence in learning analytics

Hendrik is from the OUNL. He says the talk takes 29 minutes, so he’s going to speed up. It’s on ‘understandings and expectations from the stakeholders’. They have a survey, with an organising framework.

Goals – the framework, survey, findings, and conclusions.

Framework for Learning Analytics – paper submitted to J Ed Tech & Soc, Greller & Drachsler.

Carried out a survey.

Survey information and data is available online at

Participants in multiple roles – teachers, researchers, learning designers, others. Had responses from 31 countries, mainly UK, US and NL.

Explored many different components; including competences needed.

70% didn’t think learners were competent enough to learn independently from learning analytics.

Their sample dominated by HE; low awareness of learning analytics, so only innovators and early adopters. Missing students and non-western cultures.

Main beneficiaries of learning analytics are learners and teachers followed by organisations. (!)

Trust not well developed. (I say rightly!)

Data ownership is most important topic; many organisations have ethical boards and guidelines.

Future work is around getting more insight from the dataset, partnering to reach other areas, group concept mapping.

Slides at


Someone: LMS, CMS – are those terminology differences?

H: Good one. We see sign up using CMS, use LMS to learn. See a difference in some organisations.

[I stopped here; there were more questions.]

Doug Clow: The Learning Analytics Cycle

Paul Prinsloo, Sharon Slade, Fenella Galpin: Challenges, paradoxes and opportunities for mega open distance learning institutions.

[Very sketchy notes here.]

Sharon presenting.

The ‘thirdspace’ and context from two distance learning institutions.

Thirdspace is liminal, intertextual. Third space between students world and institutional world.

Liminal between who they were at registration and who they will be at graduation.

New approach towards curriculum-based support facilitated by a learning analytics approach.

Potential is huge. At large ODL institutions, need to act on analyses and findings at scale.

Eitel Lauria, Joshua Baron, Mallika Devireddy, Venniraiselvi Sundararaju and Sandeep Jayaprakash. Mining academic data to improve college student retention: An open source perspective

Josh is presenting – Senior Academic Technology Officer.

Open Academic Analytics Initiative (OAAI). Challenge: 36% completion rate of 4-year degrees in the US.

Two datasets – student information system (demographics, SATs, GPA), and LMS.

Goal to create open source early alert system of at-risk students, deploy intervention.

Builds on Purdue’s Signals. Interventions that use “support groups” are powerful – on GPAs and completion.

Building an open ecosystem – Sakai, Pantaho Business Intelligence Suite (OS data mining tools), OAAI Predictive Model using Predictive Modeling Markup Language (PMML) will be released under and open source factor. But question around portability, and which intervention strategies are most effectively.

Have a NGLC (Gates Foundation) grant on this, to end Jan 2012.

Eitel Lauria takes over.

He sets out their model development framework.

It’s very complicated. Variability in usage of tools, no uniform criterion; too much missing data.

Moving IBM SPSS PASW Modeller approach to open source domain.

Experimental setup on Fall 2010 data sample. Trained a predictive model (train/test iteration), good summary measures, with a mean and a standard error.

Logistic regression and SVM did better than C5.0/J4.8 – but large number of false positives, a concern.

Big predictors: RMN_SCORE (partial-grades-derived score) then ACADEMIC_STANDING, CUM_GPA.

CMS generated events appear to be second tier predictors. Are they a proxy for student effort? Why low predictive power?


Hendrik: How much are the outcomes transferable to another Sakai system. We run a Sakai system, the data will be different. How much can we apply?

EL: Josh mentioned we were able to use the models, which is a methodology that can be implemented in a given structure. If data properly structured, coding may be different, and we get Sakai data, can easily use our models for prediction. Will get better results if we get training data from the institution we’re predicting. It’s hard to think you can port a model unless the universities are similar. You would expect will be differences. We do have a methodology in place, implemented as an infrastructure, we can deliver.

Josh: We’ll release an API for Sakai to help get data out to other analytics tools.

Someone 1: Dataset used for creating models, was over a single academic term?

EL: For this paper was Fall 2010, tested with Spring 2011. We have now two full semesters of data. Decided not to go back. Now making predictions on Fall 2011.

Someone 1: The events might change through the term – e.g. people who browse early on are at little risk, vs total browsing. Predictions are for end of term?

EL: That’s why we compute ratios, not only across courses, but across the amount of time consumed for a specific [] we want to make a prediction. Assumption that effort is uniform, and we know it is not. The way we aggregated the data may be responsible for the CMS data being second-tier predictors.

Someone 1: Even use a single week, use the final outcomes; try a model for the first few weeks. Would capture what’s important at the initial stage. So in the first week, no marks data, but CMS data might be good. By the time you get the mid-term grade, that’s the best predictor by far so the CMS data is less interesting.

Proposed to move the discussion of details to the bar!

This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.

Author: dougclow

Data scientist, tutxor, project leader, researcher, analyst, teacher, developer, educational technologist, online learning expert, and manager. I particularly enjoy rapidly appraising new-to-me contexts, and mediating between highly technical specialisms and others, from ordinary users to senior management. After 20 years at the OU as an academic, I am now a self-employed consultant, building on my skills and experience in working with people, technology, data science, and artificial intelligence, in a wide range of contexts and industries.

%d bloggers like this: