Liveblog notes from Wednesday morning at LAK16 [full list of blogging]: – the opening keynote session.
Fantastic opening as Dragan, Prof Sir Tim O’Shea, and many others are piped in to the auditorium to start the conference.
Welcome and Introduction
Dragan welcomes everyone. Record-breaking LAK: 467 participants, last year 320. 424 main conference, 360 pre-conference.
Prof Sir TIm O’Shea, VC & Principal of Edinburgh University, welcomes everyone. L@S left here in sunshine, but could see nothing in a blizzard, then blue skies afterwards. Strongly encourages people to climb Arthur’s Seat. Edinburgh famous for its role in the Enlightenment, Adam Smith, David Hume, and Hutton. He spent 20 years staring at Arthur’s Seat and Salisbury Crags in the C19th, then said Arthur’s Seat was a volcanic plug and was hot, and Salisbury Crags was under the sea and sedimentary rock. He came up with geology by staring at those rocks.
We educated Thomas Bayes. A protestant minister, late C18th, his objectives were to improve performance at gambling, which is where Bayes’ Law comes from. Important stats but he didn’t get rich. More recently, in the 60s we were responsible for SPSS. In the 70s, flourishing of AI in Ed, I was part of that. We were using production rules 40y ago, but didn’t have hundreds of thousands of learners. My PhD in machine learning applied to ed in the 70s. 2012 MOOCs was the big change. For this university, great growth in our online masters, MOOCs. Our colleagues have long-term work built on. For Edinburgh, this is an extraordinary time to improve access, understanding. You have a challenge and an opportunity your predecessors didn’t. It’s an incredibly special time. LA, serious understanding of how one might improve access to ideas is available to you. I wish you all the best.
Shane Dawson thanks (co-) Program Chairs Carolyn Rose, Hendrik Drachsler. And the entire chair group. And the numerous reviews. One person did 30. George Siemens did half, thanks George for your valuable contribution.
Submissions are increasing massively. We’ll have a million by 2025. The weather is getting better from -20 in Banff. Acceptance rate roughly steady around 30%. Quality improved.
Shift in themes: more behaviour modelling, alt data sources; decrease in students at risk, performance. More countries represented. Last year 34, now 44. Welcome! On a per capita basis, Australia win [Dragan has put “Show offs” over Shane’s slide.]
Dragan takes over to say thanks to LAK16 sponsors too. Brightspace/D2L (gold), McGraw Hill, IBM, Blackboard, HT2 learning (silver). Thanks to local organisers – Vitomir, Srecko, Grace, and himself.
Housekeeping arrangements. Trying to livestream all four parallel sessions. (!) This room has 330 capacity, so may overflow. Sign release forms. Don’t queue for lunch before 12.15pm. No running or spitting in corridors.
Dragan introduces Prof Mireille Hildebrandt, the first speaker.
Learning as a machine: Cross-overs between humans and machines (Keynote)
Professor Mireille Hildebrandt, Vrije Universiteit Brussels
When you invite a lawyer and philosopher, be sure the presentation will be text driven, not data. Sciences are less text-driven. So I have a cartoon-driven presentation.
Cartoon “But, sweetie, children are the backbone of our educational system”. The system is there for the individual people, not the other way round. Another one – competitive “my data is bigger than yours”.
I love the idea of elearning, of learning analytics, it’s extremely important and offers many opportunities. Having said that, I’ll move to a more critical part. Kahoot has flooded US law schools. Tweet from someone who’s done a quiz.
Another cartoon – ‘Come away from the window! You don’t want to be a child left behind, do you?’ ‘Sounds good to me!’. Assuming that data is the same as facts. We’re doing A/B testing on kids, in an invisible way, they may not want to go outside any more.
Shall we come to depend on learning analytics? I’m going to take you out of your comfort zone. Maybe LA will take institutions working in a similar way since the printing press out of their comfort zone, which is in principle a good thing. Humans learn from the models they are born, or before that. We learn because we can suffer, we can die, we will die, like all living organisms. Or nearly all. We want to delay that moment and flourish. We help others to learn.
This conference is about LA, data-driven apps that help students to improve their learning behaviours, and help schools, colleges and universities to administrate information about student performance. This is measured in machine-readable format. Whichever part is not machine-readable will not be part of learning analytics, and therefore not part of the performance metrics. To the extent that these metrics come to define learning processes, whatever is off the radar will not visibly contribute to the flourishing of the students. This has consequences for individual liberty. Democracy and the rule of law as contradictory forces: equal respect and concern for each individual person. One person, one vote. Also means human rights overrule majoritarian disrespect. Limits the dictatorship of the majority. Testing by machines may lure is in to thinking like them. We may start thinking in terms of behaviours instead of actions, be manipulated in to learning instead of being challenged to develop a critical distance.
We must distinguish two levels of intervention.
First level of LA is about interventions at the level of an identifiable student. This takes two different kinds. Collecting data from them, on one hand, and on the other hand, when you apply the results of LA to them, whether or not they’re aware of it. Identifiability includes indirect identification, and being singled out. Could be previous grades, teacher’s evaluation, tests, alumni. Linking to employment. Sounds like automation of previously manual admin. Also behavioural data, traces and tracks from relatively new software that enable elearning. Keystrokes, clickstroke, micro-level reading habits, biometric behaviour like eye movements indicating loss of focus. Also derived from sources outside the educational setting – e.g. social media, data bought from data brokers. May be correlated with educational data to see how that correlates with achievements. May be legal or illegal, or ethical or not; what’s important is that it is becoming possible. Possibility of the data becoming available to law enforcement.
Second level of LA is not about identifiable students, but analyses of the data, anonymous or personal. If it’s personal, can be pseudonymous or not. Anonymisation rules out data protection law, pseudonymous may not.
Pseudonymous data definition from the upcoming legislation. Just taking out an identifier is not enough. Processing of personal data so it can no longer be attributed to a specific data subject without the use of additional information, as long as such additional information is kept separately and subject to technical and organisation measures to ensure non-attribution to an identified or identifiable person. Need to do this to have the light-touch regime.
Developing novel, useful patterns in the data. To help data-driven machines to learn. Key classifiers that enable teachers to gain insight in to the data points that are key to learning and organisations achievements. I don’t have to explain that to you.
This level forages patterns in data to improve at the first level. They may be about when are they ready to move on, when to refer to counsellor for help, and so on. An editorial which commends this architecture for structured and unstructured data. Jinan Fiaidhi, The Next Steps for LA, IT Pro Sept/Oct 2014. Advice, relevant prediction. Let’s keep this in mind.
Anonymisation, pseudonymisation, should be a primary concern. But other major concerns we can not address by de-identifying the data. Not just about privacy, but non-discrimination, due process, presumption of innocence. Concerns affordances of learning institutions.
Want to develop an understanding of what machine learning (ML) does to contribute to human learning (HL). Three ways of learning as a machine. Data protection by design is going to be a requirement. It may help [in various ways].
First of three: the Pavlov approach. (Cartoon of dog drooling to cause Pavlov to smile and write in his book.) Stimuli trigger responses actuated by means of specific responses, in anticipation. Pavlov was one of the first behaviourists, treated mind as a black box, to supposedly develop objective fact-finding. Behaviourists trying to predict and control human behaviour.
Strong links with materialism, assuming the mind is made of matter. Assumed to be subject to the laws of causality. Seeking to understand individuals as nodes in a network, engineering a more fair society. Idea is if we know statistically how people learn, how they develop which conditional reflexes, we can reconfigure their environment to make sure they learn what we (who is we) should learn in terms of behaviour. There may be noble intent, but assumption we learn as a machine. Leads to attempts to manipulate us in to good behaviours, as if pawns in a game of chess, or Go. Plasticity of human brain, vulnerability of creativity, individual humans and society. Refers to intractability of HL – increasing computer power will push that boundary? But computability. Natural language is generative of a complex dynamic web of meaning. Uncertainty is core to natural language. Creativity, that springs from our language-saturated environment. Uncertainty and creativity holds us together, and sets us apart from other animals, and maybe also machines. Ignoring our capability for generating meanings is problematic ethically and epistemologically.
A question. Move from teaching students, putting new knowledge sharing acquired knowledge – to inducing learning processes by manipulating their environment by inducing them to acquire specific knowledge. Corrupts their autonomy (?).
Second approach: the Simon approach.
Why should machines learn? Classic article, ML was achieving next to nothing. Relevant observations about differences between human and ML. The tediousness of HL, it’s so slow, takes a long time to achieve knowledge people spend many years in school. Once a computer programmed, it can be transferred from one to the next. “We should ask whether we want a computer to go through that tedious process, or program them directly to avoid humanoid learning entirely. Only one computer will have to learn.” We may smile at the naivety. He was tongue in cheek; it will only work if it was doing the same task repeatedly. If goal is to learn a wide range of tasks, or discovering new knowledge, maybe the tediousness of HL may be the best way to optimise it.
Facing real-life problems is really complex. Simon says, instead of all hope in one program, society has numerous independent programs. Old programs do not learn, they fade away, so do human beings. Transferring human learning takes great effort, but it’s not a disadvantage. The optimal learning machine must be put to the test with alternative machines, we can not be sure it will work for others.
Third approach: Gibson approach.
I think this is very interesting. James Gibson, crucial concept, of affordance. Embedded in evolution, psychological enquiry. Skips naive division between mind and matter without discarding values. Affordance defined as what it offers the animal, what it provides for good or ill. Made up, refers to both, implies complementarity. Not interested in separating objective properties of subjective experience, but something else. The actionable properties of an environment, which depend on the agent, who depends on the environments. Each agent has a niche, the set of affordances the environment offers them. Also other humans. This celebrates creativity, doesn’t define agent or niche by their properties. Machines depend on an environment that affords them to learn one thing not another. Many machines require and artificial environment, e.g. preventing them from harming humans. The envelope, in robotics, to ensure robot can navigate its environment without harming humans. Fish don’t do well on land, we don’t do well in water.
Obviously, humans reconfigure their niche in myriad ways, with far-reaching consequences for others. Introduction of calculating and writing tools are profound, afford an entirely novel learning mechanism, including the retention of what’s learned outside the body of an individual human. Triggers formal learning, schools, universities, archives, libraries, computers.
Gibson’s approach raises question of how elearning will transform the affordances of our learning institutions, transforming our ecological niche. We are increasingly confronted with mindless agents, adapting their actions to the feedback they gain from ours. We don’t just use the machines to learn faster, but they use us to improve their performance.
Finally, may come to the point of interacting with them, training algorithms to learn how to make us learn. Train/test set is already an interaction. Neural nets. When I teach a human person, I can’t assess her brain or neural computations, and even if I could, they’d be incomprehensible; that doesn’t stop us teaching. Same goals in teaching a neural net. However, physical systems with ML have nothing to lose, they cannot suffer or fear death, they can only simulate that. Simulation is not the same as what is simulated. This may lead to cause for concern.
What should concern us, is humans have to anticipate how machines learn, e.g. training of algorithm. Students interacting with systems that only use their machine-readable behaviours. As a result, our niche may be shaped in a way that accommodates learning as a machine by us. We can do that due to the plasticity of our brain. It takes years to change the morphology and behaviour of our brains such that we can read. Once we read, this ecological niche entirely changes. We can develop a type of brain more geared towards mathematics, if we decide that is worth the effort. That might fast-track our LA. Transformative effects on what it means to be human.
Moving on to first-level issues. Fundamental rights. About interventions at the level of identifiable individuals, collecting personal data. Applying results of LA to individual students, whether they’re aware of it or not. At this level, privacy may be concerned whenever data is shared violating legitimate expectations. Notably when out of context.
When inferences are applied, confronting learner with things she was not aware of, this is another type of privacy violation. E.g. health problems, especially mental health. Or particular religious backgrounds leading to failing to get a degree. It’s not always about privacy, but non-discrimination. Specific learning disabilities can be helped by specific [actions], but implies students used as guinea pigs to secure policy goals of the institution, or more mundanely, to increase profits of a service provider. To the extent a student is unaware of this process, their due process rights are at stake.
Non-distributive. To the extent they’re unaware, they’re unable to argue. Arguments against being profiled may be restricted, you may have to object by an online system that requires arguments to fit a particular format. The people or software dealing with these appeals may have formatted responses, the simplest one is, the system does not allow this argument, or in your reasoning there is no argument that makes us change our mind. The system is already in place – tax office, telecom providers. Computer says no.
This is Kafka. Presumption of innocence. Not science fiction. Consequences of entering another ecological niche. If we don’t like the affordances, it is time to configure it in another way.
Second-level issues. Entirely different, inferences may not be individual. Anonymisation is difficult. But about patterns in the dataset. Creating hindsight, foresight, and insight. At this level, privacy is mainly protecting data against unauthorised disclosure. Many techniques to enable this, based on irreversible pseudonymisation. E.g. MIT open personal data store. Max Planck (has one too). My own research group has developed a framework (PEP). A lot more out there, there’s a market, that’s a good thing. Combining encryption with pseudonymous sharing. Run code on raw data without sharing the data. Highly relevant, important to engage in data protection.
But they do not go to the heart of the matter. Once an entity gets hold of the results of the analytics, it’s not personal data, a new set of affordances come in to play. Example of a stone, has certain affordances, if you change the stone, the interactions with other agents will change enormously. An affordance refers to the actionability of the environment.
Our capability to reflect on our own behaviours. Old school learning affords reflection. The fact you externalise something, be critical about it, that need not be an affordance of LA. Depends on how we design.
Profiling and data protection by design. Particular example of legal protection by design. General data protection regulation(GDPR) voted through, in force in two years. Reinforces current data protection directive, but will be a game changer. Especially secondary usage. New regulation will do three things:
- create a level playing field
- require a data protection impact assessment
- privilege the processing of pseudonymous data
Fines 4% of global turnover. Tort liability. Countervailing powers to mitigate problematic affordances. Speaking law to power. Impact assessment whenever high risk; obligation to implement data protection by default and design. Data minimisation in the architecture of data processing systems. The ecological niche will be reconfigured. Architecting trust, reducing privacy risks by default. Pseudonymous data, objective to enable big data, incentive structure favouring secondary use for specific purposes but reducing risk of identification. Again, transformative impact on affordances. Profile transparency, part of current framework, but further clarified. Profile transparency directly targets issue of due process and presumption of innocence.
GDPR and profiling – any processing to evaluate certain personal aspects, in particular analysing or predicting behaviour. The transparency requires three things – information to individuals – your students:
- the existence of automated decision-making; you have to tell them you’re doing this
- meaningful information about the logic involved; you have to explain how the outcome and which factors it uses
- the significance and the envisaged consequence; explain, we’ve taken this decision based on this LA, this is the logic behind it, and this is the consequences for you.
The consequences can be pretty important.
Right not to be subject to a decision based solely on automated processing, if that decision has legal effects or significantly affects them. Also a clause saying if it’s on sensitive characteristics (e.g. race or sexual orientation), that’s not allowed at all even if they agree.
Not the same right as the privacy right. Explicitly addresses due process, presumption of innocence, non-discrimination. Taking our learners seriously, engaging on their own terms, helping them critically assess learning algorithms.
Human learning cannot be reduced to Pavlov. Even animals are driven crazy. Says little about behaviour in normal ecological niches.
Simon demonstrated added value of individual learning processes. Unfathomable complexity of human society. Taking Gibson, this is connected to language and institutional dynamics. Our ecological niche has myriad affordances, renegotiated. Capabilities shape and re-shaped. Creativity and humour, need to face both life and death.
Teaching will remain critical to a democracy. The explicit presentation of knowledge so it can be the subject of debate and discussion. LA may induce learning processes, behind their back, not necessarily wrong or bad. But must be reconfigured to allow reflection not just on what but how we learn. Profile transparency, chance to object to profiling, very important.
–
This work by Doug Clow is copyright but licenced under a Creative Commons BY Licence.
No further permission needed to reuse or remix (with attribution), but it’s nice to be notified if you do use it.
Doug Clow's Imaginatively-Titled Blog 