Analysing learning: from policy to implementation at scale

Last month I wrote my column about shaping policy and encouraged you to participate in the online conversation set up by FELTAG, the Further Education Learning Technology Action Group, led by Minister for State Matthew Hancock. Since then, there has been a noticeable increase in the number of visits the site has received as well as comments and other contributions.

The report from FELTAG is now close to being published and your input has been very valuable.

One of the innovations in Learning Technology the report will address is the advent of Massive Open Online Courses, or MOOCs. I think that despite the hype, and regardless of whether you or your institution are already exploring such an online offering or planning to do so in future, there are things we can learn from them that can be applied to different contexts.

While online learning or distance learning has been provided for decades, more recent developments in technology have provided ways to provide online learning at a lower cost and a larger scale than before. They also enable us to use learning analytics to measure what learners do and how they respond to different approaches. And learning analytics, based on large data sets, can give us valuable insights into why some things work and others do not. Taking this a step further, predictive learning analytics are expected to enable providers to deploy systems in such a way, that issues are flagged up early and prompt pre-emptive action. Integrated into administration or virtual learning environments, such systems can help to enhance the learner experience.

While I certainly agree that analysing data such as this and learning from it is going to become increasingly important and useful across education sectors, I remain critical of the way in which such analysis is often portrayed as fact or understood as such. Re-reading the ever popular “How to lie with statistics” written by Darrell Huff in 1954, is a timely reminder that regardless of how complex and insightful statistics might be, it remains essential to understand what it actually conveys – and understanding the results of analysing data sets of large scale online learning is certainly not straight forward.

This vision of learning, of using ‘big data’ to shape in particular a formal education system, could cumulate in such effective use of analytics that learning design would only result in approaches that worked and support for learners at all stages of life would ensure that no one failed or dropped out.

While I admit this is a crude and simplified analogy, I can only hope that this would work better than the way parts of the online world are being personalised on the basis of analytics of enormous data sets. Otherwise learner journeys would be determined by such actions as looking for a knitting pattern for an elderly relative using a search engine and then having a few months of knit-related content as a result.

Large providers of online learning have expertise in making sense of the data they collect, enhancing their offering as a result. The organisations that currently dominate much of our experience of the web place an even greater emphasis on data analytics and depend on big data for their success.

We have gotten used to the idea that intelligent use of Learning Technology involves keeping pace with change, new technologies, the skills needed to interact with it and, importantly, pedagogical approaches that work. On a day-to-day basis, that can be difficult to implement. And yet, now it seems that we need to add another dimension as we have an increasing need to understand the data these new ways of learning, particularly at scale, generate. Otherwise someone else will shape our learning landscape, someone who knows how to understand big data.

Maren Deepwell is chief executive of the Association for Learning Technology (ALT), an independent membership charity whose mission is to ensure that use of learning technology is effective and efficient, informed by research and practice, and grounded in an understanding of the underlying technologies and their capabilities, and the situations into which they are placed