I have been looking at the potential of Learning Analytics (LA) for professional development of employees in European Public Employment services as part of the European funded EmployID project. Despite interest, particularly from Learning and Development personnel within the employment services, Learning Analytics, has made only limited impact and indeed reflects the slow take up of LA in the workplace as a whole.
The reasons for this are myriad. Universities and schools have tended to harvest existing data drawn from Virtual Learning Environments (VLEs) and to analyse that data to both predict individual performance and undertake interventions which can for instance reduce drop-out rates. The use of VLEs in the workplace is limited and “collecting traces that learners leave behind” (Duval, 2012) may fail to take cognizance of the multiple modes of formal and informal learning in the workplace and the importance of key indicators such as collaboration. Ferguson (2012) says that in LA implementation in formal education: “LA is aligned with clear aims and there are agreed proxies for learning.” The most commonly agreed proxy of learning achievement is achievement of outcomes in terms of examinations and assignments. Yet in the workplace, assignment driven learning plays only a limited role, mostly in formal courses and initial vocational education and training.
Workplace learning is driven by demands of work tasks or intrinsic interests of the learner, by self-directed exploration and social exchange that is tightly connected to processes and the places of work (Ley at al, 2015). Learning interactions at the workplace are to a large extent informal and practice based and not embedded into a specific and measurable pedagogical scenario.
In present Learning Analytics developments, there appears to be a tension between measuring and understanding. Pardo and Siemens (2014) say “learners are central agents and collaborators, learner identity and performance are dynamic variables, learning success and performance is complex and multidimensional, data collection and processing needs to be done with total transparency.” This poses particular issues within the workplace with complex social and work structures, hierarchies and power relations.
Despite these difficulties we remain convinced of the potential value of Learning Analytics in the workplace and in Public Employment Service organisations. If used creatively, Learning Analytics can assist learners in monitoring and understanding their own activities and interactions and participation in individual and collaborative learning processes and help them in reflecting on their learning. Furthermore, LA offers a potential approach to gaining rapid feedback to trainers and learning designers and data can be a tool for researchers in gaining a better understanding of learning processes and learning environments.
There is some limited emerging research into Workplace Learning Analytics and Social Learning analytics which offer at least pointers towards developing on such potential. Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings, taking into account both formal and informal learning environments, including networks and communities. Buckingham Shum, S., & Ferguson, R., (2012) suggest social network analysis focusing on interpersonal relations in social platforms, discourse analytics predicated on the use of language as a tool for knowledge negotiation and construction, content analytics particularly looking at user-generated content and disposition analytics can be developed to make sense of learning in a social setting.
Such an approach to Social Learning Analytics links to the core aims of the EmployID project to support and facilitate the learning process of PES practitioners in their professional identity development by the efficient use of technologies to provide social learning including advanced coaching, reflection, networking and learning support services. The project focuses on technological developments that make facilitation services for professional identity transformation cost-effective and sustainable by empowering individuals and organisations to engage in transformative practices, using a variety of learning and facilitation processes.
It should also be noted that although Learning Analytics has been linked to the collection and analysis of ‘big data’, MacNeill (2016) stresses the importance of fast data, actionable data, relevant data and smart data. LA, she says, should start from research questions that arise from teaching practice, as opposed to the more common approach of starting analytics based on already collected and available data.
Learning Analytics has been the subject on ongoing discussion in the EmployID project and particularly with the PES organisations. Although a number of PES organisations are interested in the possibility of adopting LA, it is not a major priority for them at present and they are aware of the constraints outlined above. Our initial experiences with sentiment analysis confirm this general interest as well as its limitations with public organisations. It has also became apparent that there are major overlaps between the Social Analytics approach and the tools and approaches we have been developing for evaluation. Our work in evaluation encompasses looking at interpersonal relations in social platforms, discourse analytics based on the EmployID MOOCs as well as learners own mapping of their progress through the self-assessment questionnaire.
We recognise that this data can be valuable for PES employees in supporting reflection on learning. But rather than seeking to develop a separate dashboard for reporting on data, we are attempting to embed representations of learning within the context in which the learning takes place. Thus, the social platform allows users to visualise their different interactions through the platform. Other work, such the facilitation coding scheme, does not yet allow real time analytics. But if proven successful as a research approach to understanding and supporting learning, then it could potentially be automated or semi-automated to provide such real time feedback.
As ever I am doing lots of work on labour market data and education and training. While we know pretty well how to take great slabs of data and turn it into various different charts – some more imaginative than others – this still leaves problems in how to illustrate ideas. data charts can be pretty dull – more than that they rely on the ability of the user to interpret that data – what we call the move from labour market information to labour market intelligence.
