An article ‘Predicting Employment through Machine Learning‘ by Linsey S. Hugo on the National Association of Colleges and Employers web site,confirms some of my worries about the use of machine learning in education.

The article presents a scenario which it is said “illustrates the role that machine learning, a form of predictive analytics, can play in supporting student career outcomes.” It is based on a recent study at Ohio University (OHIO) which  leveraged machine learning to forecast successful job offers before graduation with 87 percent accuracy. “The study used data from first-destination surveys and registrar reports for undergraduate business school graduates from the 2016-2017 and 2017-2018 academic years. The study included data from 846 students for which outcomes were known; these data were then used in predicting outcomes for 212 students.”

A key step in the project was “identifying employability signals” based on the idea that “it is well-recognized that employers desire particular skills from undergraduate students, such as a strong work ethic, critical thinking, adept communication, and teamwork.” These signals were adapted as proxies for the “well recognised”skills.

The data were used to develop numerous machine learning models, from commonly recognized methodologies, such as logistic regression, to advanced, non-linear models, such as a support-vector machine. Following the development of the models, new student data points were added to determine if the model could predict those students’ employment status at graduation. It correctly predicted that 107 students would be employed at graduation and 78 students would not be employed at graduation—185 correct predictions out of 212 student records, an 87 percent accuracy rate.

Additionally, this research assessed sensitivity, identifying which input variables were most predictive. In this study, internships were the most predictive variable, followed by specific majors and then co-curricular activities.

As in many learning analytics applications the data could then be used as a basis for intervention to support students employability on gradation. If they has not already undertaken a summer internship then they could be supported in this and so on.

Now on the one hand this is an impressive development of learning analytics to support over worked careers advisers and to improve the chances of graduates finding a job. Also the detailed testing of different machine learning and AI approaches is both exemplary and unusually well documented.

However I still find myself uneasy with the project. Firstly it reduces the purpose of degree level education to employment. Secondly it accepts that employers call the shots through proxies based on unquestioned and unchallenged “well recognised skills” demanded by employers. It may be “well recognised” that employers are biased against certain social groups or have a preference for upper class students. Should this be incorporated in the algorithm. Thirdly it places responsibility for employability on the individual students, rather than looking more closely at societal factors in employment. It is also noted that participation in unpaid interneships is also an increasing factor in employment in the UK: fairly obviously the financial ability to undertake such unpaid work is the preserve of the more wealthy. And suppose that all students are assisted in achieving the “predictive input variable”. Does that mean they would all achieve employment on graduation? Graduate unemployment is not only predicated on individual student achievement (whatever variables are taken into account) but also on the availability of graduate jobs. In teh UK  many graduates are employed in what are classified as non graduate jobs (the classification system is something I will return to in another blog). But is this because they fail to develop their employability signals or simply because there simply are not enough jobs?

Having said all this, I remain optimistic about the role of learning analytics and AI in education and in careers guidance. But there are many issues to be discussed and pitfalls to overcome.

I’ve tended to be skeptical about Learning Analytics, seeing it of limited relevance to pedagogy and more concerned with managing learners (reducing dropouts) than having anything to say about learning. Even more, Learning Analytics research has tended to docus on higher education and formal learning, having little to say about workplace learning and vocational education and training. But things are changing, especially through the integration of AI with Learning Analytics Learning Analytics and AI for future focused learning. I’m especially interested in this since we have had a project approved under the Erasmus Plus programme on AI and vocational education and training teachers and trainers.

This presentation by Simon Buckingham Shum at the EduTECH conference in Australia in June of this year introduces some of the work at the UTS Connected Intelligence Centre, where, he says “the team has been refining (for the last 3 years) automated, personalised feedback to students on higher order transferable competencies (Graduate Attributes in university-speak, or General Capabilities in the schools sector) – namely, high performance face-to-face teamwork (exemplar: nursing simulation exercises), and critical, reflective thinking (as revealed in students’ writing).”

Simon says “Learning Analytics bring the power of data science and human-centred design to educational data, while AI makes new forms of timely assessment and feedback possible. Tech researchers, developers, educators and learners can co-design formative feedback on 21st century competencies such as critical reflective writing, teamwork, self-regulated learning, and dispositions for lifelong learning. Such tools are being coherently integrated into teaching practice and aligned with curriculum outcomes at UTS, and could be in schools.

Getting the technology’s capabilities and the user experience right is impossible without meaningful engagement with educators and students. So, this talk is organised around our emerging understanding of how to align the different elements of the whole sociotechnical infrastructure. To use the language of the framework – the ‘cogs’ can be tuned to different contexts, and must synchronise and drive in the same direction to create a coherent learning experience.”

A number of things strike me about the presentation (and the videos within the presentation).

The first is the integration of the LA Framework with more traditional educational frameworks including competences and assessment rubrics. These provide a much broader reference point for proxies for achievement and reflection than the relevant proxy used in LA (and indeed in many areas of education of achievement in examination and other assessments. The design process is intended to develop a data map to these proxies.

Secondly, rather than seeking to provide feedback to students on attainment (and likely attainment – or otherwise) or to serve as the basis for intervention by teachers, the focus is on reflection. The feedback is seen as “a  provocation to deeper discussion” and as “scaffolding reflection

Finally – and as part of the refection process – the LA is designed to provide agency for the student, who, says Simon “can push back against the machine” if they think it is wrong.

All in all, there is much content for reflection here. The slides which contain a number of references can be downloaded here (PDF).

There continues to be a flurry of newspaper articles and studies of teh effect of automation and Artificial Intelligence on employment and jobs. There are different predictions about the scale of the change and particularly about the numbers of jobs which are at risk. One cause of the difference is disagreements about how many new jobs will be created, another is the speed of change. This may in part depend on whether employers choose to invest in new technologies: in teh UK productivity has remained persistently low, probably due to low wage rates.

What we do know is that organisations will need to cope with many of the changes associated with changes in the skill mix required of their employees  through learning through challenging work, training and continuing professional development etc. We also know that the changes mean it is difficult to imagine exactly how the labour market will look in say ten years but understanding the labour market can help people make sense of the context in which they are working or are seeking to work

At the same time we do not know the exact skill demands associated with unforeseen changes in the labour market, but we do know that new technical skills will be required, individuals and firms may need to specialise more to compete in global markets, and that demand will grow for ‘soft skills’ which are very difficult to automate, including complex social skills, cultural and contextual understanding, critical thinking, etc.

Yet this debate is not new. In the 1990s there were similar debates around teh move towards the ‘knowledge society’. At that time it was being predicted that low skilled work was set to rapidly decline, a prediction that pre-dated the rapid expansion in low skilled (or at any rate low paid) employment in the service sector. the answer at that time was seen to be promoting transversal skills and competences, variously called core skills, core competences etc. These emhpasised teh important of literacy and numeracy as well as communication skills and Information Technology. The problem was that such skills and competences were, in general abstracted from the curriculum as stand aone areas of learning, rather than being integrated within occupational learning. Of course, the other tendency n many Euroepan countries was to increase the number of young people going to university, at the expense of vocational educati0on and training.

What was needed then as now was to develop technical skills coupled with soft skills. Mastery of a technical skill is itself be a transferable skill whereby other technical skills can be developed more quickly as they are required . Developing latest industry-integrated technical skills is easiers if an underpinning technical knowledge base has been developed through more traditional educational provision. Retraining while in-work is very much easier than getting redundant people back into work.

Germany by Gerald Heidegger and Felix Rauner who looked at occupational profiles. Occupational profiles are in effect groups of competencies based on individual occupations. In Germany there are over 360 officially recognised occupations.

As long ago as 1996, Gerald Heidegger and Felix Rauner from the University of Bremen were commissioned by the Government of Rhineland Westphalia to write a Gutachten (policy advice) on the future reform and modernisation of the German Dual System for apprenticeship training.

They recommended less and broader occupational profiles and the idea of wandering occupational profiles. By this term they were looking to map the boundaries between different occupations and to recognise where competences from one occupation overlapped with that of another. Such overlaps could form the basis for boundary crossing and for moving from one occupation to another.

Heidegger and Rauner’s work was grounded in an understanding of the interplay between education, work organisation and technology. They were particularly focused on the idea of work process knowledge –  applied and practice based knowledge in the workplace. This was once more predicated on an idea of competence in which the worker would make conscious choices of the best actions to undertake in any particular situation (rather than the approach to competences in the UK which assumes there is always a ‘right way’ to do something).

Per Erik Ellstroem from Sweden put forward the idea of Developmental Competence – the capacity of the individual to acquire and demonstrate the capacity to act on a task  and the wider work environment in order to adapt, act and shape (design) it.

This is based on the pedagogic idea of sense making and meaning making through exploring, questioning and transcending traditional work structures and procedures. Rauner talked about holistic work tasks, based on the idea that a worker should understand the totality of the work process they are involved in.

In this respect it is interesting to see the results of recent research by Burning Glass, a company using AI and big data techniques to analyse labour market information. They say that in examine the role of Receptionist in Burning Glass Technologies’ labor market analysis tool, Labor Insight, “we can see that receptionists have a variety of related jobs they can do based on their transferable skills. Transferable skills are types of skills that a worker can use across many jobs, allowing them to more easily transition into a new role. A receptionist has many transferable skills such as administrative support, customer service, scheduling, data entry, and more. These transferable skills will allow a receptionist to move into related jobs such as Legal Secretary, Executive Assistant, or File Clerk.

According to Labor Insight, a Receptionist can transition into a Medical Secretary role which offers a higher average salary and is projected to grow by 22.5% in the next 10 years. This also offers an opportunity for the receptionist to venture into a new industry, allowing them to explore new health care roles such as Nursing Assistant, Emergency Room Technician, or Patient Service Representative.

The transferable skills that Burning Glass talk of are very similar to Rauner and Heidegger’s wandering occuaptional profiles. Rather than. as some commentators have suggested (see for example Faisal Hoque), a return to humanities based subjects in providing abstracted knowledge as the basis for future qualifications, the need is to improve vocational education and training which allows workers to understand the potentials of integrating automation and AI in the workplace. Creativity is indeed important, but creativity was always a key aspect of many jobs: creativity is part of the work process, not an external skill.

I have been working on writing a proposal on Artificial Intelligence and teh training of teachers and trainers in Vocational Education and Training. So I’ve spent a few days chasing up on research on th subject. I can’t say a lot of it impresses me – there is a lot of vague marketing and business stuff out there which shows not much insight into education.

One blog post I did like was by Rose Luckin, Professor of Learning with Digital Technologies, University College London Institute of Education’s Knowledge Lab, who has written an ‘Occasional Paper: The implications of Artificial Intelligence for teachers and schooling’, published on her blog.

Rose says there are three key elements that need to be introduced into the curriculum at different stages of education from early years through to adult education and beyond if we are to prepare people to gain the greatest benefit from what AI has to offer.

The first is that everyone needs to understand enough about AI to be able to work with AI systems effectively so that AI and human intelligence (HI) augment each other and we benefit from a symbiotic relationship between the two. For example, people need to understand that AI is as much about the key specification of a particular problem and the careful design of a solution as it is about the selection of particular AI methods and technologies to use as part of that problem’s solution.

The second is that everyone needs to be involved in a discussion about what AI should and should not be designed to do. Some people need to be trained to tackle the ethics of AI in depth and help decision makers to make appropriate decisions about how AI is going to impact on the world.

Thirdly, some people also need to know enough about AI to build the next generation of AI systems.

In addition to the AI specific skills, knowledge and understanding that need to be integrated into education in schools, colleges, universities and the workplace, there are several other important skills that will be of value in the AI augmented workplace. These skills are a subset of those skills that are often referred to as 21st century skills and they will enable an individual to be an effective lifelong learner and to collaborate to solve problems with both Artificial and Human intelligences.

Fear you are going to be seeing this headline quite a bit in coming months. And like everyone else I am getting excited and worried about the possibilities of AI for learning – and less so for AI in education management.

Anyway here is the promise from an EU Horizon 2020 project looking mainly at ethics in AI. As an aside, while lots of people seem to be looking at ethics, which f course is very welcome, I see less research into the potentials and possibilities of AI (more to follow).

The SHERPA consortium – a group consisting of 11 members from six European countries – whose mission is to understand how the combination of artificial intelligence and big data analytics will impact ethics and human rights issues today, and in the future.

One of F-Secure’s (a partner in the project) first tasks will be to study security issues, dangers, and implications of the use of data analytics and artificial intelligence, including applications in the cyber security domain. This research project will examine:

• ways in which machine learning systems are commonly mis-implemented (and recommendations on how to prevent this from happening)
• ways in which machine learning models and algorithms can be adversarially attacked (and mitigations against such attacks)
• how artificial intelligence and data analysis methodologies might be used for malicious purposes