An article ‘Predicting Employment through Machine Learning‘ by Linsey S. Hugo on the National Association of Colleges and Employers web site,confirms some of my worries about the use of machine learning in education.

The article presents a scenario which it is said “illustrates the role that machine learning, a form of predictive analytics, can play in supporting student career outcomes.” It is based on a recent study at Ohio University (OHIO) which  leveraged machine learning to forecast successful job offers before graduation with 87 percent accuracy. “The study used data from first-destination surveys and registrar reports for undergraduate business school graduates from the 2016-2017 and 2017-2018 academic years. The study included data from 846 students for which outcomes were known; these data were then used in predicting outcomes for 212 students.”

A key step in the project was “identifying employability signals” based on the idea that “it is well-recognized that employers desire particular skills from undergraduate students, such as a strong work ethic, critical thinking, adept communication, and teamwork.” These signals were adapted as proxies for the “well recognised”skills.

The data were used to develop numerous machine learning models, from commonly recognized methodologies, such as logistic regression, to advanced, non-linear models, such as a support-vector machine. Following the development of the models, new student data points were added to determine if the model could predict those students’ employment status at graduation. It correctly predicted that 107 students would be employed at graduation and 78 students would not be employed at graduation—185 correct predictions out of 212 student records, an 87 percent accuracy rate.

Additionally, this research assessed sensitivity, identifying which input variables were most predictive. In this study, internships were the most predictive variable, followed by specific majors and then co-curricular activities.

As in many learning analytics applications the data could then be used as a basis for intervention to support students employability on gradation. If they has not already undertaken a summer internship then they could be supported in this and so on.

Now on the one hand this is an impressive development of learning analytics to support over worked careers advisers and to improve the chances of graduates finding a job. Also the detailed testing of different machine learning and AI approaches is both exemplary and unusually well documented.

However I still find myself uneasy with the project. Firstly it reduces the purpose of degree level education to employment. Secondly it accepts that employers call the shots through proxies based on unquestioned and unchallenged “well recognised skills” demanded by employers. It may be “well recognised” that employers are biased against certain social groups or have a preference for upper class students. Should this be incorporated in the algorithm. Thirdly it places responsibility for employability on the individual students, rather than looking more closely at societal factors in employment. It is also noted that participation in unpaid interneships is also an increasing factor in employment in the UK: fairly obviously the financial ability to undertake such unpaid work is the preserve of the more wealthy. And suppose that all students are assisted in achieving the “predictive input variable”. Does that mean they would all achieve employment on graduation? Graduate unemployment is not only predicated on individual student achievement (whatever variables are taken into account) but also on the availability of graduate jobs. In teh UK  many graduates are employed in what are classified as non graduate jobs (the classification system is something I will return to in another blog). But is this because they fail to develop their employability signals or simply because there simply are not enough jobs?

Having said all this, I remain optimistic about the role of learning analytics and AI in education and in careers guidance. But there are many issues to be discussed and pitfalls to overcome.

Do you read books and papers on screen or do you prefer paper. I am conflicted. I used to have an old Kindle but gave it up because I am no fan of Amazon. And I used to read books on firstly an ipad and latterly an Tesco Huddle tablet – both now sadly deceased.

Like many (at least if the sales figures are to be believed) I have returned to reading books on paper, although I read a lot of papers and such like on my computer, only occasionally being bothered to print them out. But is preferring to physical books a cultural feel good factor or does it really make a difference to comprehension and learning?

An article in the Hechinger Report reports on research by Virginia Clinton, an Assistant Professor at the University of North Dakota who “compiled results from 33 high-quality studies that tested students’ comprehension after they were randomly assigned to read on a screen or on paper and found that her students might be right.”

The studies showed that students of all ages, from elementary school to college, tend to absorb more when they’re reading on paper than on screens, particularly when it comes to nonfiction material.

However the benefit was small – a little more than  a fifth of a standard deviation and there is an important caveat in that the studies that Clinton included in her analysis didn’t allow students to use the add on tools that digital texts can potentially offer.

My feeling is that this is a case of horses for courses. Work undertaken by Pontydysgu suggested that ebooks had an important motivational aspect for slow to learn readers in primary school. Not only could they look up the meaning fo different words but when they had read for a certain amount of time they were allowed to listen to the rest of teh story on the audio transcription. And there is little doubt that e-books offer a cost effective way of providing access to books for learners.

But it would be nice to see some further well designed research in this area.

Earlier this week the UK Jisc  launched the Jisc NUS roadmap (pdf) designed to support students, course representatives, and union and guild representatives to work with their institution on improving student digital experiences.

I have been working on writing a proposal on Artificial Intelligence and teh training of teachers and trainers in Vocational Education and Training. So I’ve spent a few days chasing up on research on th subject. I can’t say a lot of it impresses me – there is a lot of vague marketing and business stuff out there which shows not much insight into education.

One blog post I did like was by Rose Luckin, Professor of Learning with Digital Technologies, University College London Institute of Education’s Knowledge Lab, who has written an ‘Occasional Paper: The implications of Artificial Intelligence for teachers and schooling’, published on her blog.

Rose says there are three key elements that need to be introduced into the curriculum at different stages of education from early years through to adult education and beyond if we are to prepare people to gain the greatest benefit from what AI has to offer.

The first is that everyone needs to understand enough about AI to be able to work with AI systems effectively so that AI and human intelligence (HI) augment each other and we benefit from a symbiotic relationship between the two. For example, people need to understand that AI is as much about the key specification of a particular problem and the careful design of a solution as it is about the selection of particular AI methods and technologies to use as part of that problem’s solution.

The second is that everyone needs to be involved in a discussion about what AI should and should not be designed to do. Some people need to be trained to tackle the ethics of AI in depth and help decision makers to make appropriate decisions about how AI is going to impact on the world.

Thirdly, some people also need to know enough about AI to build the next generation of AI systems.

In addition to the AI specific skills, knowledge and understanding that need to be integrated into education in schools, colleges, universities and the workplace, there are several other important skills that will be of value in the AI augmented workplace. These skills are a subset of those skills that are often referred to as 21st century skills and they will enable an individual to be an effective lifelong learner and to collaborate to solve problems with both Artificial and Human intelligences.

Time to return to the Wales Wide Web after something of a hiatus in November and December. And I am looking forward to writing regular posts here again.

New year is a traditional time for reviewing the past year and predicting the future. I have never really indulged in this game but have spent the last two days undertaking a “landscape study” as part of an evaluation contract I am working on. And one section of it is around emerging technologies and foresight. So here is that section. I lay no claim to scientific methodology or indeed to comprehensiveness – this is just my take on what is going on – or not – and what might go on. In truth, I think the main conclusion is that very little is changing in the use of ICT for learning (perhaps  more on that tomorrow).

There are at any time a plethora of innovations and emerging developments in technology with the potential to impact on education, both in terms of curriculum and skills demands but also in their potential for teaching and learning. At the same time, educational technology has a tendency towards a ‘hype’ cycle, with prominence for particular technologies and approaches rising and fading. Some technologies, such as virtual worlds fade and disappear; others retreat from prominence only to re-emerge in the future. For that reason, foresight must be considered not just in terms of emerging technologies but in likely future uses of technologies, some which have been around some time, in education.

Emerging innovations on the horizon at present include the use of Big Data for Learning Analytics in education and the use of AI for Personalised Learning (see below); and MOOCS continue to proliferate.

VLEs and PLEs

There is renewed interest in a move from VLEs to Personal Learning Environments (PLE), although this seems to be reflected more in functionality for personalising VLEs than the emergence of new PLE applications. In part, this may be because of the need for more skills and competence from learners for self-directed learning than for the managed learning environment provided by VLEs. Personal Learning Networks have tended to be reliant on social networking application such as Facebook and Twitter. These have been adversely affected by concerns over privacy and fake news as well as realisation of the echo effect such applications engender. At the same time, there appears to be a rapid increase in the use of WhatsApp to build personal networks for exchanging information and knowledge. Indeed, one area of interest in foresight studies is the appropriation of commercial and consumer technologies for educational purposes.

Multi Media

Although hardly an emerging technology, the use of multimedia in education is likely to continue to increase, especially with the ease of making video. Podcasting is also growing rapidly and is like to have increasing impact in the education sector. Yet another relatively mature technology is the provision of digital e-books which, despite declining commercial sales, offer potential savings to educational authorities and can provide enhanced access to those with disabilities.

The use of data for policy and planning

The growing power of ICT based data applications and especially big data and AI are of increasing importance in education.

One use is in education policy and planning, providing near real-time intelligence in a wide number of areas including future numbers of school age children, school attendance, attainment, financial and resource provision and for TVET and Higher Education demand and provision in different subjects as well as providing insights into outcomes through for instance post-school trajectories and employment. More controversial issues is the use of educational data for comparing school performance, and by parents in choosing schools for their children.

Learning Analytics

A further rapid growth area is Learning Analytics (LA). LA has been defined as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” [Reference] It is seen as assisting in informing decisions in education systems, promoting personalized learning and enabling adaptive pedagogies and practices. At least in the initial stages of development and use, Universities and schools have tended to harvest existing data drawn from Virtual Learning Environments (VLEs) and to analyse that data to both predict individual performance and undertake interventions which can for instance reduce drop-out rates. Other potential benefits include that LA can, for instance, allow teachers and trainers to assess the usefulness of learning materials, to increase their understanding of the learning environment in order to improve it, and to intervene to advise and assist learners. Perhaps more importantly, it can assist learners in monitoring and understanding their own activities and interactions and participation in individual and collaborative learning processes and help them to reflect on their learning.

Pardo and Siemens (YEAR?) point out that “LA is a moral practice and needs to focus on understanding instead of measuring.” In this understanding:

“learners are central agents and collaborators, learner identity and performance are dynamic variables, learning success and performance is complex and multidimensional, data collection and processing needs to be done with total transparency.”

Although initially LA has tended to be based on large data sets already available in universities, school based LA applications are being developed using teacher inputted data. This can allow teachers and understanding of the progress of individual pupils and possible reasons for barriers to learning.

Gamification

Educational games have been around for some time. The gamification of educational materials and programmes is still in its infancy and likely to continue to advance.  Another educational technology due for a revival is the development and use of e-Portfolios, as lifelong learning becomes more of a reality and employers seek evidence of job seekers current skills and competence.

Bite sized Learning

A further response to the changing demands in the workplace and the need for new skills and competence is “bite–sized” learning through very short learning modules. A linked development is micro-credentialing be it through Digital Badges or other forms of accreditation.

Learning Spaces

As ICT is increasingly adopted within education there will be a growing trend for redesigning learning spaces to reflect the different ways in which education is organised and new pedagogic approaches to learning with ICT. This includes the development of “makerspaces”. A makerspace is a collaborative work space inside a school, library or separate public/private facility for making, learning, exploring and sharing. Makerspaces typically provide access to a variety of maker equipment including 3D printers, laser cutters, computer numerical control (CNC) machines, soldering irons and even sewing machines.

Augmented and Virtual Reality

Despite the hype around Augmented Reality (AR) and Virtual Reality (VR), the present impact on education appears limited although immersive environments are being used for training in TVET and augmented reality applications are being used in some occupational training. In the medium-term mixed reality may become more widely used in education.

Wearables

Similarly, there is some experimentation in the use of wearable devices for instance in drama and the arts but widespread use may be some time away.

Block Chain

The block chain has been developed for storing crypto currencies and is attracting interest form educational technologists. Block chain is basically a secure ledger allowing the secure recording of a chain of data transactions. It has been suggested as a solution to the verification and storage of qualifications and credentials in education and even for recording the development and adoption of Open Educational Resources. Despite this, usage in education is presently very limited and there are quite serious technical barriers to its development and wider use.

The growing power of ICT based data applications and especially big data and AI (see section 10, below) are of increasing importance in education.

The use of data for policy and planning

One use is in education policy and planning, providing near real-time intelligence in a wide number of areas including future numbers of school age children, school attendance, attainment, financial and resource provision and for TVET and Higher Education demand and provision in different subjects as well as providing insights into outcomes through for instance post-school trajectories and employment. More controversial issues is the use of educational data for comparing school performance, and by parents in choosing schools for their children.

Learning Analytics

A rapid growth area is Learning Analytics (LA). LA has been defined as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” [Reference] It is seen as assisting in informing decisions in education systems, promoting personalized learning and enabling adaptive pedagogies and practices. At least in the initial stages of development and use, Universities and schools have tended to harvest existing data drawn from Virtual Learning Environments (VLEs) and to analyse that data to both predict individual performance and undertake interventions which can for instance reduce drop-out rates. Other potential benefits include that LA can, for instance, allow teachers and trainers to assess the usefulness of learning materials, to increase their understanding of the learning environment in order to improve it, and to intervene to advise and assist learners. Perhaps more importantly, it can assist learners in monitoring and understanding their own activities and interactions and participation in individual and collaborative learning processes and help them to reflect on their learning.

Pardo and Siemens point out that “LA is a moral practice and needs to focus on understanding instead of measuring.” In this understanding:

“learners are central agents and collaborators, learner identity and performance are dynamic variables, learning success and performance is complex and multidimensional, data collection and processing needs to be done with total transparency.”

Although initially LA has tended to be based on large data sets already available in universities, school based LA applications are being developed using teacher in putted data. This can allow teachers and understanding of the progress of individual pupils and possible reasons for barriers to learning.

Artificial Intelligence

In research undertaken for this report, a number of interviewees raised the importance of Artificial Intelligence in education (although a number also believed it to be over hyped).

A recent report from the EU Joint Research Council (2018) says that:

“in the next years AI will change learning, teaching, and education. The speed of technological change will be very fast, and it will create high pressure to transform educational practices, institutions, and policies.”

It goes on to say AI will have:

“profound impacts on future labour markets, competence requirements, as well as in learning and teaching practices. As educational systems tend to adapt to the requirements of the industrial age, AI could make some functions of education obsolete and emphasize others. It may also enable new ways of teaching and learning.”

However, the report also considers that “How this potential is realized depends on how we understand learning, teaching and education in the emerging knowledge society and how we implement this understanding in practice.” Most importantly, the report says, “the level of meaningful activity—which in socio-cultural theories of learning underpins advanced forms of human intelligence and learning—remains beyond the current state of the AI art.”

Although AI systems are well suited to collecting informal evidence of skills, experience, and competence from open data sources, including social media, learner portfolios, and open badges, this creates both ethical and regulatory challenges. Furthermore, there is a danger that AI could actually replicate bad pedagogic approaches to learning.

The greatest potential of many of these technologies may be for informal and non-formal learning, raising the challenge of how to bring together informal and formal learning and to recognise the learning which occurs outside the classroom.

There was an excellent article by Andrew Smith in the Guardian newspaper last week. ‘Franken-algorithms: the deadly consequences of unpredictable code’, examines issues with our “our new algorithmic reality and the “growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on.” “Between the “dumb” fixed algorithms and true AI lies the problematic halfway house we’ve already entered with scarcely a thought and almost no debate, much less agreement as to aims, ethics, safety, best practice”, Smith says.

I was particularly interested in the changing understandings of what an algorithm is.

In the original understanding of an algorithm, says Andrew Smith, “an algorithm is a small, simple thing; a rule used to automate the treatment of a piece of data. If a happens, then do b; if not, then do c. This is the “if/then/else” logic of classical computing. If a user claims to be 18, allow them into the website; if not, print “Sorry, you must be 18 to enter”. At core, computer programs are bundles of such algorithms.” However, “Recent years have seen a more portentous and ambiguous meaning emerge, with the word “algorithm” taken to mean any large, complex decision-making software system; any means of taking an array of input – of data – and assessing it quickly, according to a given set of criteria (or “rules”).”

And this, of course is a problem, especially where algorithms, even if published, are not in the least transparent and with machine learning, constantly evolving.

Last November I facilitated a workshop at the European Skills Week event on research in vocational education and training. The workshop was entitled digitalisation in /of vocational education and training. There were some five of us in the workshop and we had about two hours to answer a series of questions based on the following framework.

Despite the too short time, I think what we came up with is a good starting point and the discussion will continue in a round table session at the European Conference on Educational Research in Bolzano, Italy in September.

Research Desiderata & Questions

The following central research questions and / or desiderata in this field were identified:

• How do processes of digital transitions and transformations impact on VET and what are the mediation processes and artefacts involved?
• Digital technologies are changing the nature and organisation of work, and the skills and competences required. This is happening simultaneously at a sectoral level and a global level. The new skills and competences are mediated in interactions between different actors but also between actors and objects. These processes of mediation to a large extent shape the practices of using digital technologies.
• In a critical appraisal of digitalisation in VET, what are the different possibilities for the future: What is and more importantly what could be?
• There is a tendency to take technologies and replicate past paradigms – hence for instance the idea of a ‘digital classroom’. Yet digital technologies open new possibilities for vocational education and training. To understand what ‘could be’ requires a critique of existing practices in VET and of the early adoption of technologies for teaching and learning.
• How do digital technologies and transformations affect the creation and meaning of work at a sectoral and global level?
• As technologies such as robotics and artificial intelligence are fast being adopted in different sectors and occupations, the future form of work and work organisation is being questioned. Alongside the digital transformations impacting in many sectors, sections of capitalism have advocated digital disruption based on new business models. The use of technology in this way raises Issues of social justice and values. What should be the role of VET in providing the skills and competences to shape the meaning and values of future work and innovation?

Explanation & Justification

Analytical Level

Macro Level

The changing nature of work due to the emergence of new technologies can potentially be shaped. To an extent how technology impacts on work is dependent on values. Equally digital transformations can build on existing skills and competences and older forms of knowledge. To understand these processes requires research at a sector level.

Technological unemployment should not be viewed as simply an issue requiring upskilling, but as questioning forms and organisation of work within society. Life skills are equally important in developing resilience for future employment.

We need a greater understanding of how old knowledge forms are transformed into new knowledge in the digital age.

Meso Level

Institutions mediate processes of skill and competence formation related to digitalisation. What is the relation between specific digital skills required in different sectors and occupations to basic and transversal digital skills? How can skills and knowledge acquired formally or informally in the workplace be linked to education and training in VET institutions.

At the same time, digitalisation provides new possibilities for teaching and learning, for example through augmented reality. This in turn requires the adoption of new pedagogic approaches for VET. Present practices in the adoption of Learning Management Systems form socio-tech systems and may prioritise or marginalise different skills and knowledge.

Micro Level

What are the skills and knowledge required not only to deal with and shape technology in the workplace (in different occupations and sectors) but also for living in the digital age? How does technology transform the work identity of individuals and how do individuals change their own identity for dealing with the changing world of work? What are the life skills that develop the residence required by individuals to deal with digitalisation at a societal level?

Analytical Focus

Learners / Students

Understanding the processes of digital transformation is critical to developing future oriented curricula for learners and students. At the same time, emergent technologies – such as robotics and artificial technologies – call into question existing societal forms of wage labour – once more requiring new curricula for life skills.

We need to focus not only on formal initial training in VET, but on informal learning in the work process leading to identity transformations.

Object / Process

Objects and artefacts play a key role in mediating learning in VET. These artefacts are themselves becoming transformed through digital technologies.

The use of technology opens up new possibilities and contexts for learning, including directly in the workplace. It also potentially empowers processes of social learning, with learners themselves acting as facilitators for other people’s learning and for developing and sharing knowledge within social settings.

This requires research for understanding how such social learning processes can be developed, how new forms of knowledge are acquired and what role objects and artefacts play in these processes.

Trainers / Teachers

There are many examples of good practice in the use of technology for learning in VET and of teachers and trainers sharing knowledge and experiences online. However, many teachers and trainers also feel left behind by the rapid changes in technologies both within occupations and for teaching and training.

Research suggests that best practices are not being generalised because existing models of professional development for teachers and trainers do not scale to meet needs.

An understanding of the possibilities for future VET, requires an understanding by teachers and trainers of the potentials of using technology in their own practice.

I like this video by Kate Raworth about Three Horizons Framework, created by Bill Sharpe, and presented as a useful tool for sharing with groups thinking about transformative change. I am not sure if I agree with their different categorisations but it does not really matter – the point is that the tool is only trying to get people thinking. And it overcomes the myth that the way we introduce technology – or any other change for that matter – is inevitable. We have choices to make about how society utilises technology. Those choices do not have to be soley in teh hands of the big corporations.

Finally the video is very well scripted and narrated. I particularly like the ideas for how the Framework could be used in a teaching and learning context

Interesting research from Harvard Business Review who have introduced the Digital Evolution Index  to trace the emergence of a “digital planet,” how physical interactions — in communications, social and political exchange, commerce, media and entertainment — are being displaced by digitally mediated ones.

They outline five “features of the global digital economy”:

• Digital players wield outsize market power.
• Digital technologies are poised to change the future of work
• Digital markets are uneven.
• Digital commerce must still contend with cash.
• Digital technology is widespread and spreading fast.

Each of these five features, they say, “contains both upsides and challenges. Moreover, how strongly each of them is felt varies depending on where you are in the world”

The report produces a map of counties digital development divided into four zones: Stand Out, Stall Out, Break Out, Watch Out.

But by far the most interesting comments come in the conclusions:

Digital innovators should recognize that public policy is essential to the success of the digital economy. Countries with high-performing digital sectors, such as those in the EU, typically have had strong government/policy involvement in shaping the digital economies.

This comes despite the popular business press obsession with so called digital disruption which poses public policy as a barrier to change and innovation.

According to the University Technical Colleges web site, new research released of 11 to 17-year-olds, commissioned by the Baker Dearing Educational Trust, the charity which promotes and supports University Technical Colleges (UTCs), reveals that over a third (36%) have no opportunity to learn about the latest technology in the classroom and over two thirds (67%) admit that they have not had the opportunity even to discuss a new tech or app idea with a teacher.

When asked about the tech skills they would like to learn the top five were:

Building apps (45%)
Creating Games (43%)
Virtual reality (38%)
Coding computer languages (34%)
Artificial intelligence (28%)