Can AI really coach like a human? This live experiment put it to the test

A unique experiment recently took place: to see if an AI coach could be better than a real-life human one. HR Grapevine sat in on it to see what happened...
HR Grapevine
HR Grapevine | Executive Grapevine International Ltd
Human facing robot illustration
AI anxiety is growing across UK workplaces. Is it warranted?

Apocryphal predictions of AI taking over everyday jobs just keep on coming.

Most recently came the turn of Mustafa Suleyman, Microsoft’s CEO of AI. He’s someone who probably knows a thing or two about the power technology, and his near future-gazing vision is amongst the most worrying yet.

“Most of the tasks of white collar work,” he proclaimed, “will be fully automated by an AI within the next 12 to 18 months.”

Gulp.

The basis for this prediction is his assertion that technology is now finally racing towards what he calls “human level performance” – that is, where the decisions made by machines are undecipherable from those made by humans. Should this come to pass, US Senator Bernie Sanders has already said this would represent an “economic earthquake” and the beginning of mass unemployment.

Can coaching be replaced?

In world of HR of course, those who push-back on this Doomsday scenario point to the fact that machines don’t have, and never will have, the necessary ‘human’ qualities they have. These are things like compassion, or empathy, or being understanding of a different person’s point of view.

And nowhere is this more forcibly asserted than in the world of coaching – where being able to relate with someone, or share a perspective, or identify nuances in behaviour is seen as being essential to the advice and counsel they give. Today being a coach is not an easy endeavor. To get an Institute of Leadership and Management Level 7 Diploma in Coaching Supervision, learners have deliver 60 hours’ worth of assessed coaching alone, as well as study the topic and demonstrate competency in coaching through written assessments.

But what if AI really is level-pegging with everything any coach of many years’ experience and continual CPD could do?

Wouldn’t that be a real turning point?

Well maybe this time is coming sooner than you think.

Recently HR Grapevine sat-in on a unique, and potentially groundbreaking experiment, which specifically pitted an AI coach against a real life human coach – to really see whether an AI-Avatar coach give traditional in-person coaching a run for their money.

The experiment was devised by Michael Coppola, founder and CEO at myAgentsAI – the technology company that has developed its own AI ‘agent’ designed to be able to have complex conversations and run coaching sessions with managers/leaders suffering from particular workplace issues.

The warmth was not there from the AI. It wasn’t that the AI was bad though, and I noted how she was straight to the point, and then quickly introduced role plays

Dr Ionna Lordanou | Reader in Human Resource Management, Oxford Brookes University

Keen to show off how its agent performs, Coppola decided to do something that even he himself described as being a “massive risk” – pitting a human executive coach against its AI agent – completely unscripted, and with no safety net.

Oh, and just to add to the jeopardy, he then decided he’d stream it live on LinkedIn. And invite CPOs and HRDs and those from the wider coaching sector to watch-on.

What could possibly go wrong?

Here we bring you the results of this experiment:

How was the experiment set up?

To position AI agent against human, the experiment would feature a simulated problem, conducted once with a real life coach: Eli Blair - ICF Master Certified Coach with more than 5,000 hours under his belt doing senior leader coaching – and once by an AI coach, ‘Maya’.

They would each – in turn – talk to Daniel, an actor, who for the purposes of the test, was playing the part of ‘Jordan’, a classic ‘accidental manager’ who is having problems with one of his top performers.

Watching on from the sidelines are three ‘judges’ who will score how well they thought each coach did at getting to the heart of the problem, and providing advice and solutions. The three judges were:

- Dr Ionna Lordanou – reader in Human Resource Management at Oxford Brookes University. Lordanou teaches modules on adult learning and development, research methods, and caching and mentoring.
- Alice Maniezki – a leading expert in organisational psychology, focusing on talent assessment, leadership development, and organisational transformations and artificial intelligence. She holds a PhD in HR Psychology from the University of Valencia
- Yevgeniya Platun – an ICF ACC certified executive coach formerly EY’s chief of staff to global technology consulting leader.

Eli had his session with Jordan first – without having any preparation for what he might want to talk to him about. The actor then repeated a conversation with ‘Maya’.

In both cases, ‘Jordan’ summarised his problem as he saw it, and then responded depended on how the live conversation took him.

You've read 29% of the article so far, subscribe to continue reading - plus lots more!


Subscribe now to myGrapevine+ and get access to our comprehensive knowledge portal.


Already a subscriber?Sign in

Welcome Back

You might also like