The real world impact of Serious Games
Game-based learning has become increasingly prevalent in academic and
corporate environments over the past ten years. The market is forecast to more
than triple in size between 2022 and 2028 to become a $50bn industry (Research
and Markets, 2021).
Although the pandemic was undoubtedly good news for this sector – as isolated
individuals found meaningful social connections in digital gameplay against
others – this growth has primarily been driven by research, which consistently
shows game-based learning to be highly effective for learner engagement and
impact. For example, studies by authors like Zafar et al (2022), Perotta et al
(2013) and Sitzmann (2011) have found that game-based learning when compared
to traditional training methods:
Improves knowledge retention by up to 90%
Improves training completion rates by more than 300%
Improves learner satisfaction by more than 80%
How to measure impact?
Despite these encouraging findings, the perennial challenge faced by most
training providers and Learning & Development (L&D) departments is to connect
the cause and effect of a particular initiative with a meaningful business impact.
This assumes that the training has overcome the initial challenge
supplier and the buyer is aligning effectively on intended outcomes. It can be
useful here to apply James Kirkpatrick’s (2016) model of training evaluation:
Level 1 – Reaction:
How did participants like the training?
Level 2 – Learning:
Did the participants learn something new?
Level 3 – Behaviour:
Did the participants’ behaviour change after and due to the training?
Level 4 – Results:
What business outcomes did the training bring for the organisation?
Level 1 assessment is quickly done with the familiar post-session “happy sheet.”
In their own right, enjoyable and engaging learning experiences are becoming
increasingly central to a positive employee experience (LinkedIn, 2021). Given
the immersive, competitive and social formats, satisfaction and engagement
scores for PBS games regularly exceed 90%. Comments like this follow many
courses:
"By far the best interactive training I've been involved in." - Armando Garcia, Visible Value Participant, 2022
This sort of feedback is more than just good marketing material: L&D buyers like
to be reassured that a solution has been enjoyable – especially in an era of
unengaged workforces – and are more likely to re-buy as a result. However, what
happens as a result of the training is more important.
Impact assessment should then progress to looking at knowledge acquisition,
Level 2 in the Kirkpatrick Model. This is best done with a post-training knowledge
retention survey. This is much easier for technical topics, where there is often a
“right answer,” than it is for leadership or strategy training, where the emphasis
falls on shifts in mindset. Though still self-assessed, answers to Level 2 type
questions – such as "what have you learnt today?" – are clearly more useful than
the Level 1 style of question. For example:
"A wonderful experience – financial terms are no longer 'Greek' to me and I have a better understanding of the impact my operational decisions have on the financial side of my division." - Renisha Singh, Bespoke course participant from JCB, 2021
In determining whether knowledge acquisition leads to behaviour changes (Level
3 in the Kirkpatrick Model), educators rely on wider and longer-term data points.
Factors like internal observation, coach/manager feedback, and additional selfassessment are often beyond the control of a training supplier. After each course, PBS asks all participants "As a result of this training, what are you going to do differently in your job?" Their responses then serve as a mini-action plan, which
should lead to relevant and trackable changes in behaviour. For example:
"As a result of the experience, I’m going to collaborate more with sales, supply chain and finance. We need a good marketing plan based on collaboration with sales in order to get their customer feedback and information on market conditions – as well as motivating them to sell the right products." - Marvin Mateo, Visible Value participant, 2022
Whilst learner engagement, knowledge retention and behaviour change might be
“Key Performance Indicators” of training impact, the holy grail remains a positive
effect on business results, Level 4 in the Kirkpatrick Model. Of course, connecting
cause and effect in organisations is rarely simple or linear.
Nevertheless, it is crucially important that the training supplier and buyer are
aligned on the real business outcomes the solutions seek to support. The regular
critique of HR, L&D and Talent departments is that they are not connected
enough to the business (Blind, 2018). Undertaking this alignment for every
solution will result in more win-win situations.
PBS’ serious games have a track record of contributing to measurable business
outcomes in some of the world’s largest businesses:
New joiners to Compass Group were taking too long to understand how their day-to-day decisions led to business outcomes. PBS has trained 1000s of Compass employees to attain a common understanding of "value," which has supported a 700% increase in share price since the program started in 2006.
A North American division of Ford was underperforming. The plant managers participated in a customised PBS commercial acumen experience. 12 months later, the division reported a $34 million increase in profits.
If a drug filing process for Roche is delayed, it can be worth up to $80 million per month in losses. Since working with PBS in 2014 to minimise this problem, Roche has completed all drug filings on time.
Seeing the business impact of a solution can take many months. This is especially
the case when the goal is people-related, such as reducing employee attrition,
increasing diversity, or even boosting the promotion rates of participants. In all
cases, therefore, it is important that clients track learning impacts related to
engagement, knowledge retention and behaviour change.
Almost all L&D buyers and suppliers like to claim that they are relentlessly
focused on the learning that impacts organisational outcomes. Even if objectives
can be agreed on, the measurement and attribution of actual impacts is
extremely challenging in this fast-moving, complex and interconnected
environments.
As ever, the unmeasurable "gut feeling" of a buyer about a solution can be just as
important. The longevity of PBS’ client relationships reflects its ability to operate
as one with the client whilst delivering on their ever-evolving needs.
The perspectives of the buyers themselves provide the best evidence of impact: "We were very impressed with the detail that PBS went into when designing our dealership simulation with us. They were eager to learn about our operating model, so they were well-placed to deliver the training too. This has achieved our aims of creating the understanding that we sought. Our dealer-facing team are now much better placed to support our dealers."
Jon Nixon, General Manager at JCB 2022
"We evaluated several partners to create our sustainable finance game. PBS had the most innovative and relevant approach to understanding and meeting our needs. Participants’ feedback has been incredibly positive. The game experience really engages participants and enables them to understand the complex challenges and interdependencies of sustainable finance in a playful and social way. We have found PBS to be an extremely capable, proactive and innovative partners and we look forward to expanding our relationship with them." Andrew Collier, L&D Director at PepsiCo 2022
"We decided to work with PBS to co-create our talent management game because they have a proven record in designing and delivering high-impact L&D solutions. Importantly, we knew that PBS would enter into a partnership with us and be able to design something that fit PepsiCo’s culture and unique context. The game has already resulted in higher learner engagement scores and significantly increased our Managers’ confidence and ability to make better talent decisions."
Klaus Woeste, CEO at Attain Learning 2022
#seriousgames #trainingimpact #learninganddevelopment #corporatetraining #experientiallearning #measuringtrainingimpact