Introduction
Over the past 7 years, I have focused on supporting organizations that are implementing successful workflow learning projects. I started my journey in 2015 when I recognized that many organizations were talking about implementing 70-20-10 as a core approach to design and development of their blended learning solutions, but in reality, they only developed solutions with components in “10” (classroom training and eLearning) and “20” (some forms of coaching and/or management support). In fact, we could better describe these solutions as “blended training solutions” instead of “blended learning solutions”. In addition, these organizations were still measuring learning consumption (number of classroom and eLearning hours) and a bit of learning outcomes (whether participants passed the test and showed some behavioral improvement) instead of truly measuring the impact on performance improvement. In this article, I present two examples of healthcare organizations that worked on designing for their target groups’ full learning and performance journeys and the impact that designing for all 5 Moments of Need (5 MoN) had on their ability to measure the true impact of these workflow learning solutions.
What Do We Mean by “Impact of Workflow Learning”?
The impact of learning differs per organization and sector. Organizational and project goals determine the specific impact that needs to be achieved to consider a workflow learning solution successful. Each organization has its own interpretation of KPIs and objectives regarding workflow learning, but what they all have in common is that the impact of workflow learning is related to employee and organizational performance.
The L&D profession needs to understand that this approach is completely different from measuring formal learning impact (e.g., of training or eLearning, outside of the work context) which, in most cases, focuses on the outcomes of learning, like the number of participants, time to complete the course, or scores from a test. But measuring these learning or training outcomes is completely different than measuring the impact of learning. The outcomes of training show the capability of the employee to demonstrate knowledge and skills outside the work context. The impact of learning shows to what extent the employee can actually apply what they learned while performing their day-to-day job. That is the impact on their behavior and more importantly the impact on the business and organizational goals.
Why do we as L&D organizations want to measure the impact of our efforts?
- To know if the project goals/KPIs that we set at the beginning of a project are met
- To create an ROI case for management to legitimize the organization’s investments vs. return on those investments
- To show impact of workflow learning solutions to other stakeholders and demonstrate the power of the performance mindset/5 MoN solutions
- To get insights on how to continuously improve our 5 MoN solutions and Digital Coaches
- To evaluate what we can do better in future 5 MoN workflow learning projects
Measuring the Impact of Workflow Learning for Employees at the Alrijne Healthcare Organization
Dutch healthcare organization Alrijne started its 5 MoN journey in 2017. After a successful pilot on mathematics for nurses, a Digital Coach with the name “AskAlrijne” was created and successfully implemented. Over the past 5 years, the Alrijne Academy has further expanded by creating and implementing workflow learning solutions based on the 5 MoN. These solutions currently support the majority of over four thousand employees at Alrijne Healthcare.
Caroline Luijbregts, L&D professional at the Alrijne Academy, explains her team’s understanding of the impact of their 5 MoN workflow learning solutions:
“Impact is realized when learning in the organization improves. An important indicator for this is the support employees experience from the learning programs we offer them. To what extent do these help in their development as professionals? In addition, the confidence that patients have in the competence of the employee is also an important indicator. Besides that, we measure what impacts there are on employee or team performance.”
In quantitative measures of the number of users working with AskAlrijne, we see a significant year-over-year increase, and we also find that users come back to AskAlrijne more and more often. This year, almost 2,500 users have already generated more than 100,000 page views in this workplace learning solution. But there are also insights into the real business impact of Alrijne’s 5 MoN solutions (including AskAlrijne). One such insight shows that hundreds of nurses save 60-120 minutes per year by using a Digital Coach to learn how to care for vulnerable elderly and prevent delirium, fall accidents, and malnutrition—on the job (instead of taking a traditional eLearning course). This 5 MoN solution for moments 1 (Learn New) and 2 (Learn More) focuses on awareness around care and preventing incidents. Required patient information and scores to analyze and interpret their risk for delirium, malnutrition, fall incidents, and physical problems were accessed more often through AskAlrijne.
For the Alrijne organization, whose goal is to be a good and caring employer, it is also important that employees feel supported by the information and assistance the Digital Coach provides. Alrijne uses the Net Promoter approach to survey nurses participating in workflow learning projects. About 90% answer that they feel better supported by their employer now that they have access to AskAlrijne as they work. AskAlrijne helps employees feel more confident during their work, so they are better able to perform tasks independently.
The Impact of Workflow Learning during COVID-19 at Erasmus MC
In March 2020, as the impact of COVID-19 stretched far and wide, many organizations faced the urgent challenge of rapidly bringing their face-to-face training programs into the digital world. Among them is the largest hospital in the Netherlands, Erasmus MC, which was challenged by how to quickly equip new, existing, and returning staff (paramedics, doctors, nurses, and caregivers) with the skills to handle the ever-increasing number of patients in new COVID-19 units, emergency rooms, and ICUs.
With the new COVID-19 restrictions and enormous time pressures, it was clear that traditional ways of upskilling via classroom training, eLearning, practical work assignments, and on-the-job coaching were not going to work. So, Erasmus MC, which was already doing some 5 MoN pilot projects, decided that the only way forward was to create a powerful workflow learning and performance support solution. Based on the 5 MoN framework, in just 10 days, the L&D team completed the design and development of an innovative Digital Coach—AskErasmus— that supports 9 key healthcare roles with all the information and support required to care for COVID-19 ICU patients.
By applying the 5 MoN with a powerful Digital Coach, doctors, nurses, paramedics, and ICU staff could obtain new and updated treatment and care practices in just 2 clicks/10 seconds in their flow of work. The impact on the organization was huge: almost 2,300 employees used AskErasmus to perform their jobs in the first three months it was available.
Beyond this internal impact, this platform was provided to other healthcare organizations as well. With an additional 2,000 external users, AskErasmus reached a total of over 173,000 page views in its first three months. The reactions from healthcare employees both inside and outside Erasmus MC were enthusiastic and grateful.
For more details about this effort, keep reading here.