How can learning leaders prove the value of training in an environment where budgets are tight and business impact matters more than ever?

In this episode of The Business of Learning, brought to you by VPS Learning, Paul Kent, director of functional development at PepsiCo, shares actionable strategies for measuring training ROI and building stronger alignment between learning and organizational goals.

More Resources:

• • [Course] Measuring the Impact of L&D Certificate
  • [Industry Coverage] ISO L&D Metrics Standards Provide Long-Awaited Framework for Training Measurement
  • [E-Book Preview] Learning Measurement for Modern Business: Making Sense of ISO 30437
  • [Article] Using AI to Measure 6 Levels of Training Impact

The transcript for this episode follows:

[Ad]

 This episode is brought to you by VPS Learning. Time to rethink how we’re asking people to learn and perform. The engineer who won’t engage, the operator who keeps doing it the old way, the manager who nods in the room but changes absolutely nothing tomorrow. Most organizations call that resistance. We call it proof that your current training isn’t working. At VPS Learning, we design learning that accounts for the human side of change, accelerating performance and driving lasting transformation. It’s time to perform beyond. Visit vpslearning.com to request your learning assessment  

Michelle Eggleston Schwartz : Hi. Welcome back to The Business of Learning. I’m Michelle Eggleston Schwartz, editor in chief here at Training Industry.

Sarah Gallo: And I’m Sarah Gallo, a senior editor. Today’s organizations are facing growing pressure to do more with less. And this often means learning leaders are being asked not only to deliver effective training, but also to really demonstrate its value to the business. As our listeners know, this is easier said than done, and it’s been a long-standing challenge in learning and development. So to learn more and offer some practical advice, we’re joined today by  Paul Kent, director of functional development at PepsiCo. So with that, Paul, thanks for sitting down with us today.

Paul Kent: Sure, delighted to.

Michelle Eggleston Schwartz: Yes, thank you for joining us. I’m super excited about this conversation today because measurement is a key challenge for learning leaders. So to start us off, how do you define training ROI today and how has your thinking evolved over time, kind of based on your experience in the field and what you’ve been seeing?

Paul Kent: Yeah. I think, you know, I think training ROI, because ROI is really popular as a term, right? And that’s what we’re always seeking. We’re always seeking for that financial number to link to the business. I think of it much more broadly than a strict financial equation. So early in the career, I fell into the trap of thinking of how much is this costing? How much are we saving, right? But actually, I’ve probably evolved more into a return on expectation and a view of impact. So, what’s really important is that in learning organizations, we set out with the impact in mind, we set out with what is the destination before we start the journey. And I … my biggest learning from doing this is that rather than looking for a financial number, it’s sitting down with the business and saying, what are we trying to achieve? What are we trying to improve? And we’ve looked at key areas, and that might be headcount loss, that might be employee movement, it might be a sustainability initiative and it even might be reducing risk. Now think about that of, you know, if you look at an ROI of reducing risk, it’s very hard to measure something that didn’t happen. because by providing the learning, you are ensuring that something doesn’t happen. How can you measure that as a return on investment, right? So I’ve spent money and something didn’t happen. Where’s the return? The return is on loss of profile loss of your image in the world as to how your organization runs. So, you know, things like food safety are critically important to us. Quality of our products are highly important, right? A loss in those areas is a huge risk. So, I’ve definitely moved away from that, ROI, evidence being the key thing. And more into that impact. Look now that. In some cases, it is much easier if you think of sales or negotiation, it’s much easier to, for us to measure a financial number. But often I think we get a little bit too bogged down now. That’s been my evolution of thinking as we’ve, as we’ve gone through my career.  Yeah. That’s so important. I love what you mentioned about kind of sitting down with the business with those key stakeholders at the beginning, and getting aligned and kind of starting with the destination in mind. I think that’s, that’s so huge from the get go.  And I think just on that point, that’s the part that, you know, if you look at learning as a service that we’ve seen in the past where, you know how this looks in in industry is someone comes to me and says, I want an eLearning. I want it to be 15 minutes long and I want a classroom course that’s for 50 people. Alright. If you look at the Henry Ford quote, if I asked them what they wanted, they would’ve said faster horses. Right? But actually, we as a learning function, think about the question that learning was often asking of the business. It was asking them, what are your learning needs for next year? Or What are your training needs for next year? That’s the wrong question. The right question is, what are your business priorities for next year? Because that’s their language. And … that language back to learning. So when we look at it, we start with the business need, business outcome, business metric, then we look at is there a learning initiative that is going to impact on that?  Because the first thing you have to be really brave about, look at our four Bs, and I can talk more about that later. But the first B in our four Bs of, bounce, borrow, buy, or build. Okay. Bounce is, this is not a learning need, right? Providing learning here will not solve your issue. And I think that’s critically important because if you do that at the very beginning with the business and you set the agenda, you set the destination and you work back and build learning objects that meet that need, then you work back to the learner, and the learner starts in the same place, they now use a learning object. But it’s a lot more taught out than. I wanted an eLearning, I’m not sure who it’s for, and I’m not sure what it’ll achieve, but let’s do that anyway. And it fails. Let’s do it again and again and again. Right. So now it’s really changed the narrative on that as well by and the business respect it more. If you can have that conversation, if you can get that, if you can make it measurable because it, what I found as well is that the business can often be vague in what they think their business priority is, right? It can be quite open. I want to increase empathy. Why? What will increasing empathy in my customer service representatives do? It’ll give us higher return rate of customers. There’s your business objective. Now let’s look at our learning. Will our learning do that? And can we then see that behavior of empathy increasing, which leads to the outcome.

Sarah Gallo: Definitely, getting specific about those goals is super important. I mentioned earlier that right now in today’s business environment. Budget scrutiny is high. Learning teams are often working with smaller budgets, not seeing, you know, the increased budget maybe they were hoping for. Kind of with all of that in mind, why do you think that prioritizing, return on investment and just really working to measure results is so critical right now?

Paul Kent: I think if, if you don’t, learning has no place. Because learning without the priority on the outcome, without the priority on measuring the right outcome, learning becomes a cost center. So if we, you know, if our measurable outcomes, this happens many times in organizations, that our measurable outcomes are hours of learning. Our reaction to training now, if business isn’t going to keep investing in that. For people to feel good about learning. Absolutely important that they do feel good about learning, but that’s on a path to something else and that something else is the outcome. So in a time of budget scrutiny, in a time where we really have to be a true partner to the business, we have to show value and not learning value. Not net promoter scores, not star ratings, but actually business value and actually what you’ll find is where you do show that true business value, you then reinvest the gain back into learning. Now that’s, you know, solid organizations are learning that very quickly. Is that where we do this properly, where we actually start with our outcome, meet our outcome. It’s increasing cycle time, it’s reducing friction; it’s reducing issues, which in itself has a monetary value, which means that learning then shows its value. But unless you link it to business, it becomes a challenge.

Michelle Eggleston Schwartz: Definitely, I’d love to dig into how should L&D leaders think about starting with the end goal in mind when measuring training impact, like, what does that look like and where does that concept of return on expectation that you had mentioned,  before kind of fit, fit in compared traditional ROI measures?

Paul Kent: Yeah, I think the conversation starts when L&D leaders actually stop trying to focus just on L&D and it’s being true business partners. You know, we talk a lot about L&D having a seat at the table. Why? You know, we’ve got to be really fair and critical on this of, well, you’ve got to earn your seat at the table. So I would, my suggestion to any L&D leader is you start small and focus. You don’t try to boil the ocean. You say, what are the big things that are a challenge for us right now, and going back to the four B’s model, if we get past bones, getting past saying, well actually, you know what, learning isn’t going to impact on that, where we can see the value saying, okay, I think that we can increase our on time in full orders by providing learning and the right learning to the right people at the right time. Then we show value to the business, and then we can look at, well, we’ve got content already. We can borrow it from other sources. We can build content, or we can buy content. Those completing the four Bs. But L&D leaders really do need to start by understanding what’s the context they’re working in, what is the problem, the business problem, business challenge that they’re trying to solve for will help them become a trusted partner.

Sarah Gallo: Definitely. Paul, our listeners love to hear those real-world examples and stories, and I’d love if we could kind of dig into some of those here. Kind of thinking about your experience and what you’ve seen, what are some of the most effective strategies that L&D teams can use to measure and prove impact? What have you found works for you?

Paul Kent: Yeah, I think when we’ve sat down and appreciated it, we’ve rebuilt our learning strategy and we moved from a learning as a service type mentality of, serving the business and ordering them to going to a model where truly the business start with their outcome. And the strategy we’ve deployed with example of, you know, I talked about customer services earlier when we defined six key areas that we wanted to look at, investment, those were sustainability, those were people movement. Those were absolutely ROI being one of them. We started to define the categories of where people would see value, the business understood it when we looked at having smart objectives around the outcomes for the business. So being really specific on how am I going to know if this works? Then business reacted to that because it was language, the understood we talked about on time in full orders for our customer service representatives. Okay? So if you think of a business challenge that came to us saying. We need to get to a higher percentage rate of on time in full where my customer gets all of their order in full. What we sought to do was understand, well why are they not getting their order on time and in full, right? So an L&D professional of the past and in many areas would seek to build learning to explain to people what on time in full meant, why it was important to the business, what people tend to do to make sure it happens; it’s divide of understanding well. We know that, right? I know that it’s important that my … it’s very logical. My customer, my consumer is getting their order on time in full. But what’s happening to prevent them from doing that? And that’s where we started with the problem statement, where we started with the what’s to solve we’re looking for here. And once we got that solved, we then, as I said, work back to say, okay, well if I provide an eLearning on why consumers and customers aren’t getting their orders on time and in full, will that change it? Often not on its own. So we broke down our learning into learning objects, smaller consumable learning objects that we then strung together to tell the story. So a video or an eLearning in isolation is most likely not going to solve a problem for you. Learning is often information plus activation to give you an outcome that you then check and test. Well, the activation part was not really happening. Alright, so you know the principles of, show me. Tell me. Let me. Ask me. So we were doing a lot of “show me.” “Tell me.” We were showing you why on time in full was important. We were telling you why it was important, and then the call to action for learning as people walked out the door, right? Picture your two-day, three-day learning seminar, and we’ve probably all seen it. As people walk out the door, we give them the call to action. you go back to the office, make sure you do all the things you learned about this week, they go back to the office and they open Outlook and they forget everything you’ve told them. And after two weeks, the Ebbinghaus curve has kicked in, the forgetting curve has happened and nothing really has changed. Right? And then we come back and say, well actually I didn’t use it. If you move to let me in the room, and you say, well, okay, I’m going to bring in an object here that’s going to give you experience that might be a simulation. worked with the business to understand, okay, tell me what’s going wrong. Give me a use case. Okay, I’m going to make a simulation or an activity out of that use case, that’s going to be an object. So now I’ve simulated the scenario or multiple scenarios, and typically you’ll find probably the top three things that we get wrong in, in a business area covers about 80% of our issues. So if we focus on those top three things and we do the Let me in the room. then we come back and we say, now ask me, where did you struggle? What did you need? Because in that room, you’ve got all the subject matter expertise. You’ve got other people who say, ah, yes, I’ve seen that before. This is how I did it. What you create is a bigger string of objects. That bundle together, give a better experience, but also become more impactful because I’ve had a chance to use that muscle memory of testing it. So that’s what we found is that, you know, we started back at that definition. We’re now at the point of the learning objects that we use to meet that definition. And then are they using them? So very basic, you test utilization, are people consuming content? they clicking on the e-learning? Are they getting to the end? How are they reacting to it? Do they like it even though I’ve been rather dismissive of checking how many people did it and whether they like it or not, part of an overall journey. It’s very relevant. Because if they don’t do it, they can’t have any impact. And then. If it falls down, we come back and say, well, okay, I can see where it fell down. We’re not getting a transfer of learning here. Behaviors haven’t changed and behaviors and we use the Kirkpatrick model a lot when we get to level three and we talk about behaviors. We look at an observable behavior. So rather than in the past or we got the level three, we measured behavior by saying, did you learn something? Invariably the attendee or the employee will say, yes. Do you feel confident? Yes, very confident because that’s the fastest way for me to get to the end of the survey. When we changed that and we asked the manager, did you observe a change in their behavior? I did not, because the manager is objective. So it’s an observable behavior at level three versus at level two. Did learning take place? Do I believe learning took place? Yes, I do. Did the behavior change? I have observed that. Yes, it did. Did we get the result we wanted?

[Ad]

As a training professional, your job is to effectively manage the business of learning. You probably tune in to this podcast to gain insights on L&D trends being used by some of the most innovative thought leaders in our market. But did you know that Training Industry also provides data-driven analysis and best practices through our premium research reports? Our entire catalog, including reports on deconstructing 70-20-10, learner preferences, the modern learner experience, and AI’s impact on L&D, just to name a few, can be found at trainingindustry.com/shopresearch. New insights create new ways for L&D to do business. Let Training Industry research reports take your training initiatives to new heights. Go to trainingindustry.com/shopresearch to view the entire catalog.  

Sarah Gallo: Yeah, great points. Kind of, working through all of those stages and it does get harder as you get to those more advanced levels for sure, of measurement. On the flip side of that, are there any approaches or common frameworks to measurement that you’ve seen not be effective?

Paul Kent: Yep, absolutely. Many, and I think I did all of them very well for very long. It’s all of the stuff that I think we know is not right. It’s the measurement of hours and people, the measurement of reaction and net promoter scores, net promoter score is a personal favorite of mine because, you know, we hung our hat on that for so long. My course is performing brilliantly. It’s got a world class net promoter score. But actually, so what? Is it having value or having impact? I don’t know. So I think any measures that we were doing that were in isolation. Right? Going back to Kirkpatrick and [we] talk a lot about level one, level two, and you mentioned it there, people not progressing beyond level two and they don’t, because there’s a perception that it’s difficult. But actually the perception that’s difficult is caused by the activity at the very start. It’s caused by not knowing what level four is. So if I take an analogy of, I drive, right? If I just get in my car and drive without knowing my destination, do I know I got where I wanted to go? I’ve ended up somewhere but was where I wanted to go. And that even sounds a bit ludicrous, right? But I’ve measured how many miles did I do and how many hours was I on the road? Fantastic. But was I trying to achieve? I don’t know. So I don’t know if those hours were good. I don’t know if where I got to is the right place. More likely when I get in the car, I have a map. Or I have in my internal brain map, I’ve set a destination. I get in the car, I drive to that destination. I know success because I got to where I wanted to be in the timeframe I wanted to be. But yet, in learning, we don’t set that destination. So the reason why the upper levels of measurement become difficult is because we don’t set the target. That’s why I’ve been steadfast in saying it over and over, even here. Of, that’s why you start with the result. That’s why you start with the outcome. That’s why you start with destination and all. [It] sounds very logical. I think everyone, anyone listening to this will say, of course you do. That makes loads of sense. But then I go back and say, but do you do that? And do you go back to the business and say, what do you mean by that? What do you mean I want to increase empathy? What value will that have to my organization? How can I measure that? I can’t measure empathy. I can’t say somebody is, 13% more empathetic than they were yesterday because they didn’t, eLearning doesn’t make any sense. What will the behavior at level three of them being more empathetic do for our customers or consumers that will add value to our business? And does that meet the outcome that we want or meet the challenge that we need?

Michelle Eggleston Schwartz: Definitely is, like you mentioned, so important to set that destination. Start with the end in mind, work backwards, like what behaviors are going to show that. Learning was effective. Like what changed? So it’s so, so important. For organizations that may not have mature measurement capabilities or maybe a robust data infrastructure, what are some realistic ways, that they can start demonstrating value?

Paul Kent: By starting really simply. I think that we always jump to investing budget in systems and over-exaggerating what we need to get to. Right? If you start with your end in mind… if you start with your goal and start with how do you measure it today? And that goes back to how do we know success, right? Are we using an Excel spreadsheet to say how many orders do we put through and then I want to increase my orders in that Excel spreadsheet. I can use pen and paper. I can be really simple in what I’m doing to tell the story. It doesn’t need to be a fully integrated data pullout of every single thing I’m doing, right? This can be really simple, and I think that’s what we do wrong as well, is we try to boil the ocean of, I want to meet all of these objectives, right? I want to boil the ocean, and I want to have a Power BI or insert tool here, dashboard that’s going to show all of this. What’s really interesting is if you look at those dashboards, and we did some analysis on this, 70% of the views on dashboards were done by the creator of the dashboard. So what that tells me is that these data points and data generation serve a purpose, not really for the audience that they’re meant to serve, but for people to put on slides and you know, it’s, and it’s typically the people who built them that don’t understand them and go into them often when I look at a lot of these data points. They don’t make sense. And I think that you can be very simplistic and there’s so many tools now freely available for us to, to do that. So I don’t think necessarily that it’s having all of these forward-thinking tools that are going to generate insights and you know it yourself. A lot of it is very simple. It’s back to basics, it’s what am I trying to do? Who are the people that I need to do it? When am I going to have the intervention and how am I going to measure what they do? And that can be done at any level.

Sarah Gallo: Definitely. While we’re talking about keeping it simple, we mentioned a few times that measurement just remains to be this common challenge for learning leaders over the years and, and many maybe aren’t as data savvy as others. So for people kind of struggling just to get started and who maybe feel overwhelmed with all of this, what advice do you have just for where to get started and where to begin?

Paul Kent”: Yeah, it is really, start with the basics, you know, it’s take one to two to three small objectives at the start of the year and show value and test it, right? We looked at an example where we wanted people to negotiate better, right? We identified, okay, how will we see that? Okay. What will we see in what they do? What behaviors will they show? How does it link to the job? Is it their job to negotiate? Because the other part of it is that in learning sometimes, we often target the wrong people for learning. So it’s about the right thing, the right place, the right time. And when you’re starting out on the journey, it’s, we were doing negotiation training for people in February. Now what we learned from that is when do people negotiate. Typically, contracts are negotiated towards the end of the year, so you’re providing learning for some. This is why we don’t, we don’t teach three-year-olds to drive a car so that when they get to 16, we say, now go drive the car. Remember, you learned a long time back. That was the old learning by appointment mentality. And the starting out is make sure your learning moves from being programmatic to being more adaptive. That it’s where people are when they need it, and. The interesting paradigm in that is I now want my learning to be personalized. Personalized, and I want it to be at scale, and I want it to be available when they’re available. Now, the old model of learning was we put the sage on a stage, we put the person in the room in front of everybody who gave them their knowledge, and that when that knowledge was needed, they would somehow go back to the inner reserves of their mind. And pull on that knowledge. Well, that doesn’t happen.  Science tells us that doesn’t happen. So for any organization, it’s starting out with what are the basic things I want to do? When do I need to do them? So when do I plan my learning intervention? How do I, how do I build and layer up that knowledge as we go through and how do I make sure it happens at the right time? One thing I’ve seen recently and I’m stealing from somebody else on LinkedIn who I can’t remember in reference now, but it’s, you think about how athletes train versus how organizations train, right? If an athlete trained the way an organization trained, they wouldn’t even get to the start line. This idea of I perform all the time; I have one small training intervention, and I continue to perform again, versus an athlete who trains. Trains again, performs again, trains more, performs more, and it’s a constant flow. And that’s the, you know, that’s the adaptive nature of learning is that as an L&D team, we’ve got to build these objects that are available to people at the right time and point them to it and understand when the intervention is needed. Working with the business and direct people to it, about the athlete or the employee. Taking that training or learning at the right time and then using it to perform and test and come back and train again. So it’s about just taking those sharp ideas of what do I need, what do I want to achieve, and what’s my goal? And I’m amazed the more and more I look into this at how easy it sounds to start with the end in mind, but how little we do it.

Michelle Eggleston Schwartz:  Definitely, because as you noted, learning must be continuous. You must continuously learn and apply and reinforce. And so it’s this very continuous cycle and it all kind of starts with keeping that end in mind.

Paul Kent: Yeah, and even like you mentioned, continuous. Because Josh Bersin has a model where learning moves from programmatic to adaptive to continuous and empowering and you do that when learning has success. You do that when learning adds value. I will learn something if it adds value to me. Right? So, you know, AI is a big one at the moment, and everyone’s talking about AI and look at how many people are self-serving in learning AI because they’ve now realized they needed it. And when learning becomes continuous, it’s okay, I’m learning the next thing on AI and I’m learning the next tool. And when you get to that continuous empowering level with learning, it’s great because you’ve created lifelong learners, you’ve created people who are seeking, okay, I need to learn more. I need to improve more, and I want to constantly cycle and improve. Versus an old model of is a program that you sit on, and you study and you do a test and you do a multiple-choice questionnaire. If you get eight out of 10, you’ve passed it. We’ve moved well beyond that in organizational thinking.

Michelle Eggleston Schwartz: Definitely. Well, on that note, it has been great having you on the podcast today, Paul. How can our listeners get in touch with you after the episode if they like to reach out?

Paul Kent: Absolutely. I no doubt once this comes out, I’ll repost it. So LinkedIn is probably the fastest way to get to me, so jump onto my LinkedIn. I am pretty active on there. You can catch me on there. Send me a message and I’m delighted to talk to people and give them my view on it. But LinkedIn is probably the best way to get to me.

Sarah Gallo: Okay, awesome. From more resources on training measurement, check out the description or visit the shownotes on our website at TrainingIndustry.com/podcast, and don’t forget to rate and review us wherever you tune in to The Business of Learning. Until next time.