{"id":41426,"date":"2019-07-29T14:15:12","date_gmt":"2019-07-29T18:15:12","guid":{"rendered":"https:\/\/trainingindustry.com\/?post_type=articles&#038;p=41426"},"modified":"2024-07-19T11:20:55","modified_gmt":"2024-07-19T15:20:55","slug":"the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi","status":"publish","type":"articles","link":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/","title":{"rendered":"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI"},"author":27,"featured_media":41429,"template":"","tags":[2902,1461,2590,3319,3635,2467,2766,3438,124,2324,2672,3624],"class_list":["post-41426","articles","type-articles","status-publish","has-post-thumbnail","hentry","tag-data-and-measurement","tag-evaluation","tag-learning-evaluation","tag-learning-measurement","tag-measurement-strategy","tag-measuring-impact","tag-podcast","tag-return-on-investment","tag-roi","tag-training-evaluation","tag-training-measurement","tag-training-roi","global_topic_tax-measurement-and-analytics"],"acf":{"sponsored":false,"gated":false,"gated_content_type":"file","file_attachment":42248,"gated_content":"","form_instruction_header":"Fill out the form below to download our free e-book \"Expert Perspectives on Measurement and Evaluation.\"","pardot_html_embed":"<iframe src=\"https:\/\/www2.trainingindustry.com\/l\/186152\/2019-07-29\/j2fvvl\" width=\"100%\" height=\"600\" type=\"text\/html\" frameborder=\"0\" allowTransparency=\"true\"><\/iframe>","author_override":true,"author_name":"Training Industry, Inc.","author_image":"","author_bio":"","excerpt":"Training Industry research has found that one of the biggest challenges facing learning leaders is measuring the effectiveness of their training. On this episode of\u00a0The Business of Learning, Kevin M. Yates discusses this challenge and shares tips.","main_content":"Training Industry research has found that one of the biggest challenges facing learning leaders is measuring the effectiveness of their training. On this episode of\u00a0The Business of Learning, Kevin M. Yates, a fact-finder for learning and development, discusses this challenge and his tips on evaluating learning and measuring return on investment (ROI).\r\n\r\nFind out:\r\n<ul>\r\n \t<li>Why measurement is challenging.<\/li>\r\n \t<li>The pros and cons of common forms of measurement.<\/li>\r\n \t<li>Why smile sheets get a bad rap.<\/li>\r\n \t<li>Whether we can boil down learning programs to ROI.<\/li>\r\n<\/ul>\r\nDon't miss Kevin's article <a href=\"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/theres-a-data-analyst-on-the-ld-team\/\" target=\"_blank\" rel=\"noopener\">\u201cThere's a Data Analyst on the L&amp;D Team?\u201d<\/a> on TrainingIndustry.com.\r\n\r\nListen Now:\r\n\r\n<iframe style=\"border: none;\" title=\"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI\" src=\"https:\/\/www.podbean.com\/media\/player\/3ujdx-e4217b?from=pb6admin&amp;download=1&amp;version=1&amp;auto=0&amp;share=1&amp;download=1&amp;rtl=0&amp;fonts=Helvetica&amp;skin=1&amp;pfauth=&amp;btn-skin=107\" width=\"100%\" height=\"122\" scrolling=\"no\" data-name=\"pb-iframe-player\" data-mce-fragment=\"1\"><\/iframe>\r\n\r\n<em>To learn more about Training Industry's Measuring the Impact of L&amp;D Certificate course, download the program brochure below.\u00a0<\/em>\r\n\r\n[hubspot type=\"form\" portal=\"47185625\" id=\"6a5894de-a352-4435-b396-ee26fdd9fe79\" version=\"v4\"]","full_width":false,"content_band":[{"acf_fc_layout":"content_area","wysiwyg":"A transcript of the episode follows:\r\n\r\nIntro:\r\nWelcome to The Business of Learning, the Learning Leader\u2019s podcast from trainingindustry.com.\r\n\r\nTaryn Oesch:\r\nHello and welcome to Episode 19 of The Business of Learning, the Learning Leader\u2019s podcast from Training Industry. I\u2019m Taryn Oesch, managing editor of digital content, here with my co-host, Scott Rutherford, head of digital operations and marketing.\r\n\r\nScott Rutherford:\r\nHi. Training Industry research has found that one of the biggest challenges facing learning leaders is measuring the effectiveness of their training, and so that\u2019s the focus of this episode: the role of measurement in corporate training.\r\n\r\nScott Rutherford:\r\nToday on The Business of Learning podcast we\u2019re talking about measurement in L&amp;D, and here today to help us with this challenge is Kevin M. Yates.\r\n\r\nTaryn Oesch:\r\nKevin is a fact finder for learning and development. As the Sherlock Holmes of L&amp;D, he uses facts, evidence and data to answer the question, \u201cDid training work?\u201d His work is focused on using measurement to show training, learning and talent development\u2019s impact on performance, behavior and organization goals. He\u2019s also a contributor to trainingindustry.com.\r\n\r\nTaryn Oesch:\r\nKevin, thanks for joining us today.\r\n\r\nKevin Yates:\r\nWow, and thank you for that wonderful introduction. Boy, do I sound good?!\r\n\r\nScott Rutherford:\r\nYou know, we try.\r\n\r\nScott Rutherford:\r\nSo Kevin, can you start us off a little bit by just giving us a little background about yourself and what you do?\r\n\r\nKevin Yates:\r\nYes, absolutely.\r\n\r\nKevin Yates:\r\nI like to say, Scott and Taryn, that I grew up in L&amp;D. I\u2019ve been in L&amp;D now for about 20 years, and my career in learning and development started at a small community bank on the south side of Chicago, where I had a role as a trainer, a day-to-day facilitator. That was one of those boots-on-the-ground roles where, almost eight hours a day, I was in front of a class and training on bank software and customer service, and then that ultimately led to a role in instructional design that led to a role in curriculum and development. Then I moved into a role of leadership where I was managing trainers and instructional designers that opened up to a global learning role where I had the opportunity to spend some time in Bangalore, India, setting up training academies. I came back to Chicago and then started working with learning solutions and learning technology.\r\n\r\nKevin Yates:\r\nThen about three quarters of the way, I guess, into my career, I was introduced to this idea that you can measure the impact of training and more specifically, that you can measure the impact of training on people\u2019s performance and organization[al] goals. So that was maybe six or seven or eight years ago, and that was a game changer for me. It literally gave me reason to re-shift and re-focus my career, very narrowly, and very specifically in the area of measurement for learning and development, and then more specifically data and analytics for learning and development as well.\r\n\r\nKevin Yates:\r\nThat\u2019s where I\u2019ve been focused for the past few years. I think I have found my niche, if you will. I am just really enjoying the opportunity I have to be a voice in the L&amp;D community for the idea that we can measure the impact of learning, that we can use fact based evidence to answer the question, \u201cDid training work?\u201d\r\n\r\nKevin Yates:\r\nSo I think that I have found the right spot for me, and for my career and my work.\r\n\r\nTaryn Oesch:\r\n[It] certainly seems like it.\r\n\r\nTaryn Oesch:\r\nSo Kevin, when we talk about measurement in learning and development, what exactly are we measuring and, why? And what are some of the common forms of course measurement? We talk a lot about smile sheets, and what are some of the other forums and what are their pros and cons?\r\n\r\nKevin Yates:\r\nYes, that\u2019s a great question.\r\n\r\nKevin Yates:\r\nFor me, to answer that question, \u201cWhat are we measuring and why?\u201d I think it\u2019s pretty simple. We are measuring the extent to which our training and our learning solutions are changing anything at all, right? So it\u2019s measuring and determining the extent to which our training and learning solutions are actually changing people\u2019s behavior, changing their performance and changing their actions.\r\n\r\nKevin Yates:\r\nAnd then it\u2019s measuring the extent to which a change in behavior, performance and actions are actually impacting their performance in a way that helps them achieve business goals or organization[al] goals.\r\n\r\nKevin Yates:\r\nSo, we want to take a look at the link between learning, the link between performance and the link between actual impact on an organization goal or a strategy. So, we want to determine the extent to which learning may be helping people execute in their role in a way that helps them execute on a particular business strategy.\r\n\r\nKevin Yates:\r\nSo, that\u2019s the \u201cwhy.\u201d Simply put, we\u2019re doing it because we need insight on the extent to which our work, our effort [and] our resources are actually producing a change or actually making an impact. So for me, when I am measuring at the highest level, I\u2019m really taking a look at the extent to which there is metrics and data in the business that shows how people are performing.\r\n\r\nKevin Yates:\r\nThere are different ways to get at that. We can take a look at actual business performance metrics, and we can take a look at causation and correlation that we could show between performance and learning and training experiences that people have had. So they\u2019re ... Taryn you asked, \"What are we measuring? What are some specific measures that we\u2019re looking at that?\"\r\n\r\nKevin Yates:\r\nThat\u2019s just one example. There are so many different measures that we can look at and what we measure is, and how we measure, is really driven by where we expect to see an impact because you can\u2019t measure the same thing the same way over and over. You really have to be looking at where you expect to see an impact and then take a look at what measures will determine the extent to which that impact was made.\r\n\r\nKevin Yates:\r\nYou reference smile sheets. Smile sheets are getting a bad rep, and I think that\u2019s because of the traditional kinds of questions that smile sheets have asked. The traditional smile sheet asks, \u201cDid you like the instructor? Did you like the classroom, and did you like the food?\u201d That won\u2019t give you any insight into the extent to which you can expect to see a performance change. I\u2019ve developed about \u2026 I think it\u2019s 92 \u2026 questions that you can use to estimate the extent to which you can expect to see an impact on people\u2019s behavior [and] performance based on their learning experience. It\u2019s not that smile sheets are bad, it\u2019s just that smile sheets are [not] always, or have traditionally [failed to] ask the right kinds of questions that get at the right type of data, that provides insight or that informs decisions.\r\n\r\nKevin Yates:\r\nDoes that make sense?\r\n\r\nScott Rutherford:\r\nYes, I think what you were saying, if I can interpret a little bit ... a little bit of my own observation from folks that I\u2019ve spoken with on this topic too, but it\u2019s the relationship of how you\u2019re measuring and what you\u2019re measuring, and how those interrelate, because there is ... the smile sheet is used as an example or at least I\u2019ve seen it use as an example of easy measurement. It\u2019s an easy measure. It\u2019s easily administered, it\u2019s easily compiled, but it\u2019s not necessarily meaningful on the other side.\r\n\r\nScott Rutherford:\r\nThat\u2019s the balance, isn\u2019t it, to come up with measurement that\u2019s both meaningful and isn\u2019t too difficult to assemble or to execute?\r\n\r\nKevin Yates:\r\nAnd that it\u2019s not too burdensome to the person from whom you need to collect that data, right?\r\n\r\nScott Rutherford:\r\nRight.\r\n\r\nKevin Yates:\r\nIt\u2019s not that smile sheets are a bad thing, but you just need to use people\u2019s time wisely and you need to take advantage of that limited time to get the right kind of data. So not being ... I\u2019m thinking we should stay away from some of those gratuitous questions that really won\u2019t provide the type of insight that informs decisions and allows us to act. But again, to your point, we should be using that precious time that we\u2019re going to ask people to give, in that three to five minute survey, or however else it is, but pushing that out, to ask the kinds of questions that are going to inform decisions and provide insights. For me. I like to use that time to collect data that gives me insight into the extent to which we can expect a change in behavior and performance in action. So the types of questions that I ask are the ones that give us just that.\r\n\r\nKevin Yates:\r\nDoes that make sense?\r\n\r\nScott Rutherford:\r\nIt does.\r\n\r\nScott Rutherford:\r\nI\u2019m going to ask you maybe to be a little \u201ccontroversial\u201d here, and I\u2019m putting that in quotes.\r\n\r\nKevin Yates:\r\nI\u2019m all for that.\r\n\r\nScott Rutherford:\r\nIs there ... does it matter to assess whether learners liked the program or does it, does that ... is that really not important? Is it more important to focus in on metrics that can really help the business leaders understand whether the program was effective? Is there a role for a happiness score?\r\n\r\nKevin Yates:\r\nYes.\r\n\r\nKevin Yates:\r\nI\u2019m going to be controversial, and my own view is, not necessarily. I think that for me, at the end of the day, it\u2019s important to determine the extent to which content and experience impacts or has the potential to impact a person\u2019s change in behavior and performance. So there might be a correlation there where someone doesn\u2019t \u201clike\u201d something, I\u2019m using air quotes, that might influence the extent to which there will be a change. But I think that you have to separate like from expectations.\r\n\r\nKevin Yates:\r\nIf the expectations for the purpose of the training and learning solution is made clear, if that is clearly expressed so that those who are engaged in that training and that learning solution have very clear ideas about its purpose and its intent, then I think that\u2019s where you want to be looking at the extent to which there\u2019s connection with that, and not so much the extent to which there\u2019s a connection between whether or not someone likes something. At the end of the day, there are performance expectations that organizations have with people and teams, so that whole idea of liking something, I\u2019m not so sure that it is as important as content resonating with people in a way where they can see the connection between it and what the expectation is for how they are expected to use that training, experience in that learning, experience to impact their behavior in a performance and their actions.\r\n\r\nKevin Yates:\r\nDid that answer your question? Does it make sense?\r\n\r\nScott Rutherford:\r\nYes, I think that makes sense.\r\n\r\nTaryn Oesch:\r\nSo moving to another big, I guess, I want to say buzzword, but [another] popular idea right now with training measurement: Can we boil down learning programs to return on investment (ROI)? And if not, how do we demonstrate the value of a learning program to the business in terms that the management team will understand [so that] that they really go beyond ROI or just dollars and cents?\r\n\r\nKevin Yates:\r\nThat is an awesome question.\r\n\r\nKevin Yates:\r\nWhen I talk about return on investment (ROI), I always like to begin the conversation by contextualizing what I mean when I say ROI, because it means different things to different people.\r\n\r\nKevin Yates:\r\nSo, when I speak about ROI, return on investment, I am contextualizing that, to mean the monetary value you gained or lost as a result of an investment in people\u2019s talent and people\u2019s development. I\u2019m talking dollar to dollar.\r\n\r\nKevin Yates:\r\nThe first thing that I think we want to think about is how far you want to go with measuring the financial ROI of a training solution or a training program, and I would say, it really depends on the extent to which that solution or that program is visible and the extent to which it is strategically connected to a business goal, and the extent to which that program is visible in the organization. So for those programs that are highly visible, that are expensive, and that have a direct link to achieving a business goal or executing a business strategy, I believe it\u2019s important for us to measure the return on investment, the ROI, for that particular training experience or learning solution.\r\n\r\nKevin Yates:\r\nI don\u2019t think it\u2019s realistic to expect that we should be measuring the ROI for every learning solution and training solution that comes out of L&amp;D. It\u2019s not practical and quite frankly, we don\u2019t have the resources to do that, but if the organization is heavily invested in a particular training program or learning solution, if it is expected to impact a large number of people, and again, if it\u2019s highly visible in the organization, I believe that we have the responsibility to show the return on investment, the monetary return on investment for those types of programs that are in the business.\r\n\r\nKevin Yates:\r\nSo then when we are able to identify the return on investment, not only does it show our impact, but we are also then speaking the language of the business. Other parts of the business are already being held accountable for [their] return on investment, and again, I mean, the monetary return on investment, so I don\u2019t think it\u2019s unrealistic to expect that learning and development should be going the same thing.\r\n\r\nScott Rutherford:\r\nOn the topic of ROI, and looking at it from the perspective of a business owner, there are parts of a business where you could look at ROI in an immediate term. You could immediately know ... if you\u2019re doing a point of sale promotion in a retail circumstance, you can understand what the ROI of a two week point of sale promotion would be.\r\n\r\nScott Rutherford:\r\nI\u2019m wondering if you agree with what I think about ROI in L&amp;D \u2014 to use a mess of acronyms there \u2014 return on investment in learning and development ... it\u2019s a time function as well. Behavior change is a time function and I don\u2019t know if it\u2019s possible for a training program, or let\u2019s take it to the extreme, you can\u2019t measure or can you, ROI from a single course. Isn\u2019t it something that has to be measured in performance change over time?\r\n\r\nKevin Yates:\r\nYou are so spot on and I think that what you\u2019re hitting on there, Scott, is a way in which we need to manage our expectations of the business, the organizations where we serve and we need to manage expectations for ourselves. Because, as you said so eloquently, and so accurately, the essence of what we\u2019re talking about that comes out of a training or learning experience, is a change in behavior. If you\u2019re looking for signs or evidence or facts that show the extent to which changes in behavior and performance have impacted some type of business goal, that\u2019s going to happen over time.\r\n\r\nKevin Yates:\r\nSo what I am saying, and what I have learned from my experience, is that if you want to really see that change, and you want to see how that change has made an impact, you\u2019re really talking, nine to 12 months post-experience. That\u2019s even keeping the idea that the performance is changing, and the performance is ongoing after that learning experience.\r\n\r\nKevin Yates:\r\nSo, that training experience ... you are absolutely right in that, more often than not, you\u2019re not going to see an immediate return on investment. That\u2019s usually going to happen over time. The caveat to that would be those instances where depending on industry, and depending on the work in that industry, you might see some immediate return on investment.\r\n\r\nKevin Yates:\r\nFor example, if you are in the service industry, at a manufacturing industry and you have employees going through a training program that shows them how to execute on a performance, or rather a process, you can see that [ROI] pretty immediately, right? Because if it\u2019s a rote process on what somebody has been trained, and it\u2019s a repeated process, you can measure this to the extent to which they have that process down right. But when you\u2019re talking about some of those higher level, performance-type situations where you might be talking about leadership, or even some types of sales training, yes, you\u2019re so right, Scott.\r\n\r\nKevin Yates:\r\nYou\u2019re talking, six, nine, 12 months to really measure the real impact and measure the real return on investment, the ROI.\r\n\r\nKevin Yates:\r\nDoes that make sense?\r\n\r\nScott Rutherford:\r\nYes, I think so. I think that it also hints at one of the challenges in reporting ROI, because one of the reasons that any business function wants to focus on ROI is you\u2019re fighting for your place at the table of budget time. So if we\u2019re talking about a measurement cycle that extends to nine, 12, 18 [or] 24 months, we\u2019re talking multiple budget cycles and, at some point, you have to build the trust with management. Don\u2019t you have to say, \u201cWell, look, we have confidence what we\u2019re doing is going to bear fruit in the two year, three year, five year timeframe,\u201d so that there\u2019s the support and patience to allow it to mature.\r\n\r\nKevin Yates:\r\nYou\u2019re right and it\u2019s also a matter of building trust and showing evidence for approach.\r\n\r\nKevin Yates:\r\nSo here\u2019s what I mean by that: You are spot on in that those cycles don\u2019t always line up with when we expect to see that change, but we\u2019re talking about a culture change. We\u2019re talking about a change in mindset, and we\u2019re talking about a change in the way that we work. So, if over time you are repeatedly using credible, reliable methodologies to show return on investment for the impact of your training and your learning programs, then you have shown and demonstrated to the organization that you are working in a way that is aligned to how the business is working with using ROI.\r\n\r\nKevin Yates:\r\nSo what I mean by that, Scott, is if you are initially in that ROI journey and that measurement journey, it might feel as though there\u2019s a disconnect between the budget time cycle and the time for which you are able to show impact, but if you are continuously doing that over time, if you are [an] organization or if you are [a] L&amp;D organization, that is where repeatedly using ROI methodologies to show impact and provide insights on the extent to which your training and your learning solutions are actually producing return on investment. If you\u2019re engaged in that over a period of years, then you have been building trust and, so then, when those budget cycles come up, you will have already demonstrated the way in which you are measuring ROI.\r\n\r\nKevin Yates:\r\nSo it might not be as difficult to get those dollars, because you\u2019ve already shown to the organization that you are diligent in the way in which you are calculating ROI and the way in which you are showing ROI.\r\n\r\nKevin Yates:\r\nDoes that make sense?\r\n\r\nScott Rutherford:\r\nOh, for sure. Yes, and I want to take maybe three steps back because I was listening as you were talking about your own career path and it occurred to me that your entry to learning and development is, it\u2019s a trajectory that I\u2019ve seen and heard before. And I\u2019m wondering if there\u2019s if there\u2019s anything we can learn from that to say... [the best way to navigate that challenge] because, learning leaders, if we wanted to paint [them] with a very broad brush, typically would cite measurement as a challenge, as an area that they maybe struggle with a little bit.\r\n\r\nScott Rutherford:\r\nIs that, do you think, related to the fact that going into a learning or into a training career path \u2026 maybe the business savvy or the business focus [isn\u2019t always there] or [maybe] there\u2019s a skill set that\u2019s required for measurement that doesn\u2019t necessarily get [taught] ... that you\u2019re not prepped for when you\u2019re starting your first position?\r\n\r\nKevin Yates:\r\nYou\u2019re absolutely right.\r\n\r\nKevin Yates:\r\nFor me, when I think about it, this idea that we measure impact, and that impact is the deliverable, is a totally new concept. So again, if you think about the fact that I\u2019ve been in L&amp;D now for about 25 years, I go back pretty far and I remember at the beginning of my career, there were never any conversations or thought about impact being the deliverable.\r\n\r\nKevin Yates:\r\nThe deliverable was the training program, the deliverable was the class, the deliverable was the e-learning, or the deliverable was the PowerPoint that the facilitator was going to use to deliver the class. So, that was the deliverable. We never had any conversations about impact being the deliverable, and if you\u2019re going to know what impact is, then you have to measure it.\r\n\r\nKevin Yates:\r\nSo on some levels, Scott, measurement is not in our DNA, because it\u2019s just not been what we consider to be what we\u2019re producing. And that\u2019s okay, because there\u2019s a shift and I\u2019m excited to see the shift. But your point here is that acumen, that expertise, that skill, that capability has not traditionally been part of what we\u2019ve been expected to do. So what I believe we\u2019re seeing now is a shift, and it\u2019s interesting because I started doing research a couple of years ago, taking a look at the evolution of this new role that has emerged and [shaped] learning and development teams.\r\n\r\nKevin Yates:\r\nSo now you\u2019re seeing things like learning analyst, learning data scientist, learning measurement manager. You see all these new types of roles that are popping up that were just not part of the L&amp;D organization even as recently as 10 years ago. So I think that, as a community as an L&amp;D community, what we are beginning to recognize is that there is a need or a unique and very specific skill set within our L&amp;D teams that is focused on measurement, that is focused on data and focus on analytics.\r\n\r\nKevin Yates:\r\nNow to be sure, I believe that as a community and as an L&amp;D organization, we need to be data literate, but to expect, say the instructional designer to devote his or her time say, I don\u2019t know, 75% to instructional design and 25% to measurement. I believe that\u2019s unfair because the ability to measure impact is an art, a science and a skill. And I think that you need time to develop that art, that science and that skill. I also think that you need time and opportunity to focus on it.\r\n\r\nKevin Yates:\r\nSo the essence of what I\u2019m saying here is that I believe that it is important that we have roles embedded on L&amp;D teams that are narrowly and specifically focused on measurement. I think it\u2019s important that [an] L&amp;D organization commit[s] headcount to having minimally one role on the team that can help answer the question, \u201cDid training work with facts, evidence and data?\u201d\r\n\r\nKevin Yates:\r\nSo, what I\u2019m saying is the[re\u2019s] a rise in some of our biggest most recognizable brands and even [in] some smaller organizations, where these L&amp;D organizations and teams are now creating roles, recruiting for roles that are focused on measurement because measurement has not been traditionally a part of skill and capability requirements for L&amp;D professionals.\r\n\r\nTaryn Oesch:\r\nKevin, you wrote an article on this topic. It\u2019s been a couple years ago now. There was a data analyst on the L&amp;D team and-\r\n\r\nKevin Yates:\r\nI remember that.\r\n\r\nTaryn Oesch:\r\nDo you think there are more data analysts on the L&amp;D team[s] now than there were two years ago when you wrote that article?\r\n\r\nKevin Yates:\r\nI do, Taryn, and actually that\u2019s a great connection to my last point, because when I wrote that article, I was inspired to write it because as I was just taking a look at job postings, and doing different types of research, I was beginning to see these roles emerge in L&amp;D with these titles that were just unheard of. Some of those titles that I mentioned earlier, like learning data analysts and learning measurement specialist, I had never seen those types of roles before.\r\n\r\nKevin Yates:\r\nSo when I wrote that article, two years ago, I was inspired to do so because of the emergence of those roles that I was starting to see \u2026 and I would say that trend is continuing now, two years later, and I expect it to grow even more as we go forward. Because I expect that more learning and development teams will begin to see that it\u2019s important and valuable for us to answer the question, \u201cDid training work with analytics and data and facts?\u201d\r\n\r\nTaryn Oesch:\r\nRight.\r\n\r\nTaryn Oesch:\r\nCan we get a little bit into the weeds here? Do you have an example of from your work maybe of how measuring a learning program has led to improve support from senior leadership for continued investment in learning and development?\r\n\r\nKevin Yates:\r\nYes, and actually, I\u2019m going to answer that question by going in an opposite direction that I think will actually answer that question. So here\u2019s what I mean.\r\n\r\nKevin Yates:\r\nThere was a time when I worked in a business where, within that business, there were five separate business units, and so the business wanted to go to market as one business unit, which meant that each business unit would have to sell the other business units\u2019 products and services. So the first response to that was that we needed a training solution that was focused on product sales and product knowledge. So we put that sales training program into place in the business, spent quite a few dollars on it and quite a few people. When I took a look at the performance data for those individuals who were part of that program, I didn\u2019t see any changes in performance. When I took a look at client engagement scores and didn\u2019t see any changes in client engagement and then the bottom line when I took a look at sales data, I didn\u2019t see any changes in sales data.\r\n\r\nKevin Yates:\r\nSo the initial facts, evidence and data said that training program made zero impact. That\u2019s a little scary. As a trained professional, that\u2019s a little scary. Then going back to what you and I were talking about Scott, there was zero ROI for that. There was no return on investment because there was no change. So after taking a look at sales data, performance data, client data, and just seeing that no behavior has changed, no performance has changed, that really that trying to [execute that] program made zero difference, I then needed to go back and collect some qualitative data. After going back, and doing a few focus group and a few interviews and having some conversations and collecting that qualitative data, what I discovered is that the business had not changed its P&amp;L structure.\r\n\r\nKevin Yates:\r\nWhich meant that if I were in business unit A and I sold business unit B\u2019s products and services, I would not get revenue recognition for that. So I was only being held accountable for the sales number that I needed to hit for my respective work innovation. So I was able to take the qualitative data that came from the interviews and the conversations, and I was able to marry that with the quantitative data that came from the performance data out of the business, the sales data from the business and the client engagement data. And I was able to tell a story. I was able to take that story back to senior leadership to say, \u201cThe money we invested had zero impact on sales, on people\u2019s performance and client engagement and here\u2019s why.\u201d\r\n\r\nKevin Yates:\r\nWe didn\u2019t see a change, because what was not done prior to this sales training program was conversations around changing the revenue recognition system. So my response, and I\u2019m paraphrasing here, is that we could train until the cows came home, but if we did not change the revenue recognition system, we would never see a change in behavior performance, sales and impact in client engagement.\r\n\r\nKevin Yates:\r\nThat was around, that was a different story to tell, Taryn not that I didn\u2019t want you to ask \u2026 but what I wanted to do here was just demonstrate how you can use fact-based evidence to tell stories and tell them in a way that are compelling. So it wasn\u2019t me going to senior leadership saying, \u201cI don\u2019t think that this program worked and here\u2019s why.\u201d It was me saying, \u201cHere\u2019s what the data says. Here\u2019s the story, oh, and by the way, here\u2019s my recommendation for how we can really see a shift and how we can really achieve the goal that we have here of increasing revenue, improving client engagement and increasing sales.\u201d\r\n\r\nScott Rutherford:\r\nAnd it\u2019s a way to act as a consultant to the business, too \u2014 rather than just staying in your lane and looking only at learning and development, you\u2019re saying, \u201cWell, look, learning and development can do many things but it can\u2019t do everything on its own.\u201d\r\n\r\nKevin Yates:\r\nMan, you are so right Scott, and to be honest with you, I don\u2019t really see myself as a trainer or a quote-unquote, \u201cL&amp;D person,\u201d air quotes again. I really see myself as a performance consultant to the organization that is using training and learning solutions to help improve people\u2019s performance. So I see myself as a performance consultant first, and it just so happens that what I have in my bag of tricks, if you will, are training and learning solutions.\r\n\r\nScott Rutherford:\r\nWell, Kevin, thanks for your time. We really appreciate it, you spending a few minutes with us here on the podcast. I wanted to give you a chance if there\u2019s any final thoughts or advice you have for other folks who are in L&amp;D and maybe are just, feel like they\u2019re starting up the hill toward measurement. What advice do you have?\r\n\r\nKevin Yates:\r\nSo here\u2019s my advice. My advice is focus on performance first, focus on a change of behavior, a change in action, a change of performance as being the deliverable, as being the outcome, as being the result for your training and your learning solutions. So my mantra, and my guiding principle is, find at least one thing about a person\u2019s behavior or performance that you can attribute to training and learning and let that lead to the facts about impact.\r\n\r\nTaryn Oesch:\r\nKevin M. Yates, the Sherlock Holmes of learning and development. Thanks for joining us today.\r\n\r\nKevin Yates:\r\nThis has been an awesome discussion. Thank you both.\r\n\r\nScott Rutherford:\r\nThank you.\r\n\r\nTaryn Oesch:\r\nFor more information on this topic, we\u2019ve got lots of great content on Trainingindustry.com. Don\u2019t forget to check out the podcast episode page for this episode, Episode 19. We\u2019ll be linking to you some great resources there as well, trainingindustry.com\/trainingindustrypodcast.\r\n\r\nScott Rutherford:\r\nAnd we do hope you\u2019re enjoying listening to The Business of Learning. Thanks for listening and if you are, please continue consider rating and reviewing us on Apple podcasts and make sure you subscribe for notifications of upcoming episodes at trainingindustry.com. Thanks for listening.\r\n\r\nOutro:\r\nIf you have feedback about this episode, or would like to suggest a topic for a future program, email us at info@trainingindustry.com or use the \u201cContact Us\u201d page at trainingindustry.com. Thanks for listening to the Training Industry podcast."}],"tice_sponsors":"","custom_dfp_keywords":""},"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.8 (Yoast SEO v27.5) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI - Training Industry<\/title>\n<meta name=\"description\" content=\"A major challenge facing learning leaders is measuring the effectiveness of their training. That\u2019s the focus of this episode of The Business of Learning.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI\" \/>\n<meta property=\"og:description\" content=\"A major challenge facing learning leaders is measuring the effectiveness of their training. That\u2019s the focus of this episode of The Business of Learning.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/\" \/>\n<meta property=\"og:site_name\" content=\"Training Industry\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TrainingIndustry\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-19T15:20:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/assets-staging.trainingindustry.com\/content\/uploads\/2019\/07\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@TrainingIndustr\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"25 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/\",\"url\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/\",\"name\":\"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI - Training Industry\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/trainingindustry.com\\\/content\\\/uploads\\\/2019\\\/07\\\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg\",\"datePublished\":\"2019-07-29T18:15:12+00:00\",\"dateModified\":\"2024-07-19T15:20:55+00:00\",\"description\":\"A major challenge facing learning leaders is measuring the effectiveness of their training. That\u2019s the focus of this episode of The Business of Learning.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/#primaryimage\",\"url\":\"https:\\\/\\\/trainingindustry.com\\\/content\\\/uploads\\\/2019\\\/07\\\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg\",\"contentUrl\":\"https:\\\/\\\/trainingindustry.com\\\/content\\\/uploads\\\/2019\\\/07\\\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg\",\"width\":1920,\"height\":1080,\"caption\":\"African-American businessman looking at charts on a computer\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/measurement-and-analytics\\\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/trainingindustry.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Articles\",\"item\":\"https:\\\/\\\/trainingindustry.com\\\/articles\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/#website\",\"url\":\"https:\\\/\\\/trainingindustry.com\\\/\",\"name\":\"Training Industry\",\"description\":\"Articles, research and tools for the L&amp;D professional. Insights for managing the business of learning.\",\"publisher\":{\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/trainingindustry.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/#organization\",\"name\":\"Training Industry, Inc\",\"url\":\"https:\\\/\\\/trainingindustry.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/trainingindustry.com\\\/content\\\/uploads\\\/2017\\\/10\\\/training_industry_logo_stacked_RGB1.png\",\"contentUrl\":\"https:\\\/\\\/trainingindustry.com\\\/content\\\/uploads\\\/2017\\\/10\\\/training_industry_logo_stacked_RGB1.png\",\"width\":371,\"height\":113,\"caption\":\"Training Industry, Inc\"},\"image\":{\"@id\":\"https:\\\/\\\/trainingindustry.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/TrainingIndustry\\\/\",\"https:\\\/\\\/x.com\\\/TrainingIndustr\",\"https:\\\/\\\/www.instagram.com\\\/trainingindustr\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/training-industry-inc-\",\"https:\\\/\\\/www.youtube.com\\\/user\\\/TrainingIndustry\"]}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI - Training Industry","description":"A major challenge facing learning leaders is measuring the effectiveness of their training. That\u2019s the focus of this episode of The Business of Learning.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/","og_locale":"en_US","og_type":"article","og_title":"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI","og_description":"A major challenge facing learning leaders is measuring the effectiveness of their training. That\u2019s the focus of this episode of The Business of Learning.","og_url":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/","og_site_name":"Training Industry","article_publisher":"https:\/\/www.facebook.com\/TrainingIndustry\/","article_modified_time":"2024-07-19T15:20:55+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/assets-staging.trainingindustry.com\/content\/uploads\/2019\/07\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@TrainingIndustr","twitter_misc":{"Est. reading time":"25 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/","url":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/","name":"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI - Training Industry","isPartOf":{"@id":"https:\/\/trainingindustry.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/#primaryimage"},"image":{"@id":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/#primaryimage"},"thumbnailUrl":"https:\/\/trainingindustry.com\/content\/uploads\/2019\/07\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg","datePublished":"2019-07-29T18:15:12+00:00","dateModified":"2024-07-19T15:20:55+00:00","description":"A major challenge facing learning leaders is measuring the effectiveness of their training. That\u2019s the focus of this episode of The Business of Learning.","breadcrumb":{"@id":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/#primaryimage","url":"https:\/\/trainingindustry.com\/content\/uploads\/2019\/07\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg","contentUrl":"https:\/\/trainingindustry.com\/content\/uploads\/2019\/07\/Business-of-Learning-Podcast_Evaluating-Learning-and-Measuring-ROI-7.29.19.jpg","width":1920,"height":1080,"caption":"African-American businessman looking at charts on a computer"},{"@type":"BreadcrumbList","@id":"https:\/\/trainingindustry.com\/articles\/measurement-and-analytics\/the-business-of-learning-episode-19-evaluating-learning-and-measuring-roi\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/trainingindustry.com\/"},{"@type":"ListItem","position":2,"name":"Articles","item":"https:\/\/trainingindustry.com\/articles\/"},{"@type":"ListItem","position":3,"name":"The Business of Learning, Episode 19: Evaluating Learning and Measuring ROI"}]},{"@type":"WebSite","@id":"https:\/\/trainingindustry.com\/#website","url":"https:\/\/trainingindustry.com\/","name":"Training Industry","description":"Articles, research and tools for the L&amp;D professional. Insights for managing the business of learning.","publisher":{"@id":"https:\/\/trainingindustry.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/trainingindustry.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/trainingindustry.com\/#organization","name":"Training Industry, Inc","url":"https:\/\/trainingindustry.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/trainingindustry.com\/#\/schema\/logo\/image\/","url":"https:\/\/trainingindustry.com\/content\/uploads\/2017\/10\/training_industry_logo_stacked_RGB1.png","contentUrl":"https:\/\/trainingindustry.com\/content\/uploads\/2017\/10\/training_industry_logo_stacked_RGB1.png","width":371,"height":113,"caption":"Training Industry, Inc"},"image":{"@id":"https:\/\/trainingindustry.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TrainingIndustry\/","https:\/\/x.com\/TrainingIndustr","https:\/\/www.instagram.com\/trainingindustr\/","https:\/\/www.linkedin.com\/company\/training-industry-inc-","https:\/\/www.youtube.com\/user\/TrainingIndustry"]}]}},"_links":{"self":[{"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/articles\/41426","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/articles"}],"about":[{"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/types\/articles"}],"author":[{"embeddable":true,"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/users\/27"}],"version-history":[{"count":0,"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/articles\/41426\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/media\/41429"}],"wp:attachment":[{"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/media?parent=41426"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/trainingindustry.com\/wp-json\/wp\/v2\/tags?post=41426"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}