Hi everybody,
I am Reena from Delhi. I am pursuing an MBA. Currently, I'm doing my summer training project titled 'Measuring ROI of employees' training'. I have theoretical knowledge about the Kirkpatrick 4-level model of evaluation and the balanced scorecard. However, these are too theoretical to be implemented. Could anybody help, please?
Thanks,
Reena
From India, Mumbai
Acknowledge(0)
Amend(0)
5I am Reena from Delhi. I am pursuing an MBA. Currently, I'm doing my summer training project titled 'Measuring ROI of employees' training'. I have theoretical knowledge about the Kirkpatrick 4-level model of evaluation and the balanced scorecard. However, these are too theoretical to be implemented. Could anybody help, please?
Thanks,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Thank you, Rajnish, for your help. I am having a problem in practically measuring ROI. This is because measuring the soft skills of employees in monetary terms or even quantitative terms is very difficult. There are a lot of models that offer theoretical solutions, but none have been implemented practically.
Anyways, thanks a lot for your help. If you have any more information or suggestions regarding this, then please send a reply. And if you have any idea of a statistical tool to measure the correlation between training and its results, then please do send a reply.
Thank you,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Anyways, thanks a lot for your help. If you have any more information or suggestions regarding this, then please send a reply. And if you have any idea of a statistical tool to measure the correlation between training and its results, then please do send a reply.
Thank you,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Reena,
I have never done an ROI exercise on training and hence do not have the tools and metrics. You may want to subscribe to
online egroup for the purpose of discussion on ROI, www.groups.yahoo.com/groups/roinet.
Thanks,
Rajnish
Acknowledge(0)Amend(0)
I have never done an ROI exercise on training and hence do not have the tools and metrics. You may want to subscribe to
Thanks,
Rajnish
Acknowledge(0)Amend(0)
Thanks Rajnish,
I'm already a member of that group, but until now, I haven't found a practical way of measuring ROI of employees' training. Anyways, thanks for your help. If you have some other matter relating to the practical implementation of ROI, then I would really appreciate it if you could send it.
Thanks,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
I'm already a member of that group, but until now, I haven't found a practical way of measuring ROI of employees' training. Anyways, thanks for your help. If you have some other matter relating to the practical implementation of ROI, then I would really appreciate it if you could send it.
Thanks,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hi all,
I have to develop a feedback form for measuring the reaction level of participants of a training program. This is part of my summer project 'Measuring Return on Investment of Employees Training.'
Any suggestions?
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
I have to develop a feedback form for measuring the reaction level of participants of a training program. This is part of my summer project 'Measuring Return on Investment of Employees Training.'
Any suggestions?
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hi Reena,
I am doing my MBA in HR from IIPM, Delhi. I did a small project on a comprehensive study of training, in which I have seen the ROI factor as well. However, among the five companies I visited, none had started implementing ROI analysis. You can also refer to ISTD, which is located in the Qutab Institutional Area, for some books or references.
You can email me at sreenivasan.r@gmail.com for any further assistance. I am currently studying at ISTD. Best of luck!
From India, Vadodara
Acknowledge(0)Amend(0)
I am doing my MBA in HR from IIPM, Delhi. I did a small project on a comprehensive study of training, in which I have seen the ROI factor as well. However, among the five companies I visited, none had started implementing ROI analysis. You can also refer to ISTD, which is located in the Qutab Institutional Area, for some books or references.
You can email me at sreenivasan.r@gmail.com for any further assistance. I am currently studying at ISTD. Best of luck!
From India, Vadodara
Acknowledge(0)Amend(0)
Thank you, Sreenivasan, Mangehr, and Martin for your inputs. The links you provided, Mr. Martin, were really helpful, and the information given there was truly valuable. Thank you very much for your help.
The training program for which I have to calculate ROI is a technical program of SDH training. It is intended for engineers involved in the process. Since I'm working for a telecom company, this training is integral to the process. Until now, the effectiveness was measured by the feedback from the participants. But now, my boss has instructed me to devise a practical way to measure the ROI of this training program. She also wants me to find a statistical tool to measure the relationship between training and its results.
Also, I want to know what are the industry practices in India and worldwide regarding ROI. Any suggestions?
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
The training program for which I have to calculate ROI is a technical program of SDH training. It is intended for engineers involved in the process. Since I'm working for a telecom company, this training is integral to the process. Until now, the effectiveness was measured by the feedback from the participants. But now, my boss has instructed me to devise a practical way to measure the ROI of this training program. She also wants me to find a statistical tool to measure the relationship between training and its results.
Also, I want to know what are the industry practices in India and worldwide regarding ROI. Any suggestions?
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hi Reena,
Interestingly, most companies I know do not look at the ROI of training, and there is a simple reason why: All trainings are supposed to have a positive change in KSA (knowledge, skills, and attitude). This, in turn, helps in achieving desired results.
Now, if we look at the cost of not training, then you immediately get your answers: loss of the entire business, added cost due to rework, loss of customer trust, erosion of brand image, etc.
ROI is a great concept in theory, but one needs to look at the qualitative aspect as well. Would you ask your mother (assuming she's a homemaker) about her productivity in the house? Sometimes you don't want to walk down that road.
Cheers,
Vivek
Acknowledge(0)Amend(0)
Interestingly, most companies I know do not look at the ROI of training, and there is a simple reason why: All trainings are supposed to have a positive change in KSA (knowledge, skills, and attitude). This, in turn, helps in achieving desired results.
Now, if we look at the cost of not training, then you immediately get your answers: loss of the entire business, added cost due to rework, loss of customer trust, erosion of brand image, etc.
ROI is a great concept in theory, but one needs to look at the qualitative aspect as well. Would you ask your mother (assuming she's a homemaker) about her productivity in the house? Sometimes you don't want to walk down that road.
Cheers,
Vivek
Acknowledge(0)Amend(0)
Hi Vivek,
I agree one won't [shouldn't] discuss his/her mother regarding her productivity. But there is a difference between home and the company in which you work. You need to show your bosses what the returns will be from your training program. Moreover, it is important to know (if possible) the benefits derived from a training program. And last but not the least, this is my summer training project, so in any case, I've got to do it.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
I agree one won't [shouldn't] discuss his/her mother regarding her productivity. But there is a difference between home and the company in which you work. You need to show your bosses what the returns will be from your training program. Moreover, it is important to know (if possible) the benefits derived from a training program. And last but not the least, this is my summer training project, so in any case, I've got to do it.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hey all,
We conducted leadership training for the top management in our organization, and every participant was required to undertake a project based on the training. We calculated the ROI by deducting the monetary benefits derived from the training expenses and opportunity cost of capital, divided by the training expenses. This method has helped us arrive at one figure which shows the benefits derived.
Regards
From India, Delhi
Acknowledge(0)Amend(0)
We conducted leadership training for the top management in our organization, and every participant was required to undertake a project based on the training. We calculated the ROI by deducting the monetary benefits derived from the training expenses and opportunity cost of capital, divided by the training expenses. This method has helped us arrive at one figure which shows the benefits derived.
Regards
From India, Delhi
Acknowledge(0)Amend(0)
Hi Reena,
Even in my project, I did not see people calculating the ROI, but they calculate the effectiveness of the training and training program effectiveness as well. Since ROI for managers is a difficult job, as you have to convert the quality aspects to figures or costs, it becomes a problem.
The top people are satisfied with the effectiveness report itself, so they are not willing to go for ROI. However, in the future, it is compulsory. Even good companies like Glaxo International, which conduct their executive coaching, do not calculate ROI.
A company like Parle, where training is essential in their structure, is not conducting ROI, but they check training effectiveness. ROI for workers is a little easier to determine.
Thank you.
From India, Vadodara
Acknowledge(0)Amend(0)
Even in my project, I did not see people calculating the ROI, but they calculate the effectiveness of the training and training program effectiveness as well. Since ROI for managers is a difficult job, as you have to convert the quality aspects to figures or costs, it becomes a problem.
The top people are satisfied with the effectiveness report itself, so they are not willing to go for ROI. However, in the future, it is compulsory. Even good companies like Glaxo International, which conduct their executive coaching, do not calculate ROI.
A company like Parle, where training is essential in their structure, is not conducting ROI, but they check training effectiveness. ROI for workers is a little easier to determine.
Thank you.
From India, Vadodara
Acknowledge(0)Amend(0)
Hi Saurabh and Sreenivasan,
Saurabh, could you please tell me how you calculated the opportunity cost of capital for your training program? Moreover, in that training program, what benefits were derived? I'm sure there must be some benefits in terms of the development of hard skills or soft skills. Did you take into account these benefits as well? If so, please tell me how. According to me, it is important to consider these benefits since they are the basic reason for imparting training to employees.
Sreenivasan, I agree with your view that it is tough to calculate ROI since converting soft skills into quantitative terms is very difficult. However, ROI, if calculated, provides proper results on the benefits derived from a training program with respect to the cost of the program. It is based on numbers, so it is more concrete.
Training effectiveness yields results in qualitative terms. Drawing inferences from them is quite subjective in nature. That is the reason why companies are trying to determine the ROI of their training programs. Motorola does it on a worldwide basis.
I'm working on my project in Airtel. So, if I could calculate the ROI for a particular training program, it might be a beginning of ROI analysis in Airtel as well. Just joking :D
Of course, I'm just a summer trainee here, but still, this is a really challenging topic to work on as a project.
What do you say?
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Saurabh, could you please tell me how you calculated the opportunity cost of capital for your training program? Moreover, in that training program, what benefits were derived? I'm sure there must be some benefits in terms of the development of hard skills or soft skills. Did you take into account these benefits as well? If so, please tell me how. According to me, it is important to consider these benefits since they are the basic reason for imparting training to employees.
Sreenivasan, I agree with your view that it is tough to calculate ROI since converting soft skills into quantitative terms is very difficult. However, ROI, if calculated, provides proper results on the benefits derived from a training program with respect to the cost of the program. It is based on numbers, so it is more concrete.
Training effectiveness yields results in qualitative terms. Drawing inferences from them is quite subjective in nature. That is the reason why companies are trying to determine the ROI of their training programs. Motorola does it on a worldwide basis.
I'm working on my project in Airtel. So, if I could calculate the ROI for a particular training program, it might be a beginning of ROI analysis in Airtel as well. Just joking :D
Of course, I'm just a summer trainee here, but still, this is a really challenging topic to work on as a project.
What do you say?
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
hi everyone, does anybody know the industry practices regarding calculation of ROI of employees' training?? if possible, could anybody plz tell me how it is practised in their company? thanx Reena
From India, Mumbai
Acknowledge(0)Amend(0)
From India, Mumbai
Acknowledge(0)Amend(0)
Hi Reena,
That is a good spirit, and best of luck to you for Motorola. I will inquire about how they do it, since Freescale is a Motorola vendor in India, and they are located in Noida.
Okay, see you. Best of luck again. Bye
From India, Vadodara
Acknowledge(0)Amend(0)
That is a good spirit, and best of luck to you for Motorola. I will inquire about how they do it, since Freescale is a Motorola vendor in India, and they are located in Noida.
Okay, see you. Best of luck again. Bye
From India, Vadodara
Acknowledge(0)Amend(0)
Hi Reena,
We calculated opportunity cost as the cost of capital, i.e., the return money would have given if it was invested, like in a bank or other investments, i.e., the weighted average cost of capital. However, the limitation of this method is that we cannot calculate intangible benefits, as you mentioned.
Regards,
Saurabh
From India, Delhi
Acknowledge(0)Amend(0)
We calculated opportunity cost as the cost of capital, i.e., the return money would have given if it was invested, like in a bank or other investments, i.e., the weighted average cost of capital. However, the limitation of this method is that we cannot calculate intangible benefits, as you mentioned.
Regards,
Saurabh
From India, Delhi
Acknowledge(0)Amend(0)
hi saurabh, dnt u think ROI calculated by u was partial only coz intangible benefits r sometimes much more important than tangible benefits generated from training program. Reena
From India, Mumbai
Acknowledge(0)Amend(0)
From India, Mumbai
Acknowledge(0)Amend(0)
hi reena, i talkd with my friend and came to know that in motorola’s sister concern which is there doesnot follow the ROI process.. so sorry for that have a nice time sweet dreams too
From India, Vadodara
Acknowledge(0)Amend(0)
From India, Vadodara
Acknowledge(0)Amend(0)
Hi Sreenivasan,
Thank you a dozen for providing me with this information. I read on the internet that Motorola calculates the ROI of training on a worldwide basis. Anyways, now I'm on the lookout for companies in India that also calculate ROI. Let's see if I can gather some information or not.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Thank you a dozen for providing me with this information. I read on the internet that Motorola calculates the ROI of training on a worldwide basis. Anyways, now I'm on the lookout for companies in India that also calculate ROI. Let's see if I can gather some information or not.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hi Reena,
Actually, I was in a hurry at that time, so I could not provide the full details. I received information that ROI was not followed even when Motorola India was operating. Subsequently, when Freescale took over Motorola's operations, ROI was still not implemented in their office. It is possible that Motorola US is using ROI, but I am not certain about that. I am specifically referring to the Indian operations only.
Take care and have a nice time.
Best regards
From India, Vadodara
Acknowledge(0)Amend(0)
Actually, I was in a hurry at that time, so I could not provide the full details. I received information that ROI was not followed even when Motorola India was operating. Subsequently, when Freescale took over Motorola's operations, ROI was still not implemented in their office. It is possible that Motorola US is using ROI, but I am not certain about that. I am specifically referring to the Indian operations only.
Take care and have a nice time.
Best regards
From India, Vadodara
Acknowledge(0)Amend(0)
hi sreenivasan thanx for the info. may be motorola calculates ROI in some place other than india. anyways, i wanted to know abt the india operations only. thanx Reena
From India, Mumbai
Acknowledge(0)Amend(0)
From India, Mumbai
Acknowledge(0)Amend(0)
Steps to Calculate ROI
1. Obtain data to demonstrate the changes in behavior, e.g., that gathered through surveys, questionnaires, on-the-job observations, post-program interviews, focus groups, performance monitoring.
2. Isolate the effect of training, e.g., through the use of control groups, trend lines, forecasting models.
3. Convert the data to monetary value by focusing on a unit of measure, determining a value for that unit, calculating the change in performance data, determining the annual amount for the change, and calculating the total value of the improvement.
4. Tabulate the program costs: this is the value of the cost of taking people away from their jobs for the training, including salary and benefits.
5. Calculate the return on investment by dividing the net benefits by the costs times 100 percent.
Ekta
From India, Ahmadabad
Acknowledge(0)Amend(0)
1. Obtain data to demonstrate the changes in behavior, e.g., that gathered through surveys, questionnaires, on-the-job observations, post-program interviews, focus groups, performance monitoring.
2. Isolate the effect of training, e.g., through the use of control groups, trend lines, forecasting models.
3. Convert the data to monetary value by focusing on a unit of measure, determining a value for that unit, calculating the change in performance data, determining the annual amount for the change, and calculating the total value of the improvement.
4. Tabulate the program costs: this is the value of the cost of taking people away from their jobs for the training, including salary and benefits.
5. Calculate the return on investment by dividing the net benefits by the costs times 100 percent.
Ekta
From India, Ahmadabad
Acknowledge(0)Amend(0)
Ekta neatly summarizes the essence of Jack Phillips' ROI approach. Thank you, Ekta!
The Phillips approach has a number of useful characteristics, but if you don't appreciate the subtleties of some of them, you can create problems if you don't watch out for certain danger spots. Let me explain further...
1. Obtain data to demonstrate the changes in behavior, e.g., that gathered through surveys, questionnaires, on-the-job observations, post-program interviews, focus groups, performance monitoring.
The potential danger to watch out for with Step #1 is the root cause - you need to KNOW what the root causes are for the changes in behavior. Just observing the changes, getting survey and interview data, especially after the training, will only show CORRELATION, not CAUSATION. Get this bit right, and you SHOULD have a number of measures of performance that are readily convertible into financial values, which will be useful later on.
2. Isolate the effect of training, e.g., through the use of control groups, trend lines, forecasting models.
There are a number of dangers to watch out for in Step #2 as well. First, it may not be appropriate to isolate the effects of training. If the training is part of a project to address performance, as is almost always the case, and if this training is a 'must-have' part of the project, i.e., no training means no required improvement in performance, then it is meaningless to isolate the effects of training. You have to look at the project as a whole, and neither can you necessarily claim 100% of the benefits of the project simply because without the training, the project will fail to deliver the required performance improvement. That said, if you do the root cause analysis properly, you will get a solid measure of how much of the problem can be addressed by the training aspect. There will be times when isolating the effect of training does make sense - just be sure you know what you are doing and why. There are plenty of CEOs and CFOs that will shoot your ROI claims down in flames because you have not been rigorous enough in this respect.
Secondly, trend lines again only show CORRELATION, not CAUSATION - unless you have done a thorough root cause analysis and your plots of trends are on the right data. Furthermore, trend line plots may not easily show problems in the processes that deliver the performance in the first place - they are not the best tools for showing if the process is 'in control'. If the process is not 'in control', then no amount of forecasting trends and extrapolating will get you credible data - where forecasts appear to be accurate will be out of sheer luck - nothing more.
Control groups are fine to use PROVIDED you account for all the major factors. It is almost always the case that comparing one group that has had the training with another that has not had the training will tell you NOTHING other than the fact that there may be a difference in performance between the two. It CANNOT tell you WHY the difference is there because you have not taken steps to consider and robustly discount other factors that could affect performance.
3. Convert the data to monetary value by focusing on a unit of measure, determining a value for that unit, calculating the change in performance data, determining the annual amount for the change, and calculating the total value of the improvement.
If you recall my comments from Step #1, if you have a decent root cause analysis, you SHOULD have some measures of performance that are readily convertible to financial values.
4. Tabulate the program costs: this is the value of the cost of taking people away from their jobs for the training, including salary and benefits.
Yes - and also the development and delivery costs of the training course, and any accommodation and travel costs that go with it.
5. Calculate the return on investment by dividing the net benefits by the costs times 100 percent.
Because these figures are based on a comprehensive root cause analysis and robust use of control groups, for example, and supported if you wish by estimates of contribution by operators and other experts impacted by the training, you can rest easy in the knowledge that there isn't a CEO or CFO alive that can shoot your claims down - except to fulfill a hidden agenda, in which case normal reasoning goes out the window, and it's all down to politics!
I speak from some experience - I followed the Phillips approach and used control groups and trend lines and observations of behavior - presented them to the Board of Directors, who then promptly shot holes in the claims I was making because I couldn't PROVE my claims. Along with 85 other people, I lost my job during the next difficult trading period about 3 months later because I couldn't PROVE the contribution of the training function - just the fact that we were spending a lot of money on training.
Use the Phillips ROI approach, but like all methods (including my own 5 Boxes), use it with care and from a position of intimate knowledge.
Good luck!
Regards,
Martin
From United Kingdom,
Acknowledge(0)Amend(0)
The Phillips approach has a number of useful characteristics, but if you don't appreciate the subtleties of some of them, you can create problems if you don't watch out for certain danger spots. Let me explain further...
1. Obtain data to demonstrate the changes in behavior, e.g., that gathered through surveys, questionnaires, on-the-job observations, post-program interviews, focus groups, performance monitoring.
The potential danger to watch out for with Step #1 is the root cause - you need to KNOW what the root causes are for the changes in behavior. Just observing the changes, getting survey and interview data, especially after the training, will only show CORRELATION, not CAUSATION. Get this bit right, and you SHOULD have a number of measures of performance that are readily convertible into financial values, which will be useful later on.
2. Isolate the effect of training, e.g., through the use of control groups, trend lines, forecasting models.
There are a number of dangers to watch out for in Step #2 as well. First, it may not be appropriate to isolate the effects of training. If the training is part of a project to address performance, as is almost always the case, and if this training is a 'must-have' part of the project, i.e., no training means no required improvement in performance, then it is meaningless to isolate the effects of training. You have to look at the project as a whole, and neither can you necessarily claim 100% of the benefits of the project simply because without the training, the project will fail to deliver the required performance improvement. That said, if you do the root cause analysis properly, you will get a solid measure of how much of the problem can be addressed by the training aspect. There will be times when isolating the effect of training does make sense - just be sure you know what you are doing and why. There are plenty of CEOs and CFOs that will shoot your ROI claims down in flames because you have not been rigorous enough in this respect.
Secondly, trend lines again only show CORRELATION, not CAUSATION - unless you have done a thorough root cause analysis and your plots of trends are on the right data. Furthermore, trend line plots may not easily show problems in the processes that deliver the performance in the first place - they are not the best tools for showing if the process is 'in control'. If the process is not 'in control', then no amount of forecasting trends and extrapolating will get you credible data - where forecasts appear to be accurate will be out of sheer luck - nothing more.
Control groups are fine to use PROVIDED you account for all the major factors. It is almost always the case that comparing one group that has had the training with another that has not had the training will tell you NOTHING other than the fact that there may be a difference in performance between the two. It CANNOT tell you WHY the difference is there because you have not taken steps to consider and robustly discount other factors that could affect performance.
3. Convert the data to monetary value by focusing on a unit of measure, determining a value for that unit, calculating the change in performance data, determining the annual amount for the change, and calculating the total value of the improvement.
If you recall my comments from Step #1, if you have a decent root cause analysis, you SHOULD have some measures of performance that are readily convertible to financial values.
4. Tabulate the program costs: this is the value of the cost of taking people away from their jobs for the training, including salary and benefits.
Yes - and also the development and delivery costs of the training course, and any accommodation and travel costs that go with it.
5. Calculate the return on investment by dividing the net benefits by the costs times 100 percent.
Because these figures are based on a comprehensive root cause analysis and robust use of control groups, for example, and supported if you wish by estimates of contribution by operators and other experts impacted by the training, you can rest easy in the knowledge that there isn't a CEO or CFO alive that can shoot your claims down - except to fulfill a hidden agenda, in which case normal reasoning goes out the window, and it's all down to politics!
I speak from some experience - I followed the Phillips approach and used control groups and trend lines and observations of behavior - presented them to the Board of Directors, who then promptly shot holes in the claims I was making because I couldn't PROVE my claims. Along with 85 other people, I lost my job during the next difficult trading period about 3 months later because I couldn't PROVE the contribution of the training function - just the fact that we were spending a lot of money on training.
Use the Phillips ROI approach, but like all methods (including my own 5 Boxes), use it with care and from a position of intimate knowledge.
Good luck!
Regards,
Martin
From United Kingdom,
Acknowledge(0)Amend(0)
Ekta neatly summarises the essence of Jack Phillips' ROI approach. Thank you, Ekta! The Phillips approach has a number of useful characteristics, but if you don't appreciate the subtleties of some of them, you can create problems if you don't watch out for certain danger spots. Let me explain further...
1. Obtain data to demonstrate the changes in behavior, e.g., that gathered through surveys, questionnaires, on-the-job observations, post-program interviews, focus groups, and performance monitoring. The potential danger to watch out for with Step #1 is the root cause - you need to KNOW what the root causes are for the changes in behavior - just observing the changes, getting survey and interview data, especially after the training, will only show CORRELATION, not CAUSATION. Get this bit right and you SHOULD have a number of measures of performance that are readily convertible into financial values, which will be useful later on.
2. Isolate the effect of training, e.g., through the use of control groups, trend lines, and forecasting models. There are a number of dangers to watch out for in Step #2 as well. First, it may not be appropriate to isolate the effects of training. If the training is part of a project to address performance, as is almost always the case, and if this training is a 'must-have' part of the project, i.e., no training means no required improvement in performance, then it is meaningless to isolate the effects of training. You have to look at the project as a whole - and neither can you necessarily claim 100% of the benefits of the project simply because without the training, the project will fail to deliver the required performance improvement. That said, if you do the root cause analysis properly, you will get a solid measure of how much of the problem can be addressed by the training aspect - so there will be times when isolating the effect of training does make sense - just be sure you know what you are doing and why - there are plenty of CEOs and CFOs that will shoot your ROI claims down in flames because you have not been rigorous enough in this respect. Secondly, trend lines again only show CORRELATION, not CAUSATION - unless you have done a thorough root cause analysis and your plots of trends are on the right data. Furthermore, trendline plots may not easily show problems in the processes that deliver the performance in the first place - they are not the best tools for showing if the process is 'in control'. If the process is not 'in control', then no amount of forecasting trends and extrapolating will get you credible data - where forecasts appear to be accurate will be out of sheer luck - nothing more. Control groups are fine to use PROVIDED you account for all the major factors. It is almost always the case that comparing one group that has had the training with another that has not had the training will tell you NOTHING other than the fact that there may be a difference in performance between the two. It CANNOT tell you WHY the difference is there because you have not taken steps to consider and robustly discount other factors that could affect performance.
3. Convert the data to monetary value by focusing on a unit of measure, determining a value for that unit, calculating the change in performance data, determining the annual amount for the change, and calculating the total value of the improvement. If you recall my comments from Step #1, if you have a decent root cause analysis, you SHOULD have some measures of performance that are readily convertible to financial values.
4. Tabulate the program costs: this is the value of the cost of taking people away from their jobs for the training, including salary and benefits. Yes - and also the development and delivery costs of the training course, and any accommodation and travel costs that go with it.
5. Calculate the return on investment by dividing the net benefits by the costs times 100 percent
Thanks, Mr. Martin, for your valuable inputs. But I have certain constraints in the calculation of ROI of training. The first and foremost constraint which I'm facing is that I'm here in this organization for approximately 15 days more since I'm doing summer training here, and the program for which ROI needs to be calculated is currently in process. Secondly, I don't have much control in the creation of control groups, etc. The people to whom the training has to be imparted have already been decided. In fact, they are currently getting the training. I wanted to know if there is any way with which I can calculate short-term ROI, i.e., short-term returns of a training program with respect to the investment made in it. Thanks for your help.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
1. Obtain data to demonstrate the changes in behavior, e.g., that gathered through surveys, questionnaires, on-the-job observations, post-program interviews, focus groups, and performance monitoring. The potential danger to watch out for with Step #1 is the root cause - you need to KNOW what the root causes are for the changes in behavior - just observing the changes, getting survey and interview data, especially after the training, will only show CORRELATION, not CAUSATION. Get this bit right and you SHOULD have a number of measures of performance that are readily convertible into financial values, which will be useful later on.
2. Isolate the effect of training, e.g., through the use of control groups, trend lines, and forecasting models. There are a number of dangers to watch out for in Step #2 as well. First, it may not be appropriate to isolate the effects of training. If the training is part of a project to address performance, as is almost always the case, and if this training is a 'must-have' part of the project, i.e., no training means no required improvement in performance, then it is meaningless to isolate the effects of training. You have to look at the project as a whole - and neither can you necessarily claim 100% of the benefits of the project simply because without the training, the project will fail to deliver the required performance improvement. That said, if you do the root cause analysis properly, you will get a solid measure of how much of the problem can be addressed by the training aspect - so there will be times when isolating the effect of training does make sense - just be sure you know what you are doing and why - there are plenty of CEOs and CFOs that will shoot your ROI claims down in flames because you have not been rigorous enough in this respect. Secondly, trend lines again only show CORRELATION, not CAUSATION - unless you have done a thorough root cause analysis and your plots of trends are on the right data. Furthermore, trendline plots may not easily show problems in the processes that deliver the performance in the first place - they are not the best tools for showing if the process is 'in control'. If the process is not 'in control', then no amount of forecasting trends and extrapolating will get you credible data - where forecasts appear to be accurate will be out of sheer luck - nothing more. Control groups are fine to use PROVIDED you account for all the major factors. It is almost always the case that comparing one group that has had the training with another that has not had the training will tell you NOTHING other than the fact that there may be a difference in performance between the two. It CANNOT tell you WHY the difference is there because you have not taken steps to consider and robustly discount other factors that could affect performance.
3. Convert the data to monetary value by focusing on a unit of measure, determining a value for that unit, calculating the change in performance data, determining the annual amount for the change, and calculating the total value of the improvement. If you recall my comments from Step #1, if you have a decent root cause analysis, you SHOULD have some measures of performance that are readily convertible to financial values.
4. Tabulate the program costs: this is the value of the cost of taking people away from their jobs for the training, including salary and benefits. Yes - and also the development and delivery costs of the training course, and any accommodation and travel costs that go with it.
5. Calculate the return on investment by dividing the net benefits by the costs times 100 percent
Thanks, Mr. Martin, for your valuable inputs. But I have certain constraints in the calculation of ROI of training. The first and foremost constraint which I'm facing is that I'm here in this organization for approximately 15 days more since I'm doing summer training here, and the program for which ROI needs to be calculated is currently in process. Secondly, I don't have much control in the creation of control groups, etc. The people to whom the training has to be imparted have already been decided. In fact, they are currently getting the training. I wanted to know if there is any way with which I can calculate short-term ROI, i.e., short-term returns of a training program with respect to the investment made in it. Thanks for your help.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hi Mr. Martin,
The training program I'm supposed to calculate ROI for is on SDH Basics. SDH stands for Synchronous Digital Hierarchy. It is a technical program.
There are 4 measures on which we are planning to evaluate how much the employee has learned after training. These are:
1. Installation & Commissioning
2. Problem Diagnosis Skills
3. Testing with Customer
4. Equipment Handling
Since I'm not a technical person, I don't have knowledge of all this. But this training program is really important for the organization since the company for which I'm doing the project is in the telecom sector.
Waiting for your suggestions.
Thanks a lot.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
The training program I'm supposed to calculate ROI for is on SDH Basics. SDH stands for Synchronous Digital Hierarchy. It is a technical program.
There are 4 measures on which we are planning to evaluate how much the employee has learned after training. These are:
1. Installation & Commissioning
2. Problem Diagnosis Skills
3. Testing with Customer
4. Equipment Handling
Since I'm not a technical person, I don't have knowledge of all this. But this training program is really important for the organization since the company for which I'm doing the project is in the telecom sector.
Waiting for your suggestions.
Thanks a lot.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hi Reena,
Thanks for sharing that.
If I understand what you are saying, the SDH program is about achieving one of your employer's strategic objectives - building an SDH capability.
As such, there may not be an immediate KPI linked to this, such as profitability. This is quite normal for a project of this type.
It also makes the ROI issue an interesting one! The financial benefits of SDH may not become apparent for some time to come. What we can do, though, is to ask senior managers for their considered views, informed estimates, forecasts, etc. - call it what you will; the important thing is it will be THEIR numbers, and use this to predict the likely ROI. We won't claim all of the benefits of SDH, but through a root cause analysis, we can determine the probable contribution of the training and estimate the ROI on that basis.
In situations like this where time is short, and/or data is short, estimates are entirely reasonable data to use, so long as assumptions underlying the estimates are made clear to all.
I'll be back in a few hours with my specific suggestions.
Regards,
Martin
From United Kingdom,
Acknowledge(0)Amend(0)
Thanks for sharing that.
If I understand what you are saying, the SDH program is about achieving one of your employer's strategic objectives - building an SDH capability.
As such, there may not be an immediate KPI linked to this, such as profitability. This is quite normal for a project of this type.
It also makes the ROI issue an interesting one! The financial benefits of SDH may not become apparent for some time to come. What we can do, though, is to ask senior managers for their considered views, informed estimates, forecasts, etc. - call it what you will; the important thing is it will be THEIR numbers, and use this to predict the likely ROI. We won't claim all of the benefits of SDH, but through a root cause analysis, we can determine the probable contribution of the training and estimate the ROI on that basis.
In situations like this where time is short, and/or data is short, estimates are entirely reasonable data to use, so long as assumptions underlying the estimates are made clear to all.
I'll be back in a few hours with my specific suggestions.
Regards,
Martin
From United Kingdom,
Acknowledge(0)Amend(0)
Hello Mr. Martin,
This is not the first time when SDH Basics training is being imparted to employees in our organization. As I've also told you previously, the company for which I'm working is in the telecom sector, Airtel to be more precise. So, for the transmission of data through communication lines, the best technology available is SDH. This training is at the heart of our organization since new employees (engineers, to be more precise) need to be taught the details of this technology.
In Airtel, until now, training evaluation was done through feedback forms only, i.e., at the reaction level. But now, we are planning to go for proper ROI calculations. I hope I have cleared a few more details.
Thanks for your interest. Waiting for your valued suggestions.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
This is not the first time when SDH Basics training is being imparted to employees in our organization. As I've also told you previously, the company for which I'm working is in the telecom sector, Airtel to be more precise. So, for the transmission of data through communication lines, the best technology available is SDH. This training is at the heart of our organization since new employees (engineers, to be more precise) need to be taught the details of this technology.
In Airtel, until now, training evaluation was done through feedback forms only, i.e., at the reaction level. But now, we are planning to go for proper ROI calculations. I hope I have cleared a few more details.
Thanks for your interest. Waiting for your valued suggestions.
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Thanks for that update.
An immediate suggestion I'd like to offer is this: try to find out the expectation of senior management with respect to evaluating at ROI level - what do they want to do with this information? Is it to determine if the training is being done as cheaply as possible yet still remain effective? Is it to decide if the training should be cancelled? ROI cannot always help in these respects. It may be unkind to suggest that senior management have recently discovered the concept of training ROI seeing as it seems to be getting lots of 'air time' at the moment, and that they have 'jumped on the bandwagon' without appreciating the limitations of this approach, but I can't totally dismiss this notion from my mind - I've certainly seen it in many other organizations!
It seems that SDH is core to the engineering operations of this telco. As such this SDH training that we are discussing is like safety and other regulatory training - it has to be done.
The benefits of doing this training therefore need to be viewed in terms of what it prevents, i.e. it provides engineers with the skills and knowledge needed to prevent them from making mistakes through inappropriate actions and decision-making. Assuming the training is effective, any mistakes they do make after the training will more likely be due to other factors, such as problems with equipment, the working culture and practices in the workplace, standards of supervision, and perhaps also issues with customer and team relations - all of these will be a factor, some larger than others.
The maximum benefit this training will deliver is therefore expressible in terms of preventing the worst-case scenario - so this scenario needs to be identified and costed. This reality is likely to require the involvement of some experienced engineers, for example, aided by a reasonably detailed process map of all the activities these engineers do, and for each of these activities, all the possible errors they can make, and for each error, the likelihood or probability of that error happening, and the costs/consequences of each. In the interests of time, this need not be exhaustive, but should cover perhaps 80% of the factors.
One tool to help manage this is to use the relationship between incident rates of major errors to minor errors, based on work in the safety industry. Typically for every major error, or say a fatality, there are between 10 and 30 (nearer 30) lesser errors or lesser injuries, and for each of these lesser incidents, there are between 10 and 30 (nearer 30) minor errors or incidents.
It is also the case that newly trained operators are less likely to make mistakes in operations IF the training they have had is effective and if the workplace culture is reasonable. So the real ROI can perhaps only be actually calculated for a particular course and group of engineers after one year, where you identify all the possible opportunities they had to make mistakes, from this calculate the maximum possible benefit that COULD be gained by not having any errors/mistakes, and then subtracting the costs of the mistakes that actually happened. Assuming you also have the delivery and amortized development costs of the training, an estimate of the ROI can be made. BUT, you still need to account for the effects of other factors such as poor supervision - which implies you probably need some form of error/incident investigation system to enable each error to be investigated for root cause etc - this is all beginning to get very complex, and you only have a few weeks!!!
I'll ponder this a little more and get back to you today!
From United Kingdom,
Acknowledge(0)Amend(0)
An immediate suggestion I'd like to offer is this: try to find out the expectation of senior management with respect to evaluating at ROI level - what do they want to do with this information? Is it to determine if the training is being done as cheaply as possible yet still remain effective? Is it to decide if the training should be cancelled? ROI cannot always help in these respects. It may be unkind to suggest that senior management have recently discovered the concept of training ROI seeing as it seems to be getting lots of 'air time' at the moment, and that they have 'jumped on the bandwagon' without appreciating the limitations of this approach, but I can't totally dismiss this notion from my mind - I've certainly seen it in many other organizations!
It seems that SDH is core to the engineering operations of this telco. As such this SDH training that we are discussing is like safety and other regulatory training - it has to be done.
The benefits of doing this training therefore need to be viewed in terms of what it prevents, i.e. it provides engineers with the skills and knowledge needed to prevent them from making mistakes through inappropriate actions and decision-making. Assuming the training is effective, any mistakes they do make after the training will more likely be due to other factors, such as problems with equipment, the working culture and practices in the workplace, standards of supervision, and perhaps also issues with customer and team relations - all of these will be a factor, some larger than others.
The maximum benefit this training will deliver is therefore expressible in terms of preventing the worst-case scenario - so this scenario needs to be identified and costed. This reality is likely to require the involvement of some experienced engineers, for example, aided by a reasonably detailed process map of all the activities these engineers do, and for each of these activities, all the possible errors they can make, and for each error, the likelihood or probability of that error happening, and the costs/consequences of each. In the interests of time, this need not be exhaustive, but should cover perhaps 80% of the factors.
One tool to help manage this is to use the relationship between incident rates of major errors to minor errors, based on work in the safety industry. Typically for every major error, or say a fatality, there are between 10 and 30 (nearer 30) lesser errors or lesser injuries, and for each of these lesser incidents, there are between 10 and 30 (nearer 30) minor errors or incidents.
It is also the case that newly trained operators are less likely to make mistakes in operations IF the training they have had is effective and if the workplace culture is reasonable. So the real ROI can perhaps only be actually calculated for a particular course and group of engineers after one year, where you identify all the possible opportunities they had to make mistakes, from this calculate the maximum possible benefit that COULD be gained by not having any errors/mistakes, and then subtracting the costs of the mistakes that actually happened. Assuming you also have the delivery and amortized development costs of the training, an estimate of the ROI can be made. BUT, you still need to account for the effects of other factors such as poor supervision - which implies you probably need some form of error/incident investigation system to enable each error to be investigated for root cause etc - this is all beginning to get very complex, and you only have a few weeks!!!
I'll ponder this a little more and get back to you today!
From United Kingdom,
Acknowledge(0)Amend(0)
Sorry, Mr. Martin for such a late response...
Your articles were really informative.
Since my training is about to end in a week, I've completed some portion of the project and planned for the rest. Here is how I'm doing it...
Level 1 - Reaction - With the help of feedback forms. Participants have already filled these forms. I'm in the process of analyzing them. Our feedback form consisted of a combination of both objective and subjective questions.
Level 2 - Learning - With the help of a quiz. Already taken by the trainer on the last day of the training, and I have received the respective scores of the participants. I still have to analyze them.
Level 3 - Behavior - We identified 4 measures that would be impacted by the training program. Participants have already rated themselves on a 10-point scale with respect to these measures on the 1st day of the training. Similar responses from their superiors are awaited.
Participants and their superiors will be asked to rate themselves again 3 months after training. Then the difference will be analyzed.
There are some limitations that we have identified:
1. Level 4 is not possible to measure since the training effects are not properly separable. Moreover, since we are calculating ROI for the first time, I think it's better to move in small steps.
2. ROI would be calculated in quantitative terms, which would then be converted into a percentage form. It won't be calculated in monetary terms.
3. This is my own limitation. Since I have to submit the project by 15th of August, I won't be able to get post-data of Level 3. Although I would be showing the process in my project report, there won't be any post-data.
Waiting for your valued feedback....
Regards,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Your articles were really informative.
Since my training is about to end in a week, I've completed some portion of the project and planned for the rest. Here is how I'm doing it...
Level 1 - Reaction - With the help of feedback forms. Participants have already filled these forms. I'm in the process of analyzing them. Our feedback form consisted of a combination of both objective and subjective questions.
Level 2 - Learning - With the help of a quiz. Already taken by the trainer on the last day of the training, and I have received the respective scores of the participants. I still have to analyze them.
Level 3 - Behavior - We identified 4 measures that would be impacted by the training program. Participants have already rated themselves on a 10-point scale with respect to these measures on the 1st day of the training. Similar responses from their superiors are awaited.
Participants and their superiors will be asked to rate themselves again 3 months after training. Then the difference will be analyzed.
There are some limitations that we have identified:
1. Level 4 is not possible to measure since the training effects are not properly separable. Moreover, since we are calculating ROI for the first time, I think it's better to move in small steps.
2. ROI would be calculated in quantitative terms, which would then be converted into a percentage form. It won't be calculated in monetary terms.
3. This is my own limitation. Since I have to submit the project by 15th of August, I won't be able to get post-data of Level 3. Although I would be showing the process in my project report, there won't be any post-data.
Waiting for your valued feedback....
Regards,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Reena - please give a bit more on possible Level 4 measures, and I’ll try to suggest links between them and training. Cheers Martin
From United Kingdom,
Acknowledge(0)Amend(0)
From United Kingdom,
Acknowledge(0)Amend(0)
Hi Mr. Martin,
I talked to my boss regarding Level 4 measures, but she is saying that we ourselves don't want to go for Level 4 this time because this is the first time we are going for ROI calculation. So she wants to go only till Level 3 as of now. Maybe later on, for some other training program, the company can opt for going to Level 4.
Anyways, thanks a lot for your valuable help.
Regards,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
I talked to my boss regarding Level 4 measures, but she is saying that we ourselves don't want to go for Level 4 this time because this is the first time we are going for ROI calculation. So she wants to go only till Level 3 as of now. Maybe later on, for some other training program, the company can opt for going to Level 4.
Anyways, thanks a lot for your valuable help.
Regards,
Reena
From India, Mumbai
Acknowledge(0)Amend(0)
Hi Reena,
It may be that I'm not understanding your situation fully. From my interpretation of your postings, your organization is planning to work out the return on investment (ROI) of your training, but is not planning to do a Level 4 evaluation. Is this a correct interpretation?
If it is, then permit me to clarify the evaluation levels.
Level 4 evaluations are about the impact on the organization - how this training has affected the performance of the organization.
Level 3 is about how behaviors have changed as a result of the training.
Return on Investment is usually concerned with determining if the benefits acquired as a result of the training are greater than the costs of effort and resources that went into developing and delivering the training.
It is difficult to see how you can determine such an ROI value without conducting a Level 4 evaluation - you simply have to know how much the training has impacted the organization's performance (a Level 4 evaluation) to be able to calculate a CREDIBLE ROI value. If your ROI value can't be credible, you may as well not bother wasting the effort required!
Kind regards,
Martin
From United Kingdom,
Acknowledge(0)Amend(0)
It may be that I'm not understanding your situation fully. From my interpretation of your postings, your organization is planning to work out the return on investment (ROI) of your training, but is not planning to do a Level 4 evaluation. Is this a correct interpretation?
If it is, then permit me to clarify the evaluation levels.
Level 4 evaluations are about the impact on the organization - how this training has affected the performance of the organization.
Level 3 is about how behaviors have changed as a result of the training.
Return on Investment is usually concerned with determining if the benefits acquired as a result of the training are greater than the costs of effort and resources that went into developing and delivering the training.
It is difficult to see how you can determine such an ROI value without conducting a Level 4 evaluation - you simply have to know how much the training has impacted the organization's performance (a Level 4 evaluation) to be able to calculate a CREDIBLE ROI value. If your ROI value can't be credible, you may as well not bother wasting the effort required!
Kind regards,
Martin
From United Kingdom,
Acknowledge(0)Amend(0)
Yes, Mr. Martin, you have interpreted it correctly.
Here I would like to clarify something. We are not going for proper ROI calculations in monetary terms. We are just going for quantitative evaluation of Level 1, 2, and 3. Later on when our organization starts doing the same evaluation for every training program, then maybe we can move to Level 4 and ROI calculation.
From India, Mumbai
Acknowledge(0)Amend(0)
Here I would like to clarify something. We are not going for proper ROI calculations in monetary terms. We are just going for quantitative evaluation of Level 1, 2, and 3. Later on when our organization starts doing the same evaluation for every training program, then maybe we can move to Level 4 and ROI calculation.
From India, Mumbai
Acknowledge(0)Amend(0)CiteHR is an AI-augmented HR knowledge and collaboration platform, enabling HR professionals to solve real-world challenges, validate decisions, and stay ahead through collective intelligence and machine-enhanced guidance. Join Our Platform.
ROI Training.zip
Training ROI Calculator.xls