(Training expenditures are a unique form of business expense. Unlike other expenses, training costs represents an investment by the firm in their employees. As with any other investment, a commitment to training is directly related to the expected returns from each dollar invested. In this article about ROI of training programs different methods of ROI calculation with relevance to training process has been discussed with examples of ROI in training calculations. ROI in FOUR levels of training evaluation has been discussed with relevant examples. A model has been developed in the end to link ROI of training with business strategic decisions.)
Training is the process of acquiring and improving the skills, knowledge, and attitudes required for job performance; it is an investment an organization makes in itself While training can take many forms, the desired end is generally the same: improved performance of job-related tasks. The evaluation of training, like motherhood and apple pie, is inherently a good thing. But, because short term priorities always crowd out their longer term competitors, it's typically something we plan to do better next year - after all, we've got away with it so far, so another year won't hurt! And even if training evaluation is undertaken, it is usually at the easiest and lowest level - the measurement of student reactions through happy sheets. Reactions are important and the happy sheets serve a purpose, but will they be enough to back up your arguments when there is a need for a greater investment in training, when major changes need to be made in direction, when there is stiffer competition for resources, when times get tough?
What is ROI?
Return on investment (ROI) is a measure of the monetary benefits obtained by an organization over a specified time period in return for a given investment in a training programme. Looking at it another way, ROI is the extent to which the benefits (outputs) of training exceed the costs (inputs).
A company may provide training to their workers, managers, customers, and sometimes suppliers. The ultimate reason for training them should be to improve the profits and repeat business of the company. It is preferred that this improvement is measurable, so that an effective ROI can be determined. When ROI is calculated they should be compared with the targets for the specific training program for which it is calculated. Calculation of ROI deserves much attention, because it represent the ultimate approach to evaluation and is becoming increasingly important part of HRD functions like training as business becomes more bottom-line oriented.
Why assess ROI in training?
ROI can be used both to justify a planned investment and to evaluate the extent to which the desired return was achieved. One of the major reasons for assessing ROI in training is to decide whether the funds or effort involved has been worthwhile. Individuals want to know whether the time and money they have spent in pursuing a certain qualification or program of skills will deliver them higher income in the short or long term, or better opportunities for advancement. Enterprises want to know whether training has led to better workplace performance in terms of increased productivity, adaptation to technology, international or domestic competitiveness, and occupational health and safety. In addition, enterprises want to know whether their training has helped them to comply with any legislative requirements. Some of the reasons why we should go for ROI are
To validate training as a business tool
To justify the costs incurred in training
To help improve the design of training
Criteria for measuring training success
Performance to schedule
Return on Investment in Training –The Process Model
This process model describes how to monitor the whole training process in order to calculate return on investment of training program.
Data collection is central to the ROI process. In some cases, post-program data are collected and compared with pre-program situations, control group differences and expectations. Both hard data, representing output, quality, cost and time and soft data including work habits, work climate and attitudes are collected.
Isolating Effects of Training:
An often-overlooked issue in many evaluation studies is the process by which the effects of training are isolated. In this step of the process model, specific strategies are explored that determine the amount of output performance directly related to the program. Because many factors will influence performance data after training, this step is essential. It will pinpoint the amount of improvement directly related to the training program
This can be achieved by the following strategies
Control Group technique
Participants, supervisors, experts and senior mangemt estimate of the amount of improvement related to training.
Customers provide input on the extent to which Training has influenced their decision to use product or service. This strategy has got limited applications except customer service and sales training.
Collectively, these strategies provide a comprehensive set of tools to tackle the important and critical issue of isolating the effect of training.
Converting Data to Monetary Value:
To calculate ROI, data collected in level four of Kirkpatrick’s model i.e. business results are converted into monetary value and compare to program cost. This requires value to be placed on each unit of data connected with the program.
The strategies available to convert data into monetary value are
output data is converted to profit contribution or cost saving
converting employee time
the cost of quality calculation
Participants, supervisors, experts and senior mangemt estimate of the amount of improvement related to training.
General steps to convert data:
Focus on unit of measure
Determine the value of each unit
Calculate the change in performance data
Determine an annual amount for change
Calculate the total value of improvement
Training Program Cost:
Design and development costs
The first category of cost to be considered is the design and development of the training programme, whether this comprises classroom events, self-study materials, simple coaching sessions or some combination. You will need to consider:
internal days of design and development
costs of external designers and developers
other direct design and development costs (purchase of copyrights, travel, expenses, etc.)
outright purchase of off-the-shelf materials
Most organizations devote effort to promote their training programmes. This second category takes promotional costs into account:
internal days of promotional activity
costs of external agencies
other direct costs of promotion (posters, brochures, etc.)
An allowance must be made for the time taken by the training department in administrating the training programme. This will typically be a factor of the number of students:
hours of administration required per student
direct administration costs per student (joining materials, registration fees, etc.)
The next category of costs relates to the delivery of the training, whether this is mediated by faculty (tutors, instructors, coaches, etc.) or is self-administered (workbooks, CBT, online training, etc.). Let’s start with the information needed to calculate faculty costs:
the number of students who will be going through the programme
hours of group training (whether classroom-based or delivered in real time, online)
hours of one-to-one training (typically face-to-face, but could conceivably be conducted by telephone, video conferencing link or in real-time, online)
hours of self-study training
additional faculty hours (preparation time, the time needed to review or mark submitted work or the time needed to correspond by email or bulletin boards with online students)
faculty expenses (travel, accommodation, subsistence, etc.).
Then there's the cost of materials:
cost per student of training materials (books, manuals, consumables, etc.)
license cost per student for use off-the-shelf materials
You will also need to allow for the cost of your training facilities, whether these are internal or external. Make sure to include the rental or notional internal cost of the following:
open learning / self-study rooms
Probably the most significant delivery cost relates to the students themselves. It is only necessary to charge a student’s cost against the programme if training is undertaken in time that would otherwise be productive and paid for, so you only need to estimate the amount of travel and training that is undertaken in productive work time, i.e. not in slack time, breaks or outside work hours. When an employee goes through a training programme in work time, the organisation is not only having to pay that person’s payroll costs, they are also losing the opportunity for that person to add value to the organisation. When a salesperson is on a course, they are not bringing in new business. Similarly, a production line worker is not creating products, a researcher is not developing new ideas and an accountant is not finding ways to save money.
If an employee can be easily replaced while they are undergoing training, then there is no lost opportunity – the cost is simply the employee’s payroll costs. In many cases, however, it is simply not practical to obtain a suitable replacement, so the output that the employee would have generated in the time that they are receiving training will be lost. In this case, the true cost of the employee being trained is the lost opportunity – the 'opportunity cost'. The calculation of opportunity costs goes beyond the scope of this article, but suffices to say, they are greater than an employee's payroll costs and need to be considered in any serious evaluation of costs. Finally, don't forget to include any direct student expenses - travel, accommodation and subsistence.
Calculation of Return on Investment:
The two most common measures for calculating are cost benefit ratio and the ROI formula.
Cost/Benefit Ratio (CBR):
This method compares the training program benefits to its costs.
The formula for CBR is
CBR = program benefits / program costs
An applied behavior management program, designed for managers and supervisors, was implemented in an electric and gas factory. The first year payoff for the program was Rs.1, 077,750 while the total fully loaded implementation program cost were Rs.215, 500. Thus the CBR of the program is
CBR= Rs. (1,077,750/215,500) =5:1
For every rupee invested in the program, five rupees were returned.
The formula for ROI is
ROI (%) = (Net program benefits / Program Costs) * 100
Net benefit = Program Benefits – Program costs
The ROI value is related to the CBR by a Factor of one, example a CBR of 2.45 is same as an ROI value of 145%. An ROI on a training investment of 50% means that the costs are fully recovered and an additional 50% of the costs are reported as earnings.
A literacy program at entry-level electrical assemblers at Magnavox Electronics System Co. The productivity and quality alone yielded an annual value of $ 321,600, while the total fully loaded costs of the program were only $38,233.
Thus ROI is
ROI= $(321600-38233)/$38,233 *100 = 741%
For each dollar invested, Magnavox received $7.4 dollars over the cost of the program.
CASE STUDY: Using Process Model for ROI
Retail Merchandise Co. (RMC), large national chain located in most major American markets, attempted to boost sales with an interactive selling-skills program for sales associates. The program developed and delivered by an outside vendor, was responding to clearly defined need to increase the level of interaction between sales associates and customers. The program consisted of two days of skills training, one day of follow-up and additional training, and three weeks of on the job application of the skills. Three groups from electronic departments in three stores were initially trained as a pilot implementation. The program had a total of 48 participants.
Post program data collection was accomplished using three methods. First, the average weekly sales of each associate were monitored (business performance monitoring of output data). Second, a follow-up questionnaire was distributed three months after the training completion to determine level three success (actual application of the skills on the job). Third, by design, the third day of program served as the follow-up session during which level three data were solicited. In this session, participants disclosed their success (or lack there of) with the application of new skills. They also discussed techniques to overcome the barriers to program implementation. The method used to isolate the effects of training was a controlled group arrangement. Three store locations were identified (control group) to compare with the three groups in the pilot training (experimental group). Store size, store location on customer traffic levels were used to match the two groups so that they could be as identical as possible. The method used to convert data to monetary values is a direct profit contribution of the increased output. The actual profit obtained from an additional one dollar of sales was readily available and used in calculation.
CBR and ROI calculation:
Although the program was evaluated at all five levels, the emphasis was on the level five calculations (level five is the addition of ROI of the Training process to KIRKPATRICK”S model). The level 1,2 and 3 data either met or exceeded expectations. The table shows the level 4 data, which were the average weekly sales of both groups after training. For convenience and at the request of management, a three-month follow-up period was used. Management wanted to implement the program at other locations if it appeared to be successful in this first three months of operations. Although three months may be premature to determine the total impact of the program it often is a convenient time period for evaluation. The above table shows data for the first three weeks after training, as well as for the last three weeks of evaluation period (weeks 13,14 and 15). The data shows what appears to be a significant difference in the two values.
Weeks After Training Trained Groups Control Groups
1 $9723 9698
2 9978 9720
3 10424 9812
13 13690 11572
14 11491 9683
15 11044 10092
Average for weeks 13,14,15 $12075 10449
Level 4 Data: Avg. Weekly Sales (Post training data)
Two steps are required to move from level four data to level five data: level 4 data must be converted to monetary values, and the cost of the program must be tabulated .The table below shows the annualized program benefits. Because only 46 participants were still in their jobs after three months, the other two participants’ potential improvement was removed from the calculation. The profit contribution at the store level, obtain directly from the accounting department, was two percent, meaning that for everyone dollar of additional sales attributed to the program, two cents would be considered to be the added value. At the corporate level, the number was even smaller, about 1.2%. First year annualized values are used to reflect the total impact of the program. Ideally if new skills are acquired as indicated in the level 3 evaluation, there should be some value for the use of those skills in year two or perhaps even year three. However for short-term training programs only the first year values are used, thus requiring the investment to have an acceptable return in one year time period. The total benefit of this training program was $ 71,760.
Avg. Weekly sales trained Groups $ 12075
Avg. Weekly sales untrained Groups $10449
Profit Contribution 2% $32.50
Total weekly improvement (multiplied by 46, as that no. of participants were still present on job) $1495
Total annual benefits (multiplied by 48 weeks) $71760
Annualized Program Benefits
The table below shows the cost summary for the three-course program. Costs are fully loaded including data for all forty-eight participants. Since a vendor conducts the program, there is no direct development cost. The facilitation fee covers the pro-related development costs as well as delivery costs. The participants’ salaries plus a 35 % factor for employee benefits were included. Facilities costs were included although the company does not normally capture the costs when internal facilities are used, as was the case with the program. The estimated costs for the coordination and evaluation were also included, for a total program cost of $ 32,984.
Facilitation Fees: 3 courses @ $3750 11250
Program Materials: 48@ $35/participant 1680
Meals/refreshments: 3 days@ 428/participant/day 4032
Facilities: 9 days @ $120 1080
Participant salaries plus benefits (35%) 12442
TOTAL COST $32,984
CBR = ($71760/$32984) =2.2:1
ROI = $ (71760- 32984)/$32984 * 100 = 118%
Thus we conclude that the program has excellent return on investment in its initial 3 months of on job application.
OTHER ROI MEASURES
Another way at looking at ROI is to calculate how many months it will take before the benefits of the training match the costs and the training pays for itself. In this approach, the annual cash proceeds (saving) produced by an investment are equated to the original cash outlay required by investment to arrive at some multiple of cash proceeds equal to original investment. Measurement is usually in terms of years and months. This is called the payback period:
= (total Investments / Net annual savings) or (costs / monthly benefits)
Payback period is a powerful measure. If the figure is relatively low – perhaps only a few months – then management will be that much more encouraged to make the training investment. As a measure, it also has the advantage of not requiring an arbitrary benefit period to be specified.
Here's an example of the final results for a ROI analysis:
Duration of training 33 hrs
Estimated student numbers 750
Period over which benefits are calculated 12 months
Design and development £40,930
Total cost £754,165
Labour savings £241,071
Productivity increases £675,000
Other cost savings £161,250
Other income generation £0
Total benefits £1,077,321
Return on investment 143%
Payback period 8 months
Example: Leadership program
Pat, a new manager with a salary of $60,000, and his supervisor and peers, agree that, prior to a training program on leadership, he is a 3: “Can perform skill with guidance.” Six months after the training, Pat’s supervisor and his peers now see him as a 4: “Can perform skill independently.” In Pat’s organization, 50 percent of a manager’s time is to be allocated to the skill of leadership. Supervisors who manage positions like Pat’s have determined that an improvement from a 3 to a 4 on a skill that is used in 50 percent of a manager’s job gets a weight of .05. The economic benefit to the organization is $3,000 (Pat’s gross salary of $60,000 x .05). In other words, Pat is performing the work of a $63,000 manager, but at an actual salary of $60,000. Finally, the cost of Pat’s two-day leadership training program was $1,200. We now have all of the data to calculate both an ROI and the payback period.
ROI = $ (3000-1200)/$1200 *100 = 150%
Payback Period = $1200/($3000-$1200/244) =4 days
Pat’s leadership training course has yielded his company an economic benefit of $1,800 ($3,000 - $1,200), an ROI of 150 percent, and the cost of the training is paid off in four days.
Example: Ms Excel training program
June, a project manager earning $55,000 per year. June has just taken a seven-hour course on Intermediate Excel. Excel is important to Jane’s job as project manager, hence the rating of 4 for “Importance of Excel on the Job.” Before training, Jane used Excel 12.2 hours per week. After her Excel course, Jane is still using Excel 12.2 hours per week, but at a new level of efficiency that is the equivalent of using Excel 16.8 hours per week. In other words, the importance of Excel to Jane’s job and her score on a real-world task (building an Excel spreadsheet) are predictors of Jane’s performance against a “normative” population of Excel users who scored the same as Jane and listed Excel as important to their jobs—the rating of 4.
We now have all of the data to calculate the ROI of Jane’s seven hours of training, which cost Jane’s company $60, and the payback period. To do our calculation, we need to convert Jane’s salary into an hourly rate or $55,000/1,950 hours: $28.21 per hour.
In order to calculate the true cost of the training, we are going to add the seven hours that Jane spent off of her job while training to the $60 that the company paid for the program: (7 x $28.21) + $60 = $257.47.
To get to the economic benefit, we need to take Jane’s hourly rate and multiply it by her virtual weekly gain: $28.21 x 4.6 = $129.77. Before we calculate the ROI and payback period, let’s project Jane’s economic benefit from the training for three months, or 13 weeks: $129.77 x 13 = $1,687. We now have all of the data to do our calculations.
ROI = $ (1687- 257)/ $257 * 100 =556%
Payback Period = $257/ ($1687-$257/244) ) = 15 days
Jane’s Excel course has yielded an economic benefit over three months of $1,430 ($1,687 - $257), an ROI of 556 percent, and the cost of the training is paid off in 15 days.
A FINANCIAL UTILITY MODEL
Godkewitsch (1987) uses what he calls a financial utility model to get
some sense of the value to a company in monetary terms of implementing a training program. This model requires the ability to ascribe a measure to any given intervention or effect. The formula used by Godkewitsch (1987, p.79) to calculate the financial utility to a company of implementing a training program is described below:
F = N [(E x M)] – C
F = financial utility
N = number of people in the training program
E = effect or outcome of the training program
M = financial value of the effect or outcome
C = financial cost per person of implementing the training program
The formula for calculating financial utility can help us to determine the types of information that need to be gathered. Although it is relatively simple to gather information on the number of training participants and financial cost per person, it is more difficult to quantify effects of training and allocate a monetary value to this effect. Godkewitsch, however, provides us with a practical way of measuring these by using the concepts related to the normal distribution of scores. This concept goes something like this. In any normal distribution you will find a lot of scores near or at the centre, and fewer scores at the lower end of the distribution or at the higher end of the distribution. In addition, we can describe scores in terms of the extent to which they are grouped together. This is called the standard deviation. In any normal distribution, 68% of the scores fall within one standard deviation from the mean or central point of the distribution. We can use information of the distribution and the standard deviation to help us to determine the effects of the training program. First, we compare the distribution of scores on a given test (for example, results on a work satisfaction questionnaire, or standardized test for measuring, say, business writing skills) administered prior to and after the training program. We then compare how these scores are grouped for each of the distributions. If the groupings (standard deviations) have changed, then we can attribute the change to the effect of the training program. According to Godkewitsch, the next step is to quantify these effects in monetary terms. A practical way for attributing a dollar value to the effect of the training is to think of this in terms of the standard deviation.
This method places the value of the standard deviation of job performance at about 40% of annual salary. This means that an employee whose job performance is at one standard deviation below that of an average worker is judged to be of 40% less value than the average salary paid for that job. This worker is performing at the 15th percentile. If the worker’s job performance is one standard deviation above that of an average worker, then he/she is judged to be of 40% more value than the average salary paid for that job. This worker is performing at the 85th percentile.
A practical example
Let’s imagine the following scenario. More Australians Covered insurance company wants to know whether the management training programs that have been conducted for their supervisors during 2001 have changed supervisors’ performance on the job. The company has decided to use the financial utility model to arrive at this decision. This model requires information on the number of supervisors that have participated in the training program for the year, the average salary for the supervisors, and the average cost per person of the resources that were used. In addition, we need to gather information on the results obtained by supervisors on two tests of performance—one administered prior to the training program and the other administered after the training program was over. There were 50 supervisors that participated in the programs, they had an average salary of $50 000, and the average per person cost of resources amounted to $5000. In comparing the distribution and standard deviation of scores on tests, we find that there has been a shift in the standard deviations of both distributions of scores. This shift is the effect size or the changes that have been produced by the training program. The size in standard
A deviation of this effect is .68. As we have already noted, the formula for calculating the financial utility is
F = N [(E x M)] – C
In allocating appropriate figures for each component of the equation we
have the following calculation.
F = 50[(.68 x .4 x $50 000)] – $5000) = $430 000
This means that the overall worth to the company of conducting this
training program is $430 000, or $8600 per person.
This information, used for calculating the worth to the company of
Providing training to supervisors, enables the company to calculate the
length of time it would take to recoup expenses incurred in conducting the
training. The per person cost of the training would be divided by the gains
that had been experienced. $5000/(.68 x .4 x $50 000) = .36 months
We can also get a figure for ROI by subtracting the cost from the gain and
dividing the result by the cost. (.68 x .4 x 50 000)/ $5000 – $5000 = 172
This calculation has given us a cost/benefit ratio of 172, which means that our training program has been highly successful because it has shown a 172 improvement in supervisors’ skills—as measured by their results on standardised tests for measuring these skills.
THE NET PRESENT VALUE APPROACH:
Another technique that can be used to evaluate investment in training is to calculate the net present value (NPV) of a particular investment. The NPV provides a method for comparing the value of money now with the value of money in the future, taking into account all the costs that are associated with that money. Before we discuss the NPV technique, it is important to understand the concept of the time value of money. This concept is based on the premise that a dollar to be paid or received today is worth more than one to be paid or received in the future. You can use this time value for money concept to help you decide on choosing the best value for your money when you are deciding on a training program.
We can use the NPV analysis to determine ROI in training. This means that we will have to consider the costs associated with the investment and compare these to the future benefits that will be derived as a result of the investment. For example, income that would have been foregone during the training period costs of courses, learning materials and other related expenses would have to be considered as costs. The dollar value of income that can be directly related to the qualification will represent future benefits. In calculating the NPV for our investments in training we will need to use the following steps:
Step 1. Select the discount rate.
Step 2. Identify the costs and benefits to be considered in the analysis.
Step 3. Establish the timing of the costs and benefits.
Step 4. Calculate the NPV of each alternative.
Step 5. Select the offer with the best NPV.
For example, let’s decide that our discount rate (the interest we would have earned if we would have invested our money in another place, or at which money could have been borrowed) will not change over the life of the investment.
The present value of any income amount = (Income amount) / (1 + discount rate) to a power.
a is the number of years into the future that the income amount will be
received (or spent if the income amount is negative).
The NPV is the sum of these present values of the individual amounts.
To calculate the NPV we can use the following formula:
NPV = I0 +I1/ (1+r) = I2 / (1+r) 2 +… + In / (1+r) n
(I = income amounts for each year, the subscripts are the year numbers,
starting with 0. The discount rate is r. The number of years of the
duration of the investment is n.)
Factors affecting NPV:
The major factors that will affect the NPV are the timing of the expenditure and the discount rate. The higher the discount (interest) rate, the lower is the present value of expenditure at a specific time in the future. If $4000 were the present value of $4400 in one year’s time at an interest rate of 10%, this present value would be lower if the interest rate were doubled. Another factor that will affect the NPV is the decay of knowledge and skills over time. An estimate of this depreciation will also have to be factored into the identification of costs and benefits. Estimating the opportunity costs (for example, the value of foregone
wages, or other benefits) needs also to be considered. These costs are not
always easy to measure and are often based on estimates by knowledgeable
people in your organization
A practical example
Keeping in mind that NPV is the future value of the program in today’s dollars, we can calculate the NPV of a training program to decide whether it is a worthwhile investment. If we wanted to use NPV analysis in determining whether or not our company should invest in a particular training program, the first thing we would have to do is to identify a discount rate that will be used. For the purposes of this exercise we will use differentiated discount rates. These rates will be set at 10%, 20%, 25%, 35% and 150%. One reason for changing
the discounted rate is to ascertain how different rates of discount affect differences in performance.
The next step is to identify the costs or benefits using the NPV analysis provide one way of estimating ROI in training. However, it does not take account of a variety of factors, which may affect the outcomes that are required. Doucouliagos and Sgro (2000) have come up with an integrated approach to evaluating ROI in
training. This approach considers the multi-dimensionality of the influences on training outcomes. Associated with the program. There are two types of benefits or conditions; one relates to a 10% increase in competence, the other relates to a 3.3% increase in competence. The timing of the costs and benefits should be the next to be described. In this example a three-year time period is used. The NPVs for the conditions are derived using the NPV formula already discussed.
NPV calculated at 10% and 3.3% increase in competence, for different discount rates
Using the NPV analysis provides one way of estimating ROI in training. However, it does not take account of a variety of factors which may affect the outcomes that are required.
ROI at all the four levels of Kirkpatrick’s Model:
ROI calculations are possible at all levels of evaluation.
ROI at Level 1: Reaction Stage
When a level 1 evaluation includes planned applications of training important data can be collected that can ultimately be used in an ROI calculation. By including a series of questions that ask how participants plan to apply what they have learned and the result that they expect to achieve, higher-level evaluation information can be developed. Participants are asked to state specifically how they plan to use the program material and the results they expect to achieve. They are then asked to convert their accomplishments to annual monetary value and show the basis for developing those values. Participants can moderate their response by indicating their level of confidence with the process making the data more credible. In the tabulation of data, the confidence level is multiplied by the annual monetary value to develop a conservative estimate to use in the data analysis. For example, if a participant estimated that the monetary impact of the program would be Rs. 10000, but was only 50% confident, an Rs.5000 value is used in the calculations.
M&H Engineering and construction Company designs and construct large commercial projects such as plants, paper mills and municipal water systems. Safety is critical issue at M&H and commends much management attention. To improve the current level of safety performance a two days safety awareness program was developed for project Engineers and construction superintendents. The program focused on safety leadership, safety planning, safety training, safety meetings, accident investigation, safety policy and procedures, safety standards and worker compensation. After completing the training program, participant’s were expected to improve the safety performance of their specific construction projects at the end of the two day program, participants completed a comprehensive level one reaction questionnaire that probed into specific action items and their monetary value planned as a result of this program. In addition, participants were asked to explain the basis of their estimates and place a confidence level on them.
Only 18 of the 24 participants supplied usable data. The total cost of the program, including participant salaries was $29000. Prorated development cost were included in this figure the monetary value of the planned improvement is extremely high, reflecting the optimism and enthusiasm if participants at the end of very effective program. As a first step in the analysis, extreme data items are omitted. Each value is multiplied by the confidence value and totaled. The resulting tabulations yielding a total improvement of $655,125. Because of the subjectivity of the process values were adjusted by a factor of two, an arbitrary number suggested by the HRD manager and supported by the management group. This “adjusted “value is $327563 or $328000 with rounding. The projected ROI based on the end of program questionnaire is as follows:
ROI = $ (328000-29000)/$29000 *100 = 1031%
These calculations are highly subjective and may not reflect the extent to which participants will apply what they have learned and achieve results. A variety of influences in the work environment can enhance or inhibit participants’ attainment of performance goals, and high expectations will be met.
ROI at level 2: TESTING
Testing for changes in skills and knowledge is a common technique for Level 2 evaluation. In many situations, participants are required to demonstrate their knowledge or skills at the end of the program and their performance is expressed as a numerical value. When this type of test is developed, it must be reliable and valid. Consequently there should be a relationship between test scores and subsequent on job performance. When there is statistically significant relationship between test scores and on-the –job performance and the performance can be converted to monetary units, then it is possible to use test scores to estimate ROI from the program, using the following steps:
Ensure that the program content reflects desire on-the –job performance
Develop an end of program test that reflects program content
Establish a statistically valid relationship between participants’ test data and output performance.
Predict each participants’ performance levels with given test scores
Convert performance data to monetary value
Compare total predicted value of program with program costs.
ROI at level 3: JOB APPLICATION
In almost all training programs participants are expected to change their on job behavior through application of program materials.
The US government redesigned its introduction to Supervision course, a five-day training program for newly appointed supervisors. The program focuses on eight competencies:
Role & responsibility of supervisor
Planning, assigning, controlling and evaluating work
Leadership and motivation
Analysing performance problems
The new supervisors’ immediate mangers indicated that the above competencies accounted for 81 % of the first level supervisor’s job. For the target group being evaluated, the annual average salary plus benefits was $ 42202. Multiplying this figure by the amount of job success accounted for by the competencies (81 %), yielded a dollar value of $34,184 per participant. If a person is performing successfully in these eight competencies for one year, the value to the agency would be $34,184.
Managers rated the new supervisors’ skills for each of the competencies before the program was conducted using a scale of 0-9. The average level of skills required to be successful in the job was determined to be 6.44. The skill rating prior to the job was 4.96, which represented 77% of 6.44. After the program the skill rating was 5.59 representing 87% of the skill level needed to be successful.
Monetary values were assigned based on the participants’ salaries. Performance at the required level was worth $34184. At a 77% proficiency level, the new supervisors were performing at a contribution value of $26322. After training this value had reached 87% representing a contribution value of $29740. The difference in value $ 3418 represents the gain per participant attributable to training. The program cost $1368 per participant.
Thus ROI of the program
ROI at level 4: BUISNESS RESULTS
Most programs for which ROI has been developed the focus is on level 4 i.e. business results that have been influenced by the training program. These output variables are expressed in terms of cost reduction, productivity increase, improved quality, increased customer service or reduced response time. There are two critical issues involved in level 4 ROI development. The first issue involves isolating the effects of training. When a program has been conducted and output measures have changed, the first step is to determine the extent to which the training program changed the output variables. The second challenge is to convert business results to monetary unit using one or more strategies. The case study of Retail Merchandise Co. (RMC is an example how at this level ROI is calculated)
THE ROI STRATEGIC MODEL:
We have to remember that the Top management of a firm should be committee to implementing strategic changes to transform the organisation by improving its internal processes and increase the stakeholder value. But the bottom line profits left over after subtracting costs from revenues, is by no means the be-all and end-all of business results. ROI measures are not an all in all evaluation process for deciding the relevance of a training program. We have to focus on two things: strategy and outfoxing the competition. Results from an INFORMATION WEEK survey reveal, “more companies are justifying their e-business ventures not in terms of ROI but in terms of strategic goals. Creating or maintaining a competitive edge was cited most often as the reason for deploying an application”. So we need to think both from the strategic point of view and from the ROI point of view when we decide to go for a training program.
So while taking a decision about a training program it becomes extremely important that we focus on both:
Strategic relevance of the Training Program in terms of overall business strategies
Return on Investment from training program, as ROI can justify the training quantitively.
Keeping these two objectives in view I have tried to develop a model for justifying training program with the help of the Portfolio Strategic Approach or The BCG matrix. This model will help us in determining which training program will serve the purpose of the organisation best, which one needs improvement, and which initiatives are not serving any purpose and hence should be eliminated.
TRAINING RELEVENCE MATRIX
STAR: A training program can be called a star program when it is of high strategic relevance and the ROI of the training process is also high. So there is no question about the implementation of the program, all other programs should be oriented towards this direction.
?-QUESTION MARK: A training program can be called a “?” Program when it is of high strategic relevance but is showing a low or negative ROI at present. Since the process is of high strategic relevance it is undoubtedly important for attaining the business results. But if it continues to yield negative ROI then the top management has to think about then how long they can continue with a cost inefficient program in today’s business world where the focus is on developing cost efficient business processes. The training team has to think of methods of converting this process into a star project by making it more cost relevant.
CASH COW: A training program can be called a cash cow program when it is of low strategic relevance but the ROI of the training process is high. These are programs, which are cost effective and are very popular through out the organisation. But these programs are most likely the ones, which the training department should eliminate in times when the organisation is going through hard times. These programs are not of much help when it comes to meeting business goals but are generally like routine training programs designed by the HR department, and are loosing focus in today’s world of scarce resources and strategy driven business initiatives. Attempts should be made to attach strategic relevance to them and convert them to star programs in case they are very popular and the training department does not want to do away with it.
DOGS: A training program can be called a dogs program when it is of low strategic relevance and the ROI of the training process is low or negative. With out a second thought these programs should be eliminated , as they only result in cash drain for the company with out serving any purpose.
Based on the above discussion I would like to conclude that a training program should be evaluated based on three factors
Strategic alignment of the training with the Companies business goals
ROI of the training program
Perceived value of the training in organisational development
Although ROI calculation processes mentioned above is sufficient to quantify the expected impact in the form of a forecasted range of financial values, observation shows that it is often insufficient to evaluate or compare all the benefits. Management is often quite willing to discount a very tangible financial measure in favor of less tangible measures, such as business alignment, synergies, branding, and other non-economic indicators. Although these indicators may seem subjective, this seeming tolerance for ambiguity actually reflects the objective values embodied in a strategic direction. The interaction between the “strategic value” (or strategic alignment) and financial values can skew the final evaluation to the extent that outsiders may think beyond reason. However, this interaction actually takes place within well-defined parameters. 13th October 2006 From India