Jack Phillips who proposes a 5th level to Kirkpatrick's model, which is Return on Investment or ROI, suggests to evaluate at levels 4 and 5 you need to evaluate to some extent at the lower levels too.
I'm not so sure that is necessarily the case.
I think the best way to determine how effective a training course has been is to be very clear before attending the course just what outcomes are required - what new knowledge, skills, attitudes and behaviours are required - of course the assumption is that the training actually addresses these areas - perhaps not in one course but several?
After the training ask the managers or reporting officers of those people who attended the course how many of these objectives of knowledge, skills, attitudes and behaviorus have been met.
I ask reporting officers to state their requirements in terms of key performance indicators, and in terms f being able to do specific things, using for format of TASK, CONDITIONS (i.e. what conditions to do task under, e.g. at night, or with a particular tool etc) and STANDARDS, i.e. how this task will be measured and to what standard of success, for example "in accordance with BS 7799", or "according to the Standard Operating Procedure #12-67 and with no errors" or "45 words per minute with 95% accuracy".
I use a simple form - on side 1 is the name of the employee, name of reporting officer, department name, title, date and venue of the training, costs of the course, accommodation and travel, and then below this space for justifying the training - both by the employee and the reporting officer, followed by Side 2 where the objectives are listed along with how success for each will be measured. Then below this is space to put notes on an after-training meeting and details of an action plan to take the lessons of the training back to the workplace - actions, resources needed, responsible person for each action and deadline. I also use this form then to track the financial costs, to audit the training request and delivery process and to hold reporting officers and employees to account and see how they are progressing.
I'll attach a copy to a later post and also put it on my web site later.
Regards
Martin