I just finished this task a few weeks ago. My office has only 18 staff, but they are all IT developers. It was a tough task.
Identifying Important Criteria
First, you will need to identify which criteria are important for the staff: punctuality, knowledge of the job, soft skills, leadership skills, etc. The more detailed the criteria, the better it is. Typically, people use a 3 or 5 score system for each criterion from basic to professional. I developed a 5-level system to evaluate from fresh graduates to managers.
Defining Levels Based on Requirements
Next, you will need to provide definitions for each level based on your job requirements and the company's or boss's expectations. For instance, in my office, for punctuality, a staff member is scored 1 if they arrive late more than 5 days a month, 2 if they are late 3 days a month, and so on, with a score of 5 if they consistently arrive early and positively impact other team members.
Aligning with Management
After that, you should meet with the boss and other managers to ensure that your definition table aligns with their expectations.
Implementation and Results Gathering
Once you have the final, perfect table, instruct people on how to use it and then gather the results.
Good luck!