Copyright © 2006, Boyd Associates Page 1
The lack of Accuracy and Credibility of
Self-Reported Pay on the Internet
H. Michael Boyd, Ph.D.
“Well, It’s on the Internet, boss. It must be accurate.” It’s like
the tabloid newspapers that report the birth of the alien child.
People often believe it because it is in print. The Internet is
sometimes like the tabloids. The question is whether the
information found there is true or not. Occasionally, the
employee brings pay data they found on the internet to HR to
support their claims of being underpaid (I haven’t heard of
anyone claiming they are overpaid, yet.). What do HR
professionals think?
.. Overall, HR professionals reject the practice of attaching
credibility to self-reported pay data, or of accepting it as
accurate data.
.. 95% of HR professionals agree that self-reported pay data
found on the internet is not accurate.
.. 79% of HR professionals will not use this information in
determining pay.
Copyright © 2006, Boyd Associates Page 2
Human Resource Management Demands Professional
Compensation Management.
One of the most vexing problems faced by modern human resource
management is how to assure that the employees of the organization are
being compensated appropriately for their contributions. One critical part
of that determination, of course, is whether their “pay” is the correct
amount within the context of affordability, equity, and market conditions.
Affordability and equity are generally (not always, but usually) internal
organizational issues that management can rely on internal data and
information to analyze and manage. Determining market conditions,
however, has always been difficult.
Over the past century the methodologies, practices, and science of arriving
at market pay information and intelligence has evolved from chamber of
commerce and industry and professional association information sharing to
sophisticated survey design, implementation, and analysis. While simple
arithmetic might have been sufficient in 1950, advanced mathematics, and
statistical analyses are required today.
In 1974 I participated in a position comparison project being led by IBM
compensation to establish benchmark positions for pay surveying for
electronics technicians and engineers in the electronics industries. The
effort was all manual and face-to-face. This was before PCs, email, and
the Internet. The process took almost a full year. It was normal in 1974
for published compensation information to be six months to a year old by
the time it was used by an organization as a component of their
compensation and pay range determination. Depending on their salary
planning cycle and timing, it could mean that some individual’s actual pay
rate could end up being effective as much as two years after the point in
time that the original data was collected. Companies adjusted their ranges
and midpoints by factoring in trends and anticipated market changes; but,
at best, it was an educated guess.
Accuracy Requires Science and Rigor.
In the 21st century the compensation profession relies heavily on expert
external compensation professionals who have specialized in producing
market pay data and analyses. Consulting and human resource
management service firms began collecting data continuously and applying
All compensation
considerations are
within the context of
affordability, equity,
and market
conditions.
Copyright © 2006, Boyd Associates Page 3
the most sophisticated mathematics and data processing technologies
available to determine market averages and information. Initially, these
firms relied heavily on their math experts and data processing resources.
The work was primarily data management. As technology improved, the
firms became expert at insuring the quality of the data collected through
survey design and implementation expertise.
Survey methodologies and accuracy are fully dependent on the willingness
of company compensation professionals to share their confidential and
proprietary employee pay data with the people collecting the data and
managing the survey. Participation not only requires willingness, it also
must provide resources. Someone must collect, prepare, and submit (enter)
the data. Generally, the return benefit for participation is access to the
aggregated pay ranges and data.
The new tools of the 21st century compensation profession such as more
sophisticated computers and compensation management application
software packages coupled with advanced communications capabilities
such as the Internet and shared database and application software services
allow market pay averages and ranges to be determined and applied to a
company’s pay programs almost dynamically, if desired. Such a practice is
inadvisable, but technically possible. No company need find their pay
scales and actual pay as much as two years out of date any longer.
Of course, what I have described is costly and resource dependent. Every
compensation function finds itself scrutinized by company management
regarding its budget including how much it spends to obtain market pay
data.
The Question was about self-reported pay information
accuracy and credibility.
The purpose of this paper is to address a question I was asked about the
accuracy and credibility of different survey methodologies, practices, and
data. Specifically, I was asked about the use of self-reported (by
employees) pay information in determining pay averages that could be
used to construct company pay plans.
My quick response was that collecting pay data is much too complex a
process for anyone but a trained compensation specialist to be able to do
I was asked
about the use of
self-reported pay
collected on the
Internet in
determining
company pay
averages.
Copyright © 2006, Boyd Associates Page 4
accurately. Self-reported data does not differentiate based on job content
or performance requirements – only on entered job title. Actually, in many
cases the internet site uses a drop-down selection of job titles to choose
from; so, it depends on what title the person entering the data believes is
closest to their position. We all are aware that an accountant at one
company may do a very different job than an accountant in another
company; but the job titles captured in the internet application are the
same.
The definition of what the employee believes (or reports) constitutes “pay”
can differ from person to person. No matter how you ask the question,
people will often report what they feel should be their pay, or what they
want their pay to be. This is particularly true in the internet applications
because there is autonomy and no probability of challenge. Salary,
overtime, bonuses, incentives, profit sharing, benefits, etc. are components
of compensation that may be included or excluded by the employee
depending on the salary figure they believe is justified. While there are
other reasons why self-reported data, the two just presented ( lack of
comparability and perception of what pay amount should be entered) are
sufficient to make self-reported pay data highly inaccurate and undeserving
of credibility.
The question was asked because of the cost pressures that compensation
departments are always under and because of the proliferation of internetbased
self-reported pay information. There are thousands of internet sites
that collect and report people’s pay as those people enter it (In many cases,
it is a choice among salary ranges that the participant selects from). There
are hundreds of job boards, for example, that collect pay data as a part of
the individual profile for each job seeker. It is reasonable to assume that
the majority of job seekers will enter pay data that they want perspective
employers to accept as their current pay and worth. There are thousands of
business, industry, professional, news, magazine, and periodical web sites
that ask their visitors to enter their pay data. When I was managing a
human resource research practice at a major research firm, survey
participants that I interviewed freely admitted that the personal salary data
they entered in web sites were most often an amount that they felt it
“should be” rather than the actual amount.
There are
thousands of
internet sites
that collect
and report
people’s pay
as those
people enter
it
Copyright © 2006, Boyd Associates Page 5
It was important to ask the human resource
professionals what they think.
While the answer to the question of internet collected self-reported pay
accuracy and credibility seemed obvious, it seemed appropriate to find out
what the human resource profession, particularly the compensation experts,
felt. A very brief and quick survey (A copy is attached at the end of this
paper.) was created and sent to several thousand human resource
professionals, most of whom have some responsibility for compensation.
410 people participated in the survey; and 95% do not accept self-reported
pay data found on the internet as accurate.
We also asked how much credibility they attached to self-reported pay
data. This is different from the question of accuracy because it is an issue
of behavior rather than a technical compensation issue. Only 7% of the
participants attached more than 50% credibility to such data, and 31%
attached 0% credibility to that data. This question really goes to the
behavior of people reporting their own pay data on the internet. 27% of
our survey participants felt that the data collected would be 50% credible
and 35% felt it would be about 25% credible.
Overwhelmingly,
HR professionals
attach little
credibility to selfreported
pay data
It seemed
appropriate to
find out what
the human
resource
profession,
particularly the
compensation
experts, felt.
Just about
everyone
believes that
internetcollected
selfreported
pay
data is not
accurate.
Copyright © 2006, Boyd Associates Page 6
Credibility of the data is based on assumptions about the people selfreporting
their data. Among the assumptions our survey participants had
were:
.. There is a machine distancing effect from the internet that allows a
perception of autonomy and lack of responsibility.
.. Many people intentionally enter false data.
.. The autonomy of the internet creates emotional detachment and
less consideration of personal ethics.
.. People are not able to relate their job title to an industry-standard
job title.
.. There is no ability to compare one person’s pay to another’s.
.. Some also believe that some internet based reporting intentionally
misrepresents the information.
A final question in the survey asked how the survey respondents “would
respond to an employee who brought self-reported pay information that
they found on the Internet and asked you to use it to calculate their salary?”
79% of the participants would either tell the employee that the data isn’t
accurate, or they wouldn’t use the data even though they had told the
employee that they would consider it. 14% would consider the data in
reviewing that particular individual’s pay. There is some logic to that
answer because the employee has an expectation that it will be considered.
It can be considered and rejected as invalid.
The majority of
HR folks would
not use the data in
pay determination
even if they let the
employee believe
they would
consider it.
Copyright © 2006, Boyd Associates Page 7
Overall, the survey participants validated the belief commonly held across
the human resource management profession that self-reported pay data
found on the Internet is both inaccurate and without much credibility.
Most HR professionals would not use the data at all.
The conclusion is simple and straightforward.
So, as I always challenge the students in my human resource management
courses: “So what?” The real learning is that the common sense and
wisdom of the human resource management profession is still intact.
While it would be easy to use cheap and easily accessible internet selfreported
pay data, it is not accurate nor credible, and professional
excellence requires that it be dismissed.
Hundreds of HR professionals offered comments.
Following are some of the comments offered by survey participants:
They answered the question:
“What is your opinion of the practice of collecting self-reported pay data,
and then representing it as accurate pay information?”
.. I think the web can provide valuable data, but self reported pay doesn't include
the list of variables taken into consideration when looking at salary.
.. I think some individuals are very honest when they participate in on-line surveys.
However, I think some may inflate their information or may include other items
included in compensation such as annual bonuses, stock options, spot awards or
target or expected bonuses, stock options etc....
.. There is no system of checks and balances. The pay data may be accurate but is
the actual position benchmarking accurate? In addition, those sites do not provide
participant data. It is, therefore, difficult to be certain that the data is not only
accurate but valid and comparable to your peer group.
.. I think that it may be of some value in situations where market data is otherwise
unavailable. However, I would not present it as the basis for my
recommendation.
.. Because people tend to inflate their earnings, we do not use this when reporting
our data to staff on pay scales, etc. I feel that you get what you pay for...if the
survey is a 'free' online tool, and you don't know who is compiling the data, as
well as what cuts are being used, it is difficult to assess it's validity.
Generally,
participant
comments offer
critical and
informed
explanations of
why the data is
not useful.
Copyright © 2006, Boyd Associates Page 8
.. It's irresponsible since we do not have any backup information on this data -
company names, sizes (employee/revenue) etc. There are more questions than
answers here.
.. It is unreliable at best and flagrantly biased at worst. It can not be trusted and
should not be used by compensation professionals.
.. This practice is understandable, as employees feel that the process used to
determine their pay is shrouded in mystery. Even with rigorous statistical
methodologies, the professional HR survey houses (Towers Perrin, Hewitt,
Watson Wyatt, Mercer, etc.) still exhibit quite a wide variation in market-based
pay levels for a given job; without the independence and rigor of such an
approach, I feel that self-reported pay data is almost guaranteed to be inflated and
misleading.
.. I find it misleading and inaccurate. Most users of such data look for a job title
only, with no additional consideration for factors that are vital for validating pay
levels. This is totally incorrect, as comparisons should be based upon job
content. When you add additional variables such as industry type, geographic
location, public Vs. non-public, and company size, you can understand why most
compensation professionals completely ignore self reported data. Most
companies measure themselves against a specific labor market based upon a
select group of companies and further broken down by location. Furthermore,
where they position themselves against this sub set of the market is a product of
their philosophy. As such, base pay is much more complex, and is the result of a
total compensation strategy that should encompass base, variable, equity and
benefits.
.. I put it slightly above reports of the Loch Ness Monster and other unverified
natural/unnatural phenomena. There could be a grain of truth in it but I have yet
to see it when compared with legally done surveys.
.. First, people typically have an inflated sense of what they're worth. If they only
report what they actually 'make', then that may be all they get offered in a
comparable job. I believe they tend to report what they think they 'should' be
making. Secondly, there are a lot of different factors behind why an individual is
paid what they are paid. There are variations of job responsibilities as well as
how people perform those responsibilities and what other skills sets they may
bring to that particular job. All of those factors are taken into account when
paying a person. Finally, when data is gathered through a survey house (vs. selfreported),
there are statistical tests/analysis to try to weed out some of the
anomalies that will be imbedded in self-reported.
.. I have worked in HR and Compensation for 20 years. Employees can't read their
pay stubs much less accurately record their annual salary on a survey. Plus,
many of these surveys ask for total pay, including overtime.
.. I have a very low opinion of this type of data. My major issue is you have no
assurance that equal or equivalent jobs are being matched. For the most part the
match is based on title alone, and therefore it is highly unreliable.
Copyright © 2006, Boyd Associates Page 9
.. Most of the inaccuracies in self-reported pay is the lack of knowledge in placing
yourself in the correct job. The other issue is the lack of knowledge of the
individual using the pay data in matching themselves with the correct job. Also,
in the collection of data, if it is not organized by type of industry, location of
industry, other inaccuracies are introduced into the picture. In terms of question
2, I usually push back with the surveys we use to benchmark our jobs. Not that I
do not look at other data, but more for a feel in the market, not to benchmark the
job.
.. The problem with self-reported pay is not just inflation of the amounts but that
fact that you have no idea what level in a job group the incumbent comes from.
A branch new accountant might be looking at salaries reported by people with
long-term experience.
.. The accuracy of the data reported is subject to the interpretation and honesty of
the individuals submitting the information. Here are some of the reasons I don't
use self-reported data: * Job matching may not be accurate * Individuals may
not enter their correct salary (for various reasons, including privacy) * Survey
providers go through a rigorous process to analyze data before it is summarized,
validating job matches, removing outliers, etc.
Copyright © 2006, Boyd Associates Page 10
SURVEY INSTRUMENT
SELF REPORTED PAY – REAL or IMAGINED?
Recently, I was asked about the credibility of self-reported employee pay data.
The question was: “Could you take the pay amounts that individuals enter on various internet sites (like
job board sites, for example) and roll them up to determine accurate pay averages?”
I would really like your professional opinion by completing a short online survey titled “Self Reported
Internet Pay - Real of Imagined?”
Thanks for your participation. I will send you a summary of the survey results.
Best regards,
H. Michael Boyd, Ph.D. (mboyd@Bentley.edu)
Professor of Human Resource Management, Bentley College, Waltham, MA. USA
1..Do you accept self-reported pay data found on the Internet as accurate?
2.. As a professional, how would you respond to an employee who brought self-reported pay information
that they found on the Internet and asked you to use it to calculate their salary?
3.. How much credibility do you attach to self-reported pay data?
4.. What is your opinion of the practice of collecting self-reported pay data, and then representing it as
accurate pay information?
Include your email address and I'll send you the summary when it's completed.
Professor H. Michael Boyd, Ph.D.
Bentley College; 175 Forest St., Waltham MA 02452
[Login to view]
Note: There was an additional question on the survey that only 2 people answered.
Copyright © 2006, Boyd Associates Page 11
H. Michael Boyd, Ph.D.
Dr. Boyd is a full-time Professor of human and organizational resources at Bentley College and President
of Boyd Associates consulting to enterprises from Fortune 100 companies to startup businesses.
Dr. Boyd is an internationally recognized expert in the field of human resources with over 30 years of
corporate and consulting experience and is professionally active as an educator, consultant, writer, and
speaker. He founded Boyd Associates to focus on human resources strategies critical to organizational
success. As the creator and Practice Manager of the Human Resourcing Strategies program at IDC he
concentrated on critical workforce factors including hiring, retaining, developing, and managing the 21st
century workforce. Prior to becoming a professor, Dr. Boyd was a key contributor at IDC,
Quantum Corporation, Digital Equipment Corp., Honeywell Inc., The Foxboro Company, GTE Sylvania,
Union Carbide Corp., and the U.S. Army (Medical Corps).
Dr. Boyd is professionally active in the HR profession as an instructor, lecturer, consultant, and
contributing author. He is a founder of the Association of Employment Professionals; was President and
founder of the New England Personnel Network (NEPN); served as President of the Norfolk County
Personnel Association; executive committee member of the Electronics Industries Personnel Association;
Vice-President of the Association of Human Resource Systems Professionals; Chairman of the
Massachusetts area public welfare advisory board; and, an arbitrator for the Industry/Agency Arbitration
Committee. In addition, he has been an active member of the Northeast Human Resources Human
Resources Association, Society for Human Resource Management, American Society For Training and
Development, Human Resource Planning Society, Human Resources Council, and American Sociological
Association.
He received a BA in Pre-Law from The Pennsylvania State University, a MS in Management from Lesley
University, and a Ph.D. in Sociology from Northeastern University.
Http://www.BoydAssociates.net --- [Login to view]