Employee theft if a serious problem, especially for the retail industry. Inventory shortages often range from two to five percent of sales and over half of these loses are usually attributed to employees. Many retail companies rely on pre-employment screening tests to help them hire applicants who have lower probabilities of stealing from the company.
When human resource managers consider using an honesty test, the first question they ask is “How reliable is it?” Although reliability is important, what most people really want to know is “How valid is it?”
Both reliability and validity are important characteristics of all personnel tests, including honesty tests. Before a company uses a test to hire new employees, it should make certain that the test is both reliable and valid.
Reliability refers to the repeatability of the test. A reliable test produces approximately the same score for each applicant every time it is administered. Test specialists define reliability as consistency of measurement.
The best way to estimate the reliability of a test is called the test-retest method. This method involves administering a test to a group of people two different times with a brief interval between administrations and correlating the two scores. If the correlation coefficient is greater than 0.80, the test is considered reliable, although correlations as low as 0.70 are sometimes considered acceptable.
To illustrate how reliability is estimated, the reliability of an honesty test was examined by asking 44 college students to take the exam twice with a one week interval between. Twenty-five of the students were undergraduates and the other 19 were graduate students. The correlation coefficient was 0.83, suggesting that the test is reliable. This correlation is slightly lower than what has been found on other occasions, probably because the students were not applying for a job. Four students had identical percentile scores on both tests, while one student had scores that differed by 39 percentile points.
Validity refers to the job-relatedness or the predictability of the test. A valid test measures what it is expected to measure. Since personnel tests are used to hire people who will be good employees, a test is considered valid if test scores are correlated with measures of behavior or job performance.
Honesty tests are supposed to measure a stable personality characteristic that somehow influences whether people tell lies, give false impressions, or take things that do not belong to them. Employees who steal money or merchandise from stores are considered dishonest and these people would not be expected to obtain very high scores on an honesty test.
Validating an honesty test is not an easy task, but the evidence is accumulating. There are three different methods of demonstrating validity — content, concurrent, and predictive — and all three have been used to show that honesty tests are valid.
The first method, called content validity, simply consists of examining the content of the test and subjectively assessing what the test appears to measure. Most honesty tests contain items that ask applicants about their attitudes regarding theft activities, the prevalence of dishonesty in society, the punishment of theft, and how they would respond to temptation. Since these tests appear to be measuring honesty, it is appropriate for someone to say that they have content validity. But content validity does not indicate whether someone with dishonest attitudes would therefore steal on the job.
Concurrent validity examines the relationship between test scores and the performance of present employees. We would say that an honesty test has concurrent validity if people who are honest also have high test scores. A major hurdle in validating an honesty test is measuring the amount of money or merchandise employees steal — only a fraction of those who steal are caught.
One method of overcoming this measurement problem is to ask people to report their own personal honesty. Although self-admissions have to be treated with some caution, most people are willing to openly describe their dishonesty, especially if they believe it is for research purposes or if it is obtained anonymously.
To illustrate concurrent validity, five classes of students studying human resource management at two different universities were asked to complete the Applicant Review. They were also asked to answer four questions regarding how often they (a) lied to their spouse or family members, (b) stole merchandise from stores, (c) cheated in school, and (d) lied to friends. They indicated whether they did it always, frequently, occasionally, seldom, or never and their answers were scored 4, 3, 2, 1, or 0, respectively. The four scores were added to create a scale called “Admitted Dishonesty” that had a possible range from 0 to 16. The data for the five classes are shown in the following table:
Class # Students Average Honesty Average Dishonesty Correlation
Coefficient
1 23 51.5 3.0 -.68
2 25 56.0 2.2 -.65
3 25 65.6 2.6 -.47
4 30 38.0 2.5 -.38
5 44 62.2 2.5 -.26
These correlation coefficients are very impressive for this type of field research. The correlations are negative because students who admitted more dishonesty had the lowest honesty test scores. Negative correlations are just as useful as positive ones. While reliability correlations have to exceed .80 to be considered impressive, validity correlations above .40 are usually considered quite good. Correlations above .60 are rather unusual. Although the lowest correlation (-.26) is rather modest, it is statistically significant at the EEOC’s recommended level of significance (.05). These correlations are even more impressive when one realizes how narrow the range of scores were on admitted dishonesty. Predictive validity is measured by obtaining the honesty test scores before applicants are hired and then correlating their scores with the amount of money or merchandise they steal after they are hired. Although predictive validity is considered the best kind of validity evidence, it is extremely difficult to obtain for honesty tests. Companies cannot reliably measure the amount each employee steals. Consequently, the validity of honesty tests has been challenged because of the lack of predictive validity evidence.
Even though companies may not catch all theft, what they do catch can be studied. John Bernardin and Donna Cooke conducted a predictive validity study of an honesty test by collecting data from a convenience store chain during a three-year period (Academy of Management Journal, 36, (1993): 1097-1108). The honesty test produced a percentile score for each applicant where higher percentiles indicate higher honesty. The company provided data for all employees (N=54) who were hired and subsequently terminated for theft. These 54 people were compared with 57 other employees who were selected because they matched the theft group on dates of employment and conditions of work.
The comparisons between the theft group and non-theft group revealed that the average honesty score of the theft group was 10 percentile points lower than the non-theft group. However, there were no significant differences between the groups with respect to age, race, and gender.
These reliability and validity studies suggest that an honesty test is a useful instrument for hiring employees who will occupy positions of trust and handle company money.