OA Part 4. How to Recognize Old Assessments

Welcome to our new series called Observations on Assessments. Over the next 10 weeks Chuck will dive into the in's and out's of the assessment industry. Stay tuned, we will have new posts up every Thursday!

​There are some simple ways to recognize old assessments, even when they look new in the marketing materials and on the Website. The simplest way is to look at the format of the questions, or items as they are called in the psychometric world. In the last 80 years, scientists have learned more and more about how to build better and more effective items. This means that newer items help the participants answer the questions in such a way as to produce a clearer and more accurate picture of who they are. They do a better job of getting frank and honest responses. ​

The oldest and simplest form of personality assessment is an adjective checklist. It consists of a list of adjectives, such as persuasive, timid, shy, direct, enthusiastic, reserved, organized, and messy. The participant is instructed to select those adjectives which they feel best describe them. They are then asked to respond to a second list of the same adjectives by selecting those adjectives that their friends might use to describe them. This type of exercise can certainly be fun and provide the basis for an interesting discussion. However, few sales job candidates select timid, shy, reserved, or similar adjectives that would be seen as negative or undesirable characteristics for sales positions. The failing of this type of item format is that the value of each answer is transparent concerning a type of job. This format has been widely sold since the 1950s with little change to its simple model. 

The most common form of item is that of a forced choice, in which the participant is asked to choose which of four words or phrases is most and least like themselves. This is the basis of DISC-type assessments. This basic format was developed by William Marston in 1928. It was an improvement over the adjective checklist because it was more difficult for participants to perceive the “better” answer. Unfortunately, a more serious problem lay within the forced-choice format. It produced ipsative scores, a psychometric term from the Greek word ipsa, meaning self. Essentially, the scores from a forced-choice questionnaire are self-referential, or meaningful only to the participant. The scores of one individual cannot be meaningfully related to any other individual, even though the questionnaire and the terms are the same. The reason is that the most - least choice suggests a difference or gap between the individual’s preference for each of these choices. However, there is no way to determine how wide the gap is. It is not known whether one person saw little difference in the choices or whether they were seen as miles apart. This means that these types of assessments cannot be used for hiring decisions, comparing people, or creating norms. This does not mean that such assessments are not sold for those purposes or promoted in doing that. 

More information on the qualitative differences in assessments is available at www.aboutassessments.com. Part 5 will continue the discussion on how to tell the age or capabilities of the many assessment products competing for your business.