Precious few personality assessments are known to be reliable, and researchers say their use outside academia is debatable
By Angus Chen
If you’re looking for insight into the true you, there’s a buffet of personality questionnaires available. Some are silly—like the internet quiz that tells everyone who takes it that they are procrastinators at the core. Other questionnaires, developed and sold as tools to help people hire the right candidate or find love, take themselves more seriously.
The trouble is, if you ask the experts, most of these might not be worth the money. “You should be skeptical,” says Simine Vazire, a personality researcher at the University of California, Davis. “Until we test them scientifically we can’t tell the difference between that and pseudoscience like astrology.”
Personality questionnaires began evolving about a century ago, says Jim Butcher, an emeritus psychologist at the University of Minnesota. “They started asking questions about an individual’s thinking and behavior during World War I,” he says. “These were to study personality problems and mental health problems.” And importantly, he adds, the U.S. military wanted the questionnaires to help weed out soldiers who weren’t fit to fly military aircraft.
According to Butcher, during the first half of the 20th century, many academics started creating different personality scales. “Not just on mental health diagnoses, but what personality is like,” he says. The problem with practically all of the assessments at the time as they were built on the creators’ subjective feelings about personality, he notes. “Then people started to raise questions about do they really measure what they think they’re measuring? How reliable are those conclusions, and are they valid?”
Butcher describes what followed as a mass culling of personality systems and questionnaires by the scientific method. There is one personality model that did survive the 20th century, though. It is popular among academics today and is what Vazire uses in her research. It’s called the Big 5 Personality Traits (aka 5-Factor Model), and it was developed over three decades beginning in 1961 at Brooks Air Force Base. From then to the 1990s, several psychologists including Lewis Goldberg, Warren Norman, Paul Costa, and Robert McCrae helped develop the model into its modern form.
Vazire says in developing the Big 5 Personality model, psychologists tried to avoid pitfalls that plagued early personality researchers—like selecting criteria based primarily on intuition. Instead, the Big 5 model took a holistic tack by compiling every word that could be considered a personality trait and creating simple, straightforward questions about them. For example, on a scale of 1 to 5, are you outgoing, sociable? Have a forgiving nature? Based on how people answered initial surveys, researchers used statistical methods to group traits that seemed to cluster together (like “talkative” and “sociable”) into five basic categories: extraversion, conscientiousness, agreeableness, neuroticism, and openness to experience. The other model, the HEXACO model of personality structure created in 2000 by psychologists Kibeom Lee at the University of Calgary and Michael Ashton at Brock University in Ontario, is similar but adds an extra category: honesty-humility.
The key to the Big 5 model is its simplicity. It doesn’t sort anybody into a “type,” just informs them where they fall on a continuum of personality traits. There are no tricks and no surprises to be revealed, Vazire says. “In a way, it’s disappointing. It just means that personality tests can only tell you what you tell it.” You won’t learn anything that you didn’t already know about yourself, she adds, and its accuracy comes entirely from how honest and self-reflective you were with your answers.
At best, Vazire says you could use it as a comparative tool that can tell you how you rank on extroversion compared with others who have taken the same test. There have been studies that show certain Big 5 factor scores correlate with certain outcomes—conscientiousness correlates with longer life, for instance, and extroversion correlates with higher sales for sales reps. “But that doesn’t mean someone with high extroversion will be a better salesperson,” Vazire says. Correlations are just that; they could be incidental. But commercial personality assessments seem to depend heavily on such correlations. For example, one assessment from The Predictive Index, a company that measures behavioral characteristics and matches personality profiles to jobs, views such correlations in their own studies as a measure of success. “[We showed] in one client, a retail jeweler, that increases in dominance or aggression were responsible for $125,000 in revenue,” says Thad Peterson, one of the company’s executives. The idea behind The Index, Peterson says, is to use those measures to help “marry people to [job] positions.”
Such personality assessments—particularly those targeted toward hiring recruiters and managers—aim to uncover a kind of “hidden truth about the person,” says Randy Stein, a psychologist at California Polytechnic State University, Pomona. “They assume that there is an essence of you and an essence of the job, and you should be matching up those two things in hiring,” he says. “But I don’t think there is a hidden truth—and even if there is, a personality test doesn’t do it.”
Like the Big 5 model, any personality or behavior assessment can’t know things you haven’t explicitly answered in the questionnaire, Stein says. Sometimes commercial personality tests ask odd questions—like, Do you identify with snakes? or How do you react to a certain color?—and try to draw inferences from your answers. Those kinds of conclusions venture into the pseudoscientific, Stein says.
There are other reasons why Stein thinks some personality assessments may be pseudoscientific. “What those tests will tell people is true or false is determined by what people are willing to pay for,” he says. “Their process as a company is to tell people whatever will sell the product.” By contrast, the Big 5 and HEXACO models were shaped by an empirical process and independent peer review that showed people’s scores tended to be consistent, and predictions made using the models are reproducible. Without that, Stein says personality tests should be treated with extreme suspicion.
Some companies like The Predictive Index say their product meets such standards. The company invested in an audit, paying over $20,000 dollars to Norwegian classification firm DNV GL to review their product and certify that it complies with a standard set by the European Federation of Psychologists’ Associations. Two Index representatives, Greg Barnett and Austin Fossey, also say predictions based on their methods are accurate.
Perhaps. U.C. Davis’s Vazire says it is fairly easy to reach some level of validity. “If I just asked you to make a questionnaire on extroversion, you would probably do a pretty good job,” she says. It is because we are all judges of character, and we often do well at intuiting whom to date or hire and who we are, Vazire says. If the process seems confusing or if questions veer off into the abstract, that’s a red flag. Personality, she says, is just not that mysterious.
