The problem with standardized assessments in hiring

Today, most of the standard assessments that companies rely upon to evaluate job candidates are aptitude, personality, and behavioral ones. While this type of psychometric testing is indeed typical, it’s also incredibly problematic, because potential employees should instead be considered based on the skills they would bring to the particular role they are up for. Often, smart, resourceful, and talented individuals see opportunities disappear in front of them solely because of the results of a traditional test.

Janice Burns, Chief People Officer at Degreed—a technology suite for lifelong learning and data-driven development—works to spread the word on the inherent biases in these types of assessments, as well as the necessity for workplaces to opt to test candidates based on skills instead. Burns, whose background as a public-school teacher largely influenced her work in this space, sat down with us to discuss the ins and outs of psychometric versus skill assessments, why there is so much confusion around how to best test job candidates, and how companies can do better in this area.

Can you offer some details on your background and how you came to work in this space?

My current position is Chief People Officer at Degreed. We work with a lot of companies to help upskill their employee base and democratize learning within their own work environment so that people have better access to learning and upskilling resources.

Prior to joining Degreed three years ago, I spent most of my career at MasterCard. I started there in product management, and eventually moved into human resources, starting their first diversity initiative and heading up talent acquisition. In the last seven years at MasterCard, I was the Chief Learning Officer. But my first job in my career was as a teacher in the New York public school system.

Wow! How did your experience in education influence the work you do today?

I’ve always had a passion for access and equity within education. But there were a lot of things that happened during that year I was teaching in the New York public school system that made me start questioning standardized testing and how it was being applied in education. I saw it gain more prominence in the work environment, and that’s what piqued my interest and concerns around different types of tests that are used in the work environment.

There is one particular situation that sparked my concern in this space. I taught a fourth-grade class, which in the late 80s was what they called a “gates grade.” Basically, in order to get promoted to the fifth grade, students had to pass a standardized test. And I had a class of mostly kids who had failed the test the year before—this was their second year in fourth grade. One particular student—English was not his first language, but he was very smart—was top of the class in the normal tests I gave him but he didn’t do well on his GATE exam.

The school wanted to leave him behind and put him in courses for kids who needed more intensive help, which would’ve set him up for a different educational track. But I appealed that decision, working with his parents, because I felt that the issue with his testing was related to his language—not understanding some of the words or getting confused—and not with his true aptitude or knowledge. After they changed the conditions for him to pass the test, not only did he do well, but the next year he skipped a grade and in middle school, he won the state chess championship. That left a significant impression on me about how the constraints and the conditions by which children are tested can have a devastating impact on their future and can limit their opportunities. So, that’s really where this passion around how we apply testing originated with me.

What are your general thoughts on inherent biases in aptitude and behavioral assessments?

I have three primary concerns with aptitude testing in the work environment, which all deal with biases that are created due to the constraints that are associated with aptitude tests in particular.

Most aptitude tests, which presume to test for some type of inherent knowledge, whether it’s logical reasoning, problem-solving, mathematical reasoning, or whatever … they’re time-bound and often are a combination of multiple-choice questions. Questions that you have a minute or less to answer have a natural bias associated with them, because not everyone processes information within that limited period of time. So, they naturally negatively impact people who are not fast processors, and multiple-choice questions do a similar thing. They naturally exclude a group of people who are either not trained in test-taking environments like that, or who don’t process information in the same way. This is the same bias you see in standardized tests like SATs. Fortunately, a lot of colleges today are either giving kids a choice as to whether or not they want to submit test scores, or eliminating the tests completely.

The second issue that I have with these aptitude tests is the profiles that they use to determine success to begin with. Usually with aptitude tests in a work environment, you’re looking to see if someone meets a score based on a profile of someone who was successful in that role before. But if the group of people who have been successful in that role before are biased towards a particular demographic, then what you’re measuring for success has a natural bias built into it.

My third concern is that these tests are mostly used to weed people out and save time. They’re an easy way to eliminate a bunch of people. But if what you’re basing those first selection criteria on is one factor, then you haven’t really considered all the other factors that lead to someone’s success in a job, which may not be related to their aptitude, but to other things like how well they adapt to an environment, the support that they’re going to get in the workplace, and the onboarding that they’re going to achieve.

Regardless of how many aptitude tests we use during the hiring selection, it’s not decreasing the attrition that employers are experiencing within the workplace. I feel like employers are not being holistic in predicting what’s going to lead to a successful employee, because they’re not looking at all the factors that are involved in someone not only doing well in a role, but wanting to stay in that role.

Coming off of that, can you define for our readers what psychometric assessment is?

Psychometric assessment can be a variety of things, and that’s another problem with the whole term. They can test personality, aptitude, cognitive ability, and behavioral traits.

But if you’re testing personality, well, none of us have real control over our personality. There is no way for us to change our personalities. In most cases, your personality is not the primary factor that’s going to make you successful or unsuccessful. And depending on what the basis of the test is, it could be inaccurate, and research shows that there is no information that really supports the validity of these assessments. So, real personality tests are administered by psychologists in a clinical environment and are used for purposes that have nothing to do with employment. But at most companies, a recruiter is helping to administer the test and taking a standardized approach—you don’t have a qualified person reading the test.

If a psychometric assessment is more cognitively based, which is looking for different types of reasoning, it becomes more like an IQ test. And we know all of the inherent biases that are built into time-bound IQ tests—the structure of the questions is often multiple-choice or associated with what we typically think of as standardized. And you usually have a bias that gets created based on people not processing information in a way that is conducive to doing well in that type of test, or not just being good test takers.

Then there are behavior-based psychometric tests, which usually review preferences in certain behaviors. This testing has less bias because the taker can determine which behaviors to demonstrate. But again, the behaviors that you’re looking for as it relates to a successful profile are often biased because they are based on people who’ve been in the job. And if you don’t have unbiased profiles to begin with, you’re going to skew those tests.

Clearly, hiring managers should be testing people on their skills needed for the job, and not on these other factors. Why is there so much confusion over this?

I think people confuse natural ability or aptitude with skill. Aptitude is your natural ability towards something. Skill is the knowledge that you have learned and can apply. And it is in many ways easier to test for aptitude than it is to test for skill. To truly test for skill, you have to have a test that forces the person to demonstrate the skill.

So that’s easy in some types of professions. Like if you’re testing someone for their skill in accounting, you can test their knowledge by giving them scenarios or problems to solve. That’s easier to do for computer skills, certain types of programming skills, and those types of things. But for other jobs that require more creativity and softer skills, it’s a bit harder. That’s one reason that companies gravitate more towards aptitude tests—they’re easy to administer. You can administer them to a bunch of people very quickly and try to predict the potential for success versus actually giving them a skill-based test that shows whether or not they can truly use that knowledge, demonstrate that knowledge, and apply that knowledge in different circumstances.

So, companies like Degreed are working to improve things in this space. Are you optimistic about the future here? Do you see a shift coming?

I’m optimistic about the future as it relates to skills for a number of reasons. At most companies, CEOs and boards are realizing that there’s a skill shortage that is only going to increase. So, in order to recruit and retain the talent with the skills that they need to execute their strategies, they have to start focusing less on personalities and more on skills.

Plus, they actually have to start investing more in helping people develop new skills, rather than only looking for people who have those skills already. You’re seeing more and more companies actually helping people in the pre-employment or onboarding phases to develop the skills they need for the future.

Skills, to me, are the great equalizer because your ability to do something is not based on your personality, it’s not based on your gender, it’s not based on your race—it’s based on your ability to do a particular thing. So, the more that we focus on skills, the more we pull bias out of the equation, and we create better access for people to be able to get jobs done.

I also think that we’re going to see more focus on skills in the future because there’s definitely a movement toward gig work. Employees today want to have control over who they sell or trade their skills to. It’s predicted that 50% of the workforce will eventually be gig workers in the next decade. If you’re hiring for gigs, you should know that you’re hiring for specific skills to get certain work done, not for competencies that involve personality and behaviors for a long-term job. So, I think companies are going to be forced to focus more on skills just to be able to hire the workforce that they need, whether it’s permanent or gig work.

If a job candidate must undergo psychometric testing in aptitude or behavior instead of skills, what is your advice to them?

My advice would be to try to inquire what type of testing is going to be performed. Is it cognitive? Is it psychological? Is it behavioral? And to try to understand the profile of the company in terms of the culture. Because in some of these tests, the answers are not black-and-white, but more situational. In order for you to understand what may be acceptable, you need to make sure you understand the culture. I would suggest that they reach out to others who may have applied for a role at that company, or who work there, or to look at sites like Glassdoor to see if they can get some indication on the types of questions that are going to be asked. Then, they can practice.

The more you practice psychometric tests, the more ease you’ll have while taking them and the better you’ll do. Often, confidence going into a test can determine how well you do on a test. So, think about getting enough sleep the night before, eating right, and trying to be relaxed. Because the more you can control the environment that you’re taking the test in, the more likely you are to succeed.

In the end, though, you have a choice of whether or not you want to subject yourself to that sort of testing. Not every company is going to ask you to submit to a psychometric test. And I’m seeing more and more applicants say, “I know I’m not a great test taker, I don’t believe that these tests are an accurate reflection of my skills and abilities, and I’m not going to take the test.” And you always have that option—to decide that that’s not the best fit for you.

Is there anything else you think is important to talk about that we didn’t touch on?

One important thing is that companies start thinking more about people as people—and not as statistics. And, when you’re hiring a person, you have to look at people holistically. And in looking at people holistically, you have to look at not just their aptitude, but at their knowledge, their skills, their experience, their interests, and their passion.

The employer also has to think about the environment and the conditions that they are going to bring the employee into, and whether they are setting them up for success.

Also, potential and current employees need to take accountability for their own skill profiles and be cognizant of the skills that they have.

In the end, the more a job candidate understands the value of their skills, the better able they are to negotiate their price point and the type of work that they want to do going forward. These things aren’t necessarily related to aptitude or psychometric tests but are important to consider when discussing how we need to start thinking about work and skills in the hiring process and the work process going forward.

Photo: Janice Burns

Follow Welcome to the Jungle on Facebook, LinkedIn, and Instagram, and subscribe to our newsletter to get our latest articles every day!