Back to the Archive Menu
This Is Not A Test:
A hiring tool for managers
By Alain Raynaud
Hiring good software engineers is a challenge. The
company I joined three years ago was still in the startup stage, with a grand total of six
engineers. Two years later, more than 45 engineers make up the Research & Development
At first, we hired a lot of people using traditional
interviews, but we were less than happy with the results: some engineers who had great
resumes would turn out to be poor software programmers. In our fast-paced environment,
this was not acceptable and it was creating a serious problem delaying our product
introductions and generally reducing our efficiency.
A New Approach to Hiring
So, I developed a new method for interviewing candidates for
software positions. I saw about 35 engineers in one year, and we hired less than half of
them. I believe the method worked: I know because we hired a lot of good people, many of
whom still work for the company.
Most of the articles on Ask The Headhunter are devoted to
teaching job candidates how to go into an interview prepared to demonstrate their ability
to "do the job". In this article, I will try to show how a hiring manager can
apply the New Interview concepts to determine whether a
candidate can do the job. When a candidate is given the best opportunity to demonstrate
what he or she can do, everyone benefits.
I believe the approach I used can be applied with success
in many industries and areas of work, not just software engineering. I hope you find that
it stimulates your own thinking about interviews, and about how to assess a job
candidates ability to do the job.
I started developing a new approach to interviewing with the
assumption that a candidate is often limited in an interview not only by his or her
skills, but by the interviewer. I wanted to take responsibility for eliciting the best a candidate has to offer. Instead of simply relying
on interviews where we chat with the candidate about his skills, I added a technical test
which eventually looked very similar to the Ask The Headhunter "do the job to win the job" approach, except it is
initiated by the employer.
I will refer to this approach as a "test", even
though its not really a test. It is the best "real life" tool I know to
assess a job candidates abilities to do the job at hand.
The candidate is given what looks like a two-page
software test, but I insist that he solve the questions orally. The atmosphere Im
trying to create is that of a discussion between two engineers who are working together to
solve a problem:
"Imagine you have just been hired, I'm a
colleague, and I'm having some problems with the code I wrote".
This situation actually happens quite often: engineers
ask another one for guidance, even though they don't formally report to one another. It
also illustrates the company policy of having people cooperate as part of the same team.
It's impossible to actually have a candidate sit down and
really complete a real, live software project. Im sure this is true in most
businesses. There's just not enough time, and it wouldn't be fair to the candidate. So, we
focus on the sample code that is on the test, and the conversation develops from there. It
allows me to see how much the candidate knows, how he reacts and how he thinks. I don't
care if he finds the right solution or not. (Sometimes several solutions are possible,
because this is real-life, not an academic exercise.)
What I'm Looking For
I have two main goals during my time with the candidate: to
see which candidates are able to do the work, and to quickly see which ones are really
interested in the work. When a candidate has opinions and ideas to express; when he can
back them up with solid reasoning; and when he can work with me to explore ways
to get the job done, I know Ive got a solid contender on my hands. My challenge is
to draw out the candidate.
Some candidates are not proficient in C (the programming
language the test uses). That's okay, because I will let them pick any language they wish.
The point is not to test for extensive knowledge about the language itself; its to
see what they can accomplish with the information they have, using tools they are familiar
with. Toward this same end, I instruct the candidates not to be afraid to ask any
questions they may have, or for any information they need.
The test is mostly a way to get programmers talking about
fragments of computer code. (In your business, you would be trying to get your candidate
to talk about specific aspects of the job and about specific processes, problems and
tasks.) After a short period where anxiety dominates, candidates open up. It becomes quite
easy for an experienced engineer to evaluate a candidates experience and to explore
how he or she would fit with the rest of the software team.
Some candidates are purely analytical and very careful;
others are more daring and propose plenty of ideas. Some of them don't see even the most
obvious traps (there are always some), which is a sign that they may not have much
experience coding significant projects. My objective is always to make it easy for the
candidate to reveal both the breadth and depth of his problem-solving abilities by giving
him a natural task to tackle.
Ask a programmer for his strengths and weaknesses, and
you usually won't get any answer. This is why conventional interviews are often fruitless:
a lot of programmers are introverted or are not very comfortable with communicating. The
test is good way of having a conversation in which they are free to actually share
opinions which may be of interest to the company. In my experience, the most introverted
engineer can talk for hours on topics such as "Unix vs. DOS" or "C vs.
C++". And, I learn a lot about the candidate's capabilities while doing so.
[Due to the stress created by the interview process,
a great many candidates regardless of their line of work clam up and find it
difficult to communicate their best ideas or to demonstrate their most important
abilities. Thats why this approach, which helps candidates focus on what
theyre best at their work can be a powerful tool for almost any hiring
manager. It isnt just for "introverted programmers". N.C.]
Even though the test is not tailored for it, one can tell
quickly whether candidates come from a PC or a Unix background by the solutions they
propose. (This distinction is critical to our work.) Therefore it's important that the
engineer giving the test have as broad a base of knowledge as possible. The test would be
unproductive if the interviewer were to force his prejudices on the candidate.
[In your business, you probably want to know what a
candidates biases are with regard to how he approaches his work, whether its
marketing, sales, customer support, accounting or whatever. The point is to encourage the
candidate to reveal his fundamental work habits, not to impose your own expectations.
Remember: youre trying to figure out what kind of baggage the candidate is bringing
For the same reasons, the person conducting the test
should be very proficient in what he is testing. If a junior engineer were to do the
testing, he might actually reject a candidate because the engineer knows less than the
The manager (or staff member) doing the interviewing
should have a broad knowledge of all the jobs carried out in his group, so that he can
recommend the most appropriate position for the candidate. Different jobs have different
requirements and a candidate may be a perfect fit for one position but a very poor fit for
another. This is where the interview becomes a creative kind of "resource
allocation", and its why only a skilled company employee with broad knowledge
of the business should do the interviewing. Otherwise, you might waste a perfectly good
Because each candidate is a complex mixture of knowledge
and weaknesses, the testing should not be used to come up with some kind of
"objective score". That's why it's best to let the same interviewer perform all
the tests, so that he can give an opinion based on comparisons for a given position. For
example, "John was way better than Peter for firmware coding, but Bob looked like he
would do great in algorithms."
This approach is a lot of work. It can burn a manager out.
In my case, after one year of conducting these interviews (more than one every two weeks),
I was feeling very tired of it. We finally assigned someone else to do them.
When transitioning, one has to be very careful to pass
this duty on to a senior staff member who has enough experience. The challenge is to
convince experienced staff members that it's an important task. When I transmitted the
knowledge to the next engineer, that's when I noticed how much I had learned from this
method or interviewing. While it's designed to last 30 minutes, I could talk for more than
two hours about all the ways candidates would answer, and what to think of it.
The mistake thats easy to make with this interview
method is to administer "the test" as if it were a test. It's not! It's a way to
stimulate a directed discussion about the work itself. In the hands of Human
Resources or junior staff members, there would be a temptation to turn it into a quick,
rote quiz that would result in nothing but quick, meaningless results. I believe that to
apply this method effectively, it must be done by a seasoned manager (or staff member) who
really understands the work and the business of the department thats doing the
hiring. Thats why I believe its critical, when a person burns out from doing
this, that the next person who takes on these candidate evaluation duties is really
This interview method has really helped me sort between
skilled programmers and ones with limited abilities. While I'm not saying it's a perfect
solution, I honestly think this method has been quite successful at determining who was
going to be really good at software engineering and who had just added some keywords to
his resume to make it look good. In other words, theres no faking it!
In retrospect, it looks very much to me as if Ive
been applying Nick's Ask The Headhunter approach to get the
candidate to "do the job" in the interview. Of course, you can't expect a
candidate to do a full design if youre in the software business (it would take
weeks), or to complete an entire business project in your own company. But, this
"test" is the best way I know to evaluate a person's ability to actually do
the job. I hope you find these ideas helpful in your own hiring.
Alain Raynaud is a senior engineer with one of the
leading electronic CAD companies. He is based in Paris, France.
The contents of this site are Copyright (c)
1995-2015 North Bridge Group LLC.
All rights reserved.
This material is for personal use only. Republication and redissemination,
including posting to news groups, is expressly prohibited without prior written consent.
Ask The Headhunter, Fearless Job Hunting, the ATH logo and other ATH titles are trademarks or registered trademarks of North Bridge
Group LLC and
Nick A. Corcodilos.
agreement, legal information and disclaimer.