Ask The Headhunter Home Page
Ask The Headhunter Home Page
More about Nick's book...
Go to Archive Menu The Archive


Back to the Archive Menu

This Is Not A Test:
A hiring tool for managers

By Alain Raynaud

Hiring good software engineers is a challenge. The company I joined three years ago was still in the startup stage, with a grand total of six engineers. Two years later, more than 45 engineers make up the Research & Development department.

At first, we hired a lot of people using traditional interviews, but we were less than happy with the results: some engineers who had great resumes would turn out to be poor software programmers. In our fast-paced environment, this was not acceptable and it was creating a serious problem delaying our product introductions and generally reducing our efficiency.

A New Approach to Hiring
So, I developed a new method for interviewing candidates for software positions. I saw about 35 engineers in one year, and we hired less than half of them. I believe the method worked: I know because we hired a lot of good people, many of whom still work for the company.

Most of the articles on Ask The Headhunter are devoted to teaching job candidates how to go into an interview prepared to demonstrate their ability to "do the job". In this article, I will try to show how a hiring manager can apply the New Interview concepts to determine whether a candidate can do the job. When a candidate is given the best opportunity to demonstrate what he or she can do, everyone benefits.

I believe the approach I used can be applied with success in many industries and areas of work, not just software engineering. I hope you find that it stimulates your own thinking about interviews, and about how to assess a job candidate’s ability to do the job.

The Method
I started developing a new approach to interviewing with the assumption that a candidate is often limited in an interview not only by his or her skills, but by the interviewer. I wanted to take responsibility for eliciting the best a candidate has to offer. Instead of simply relying on interviews where we chat with the candidate about his skills, I added a technical test which eventually looked very similar to the Ask The Headhunter "do the job to win the job" approach, except it is initiated by the employer.

I will refer to this approach as a "test", even though it’s not really a test. It is the best "real life" tool I know to assess a job candidate’s abilities to do the job at hand.

The candidate is given what looks like a two-page software test, but I insist that he solve the questions orally. The atmosphere I’m trying to create is that of a discussion between two engineers who are working together to solve a problem:

"Imagine you have just been hired, I'm a colleague, and I'm having some problems with the code I wrote".

This situation actually happens quite often: engineers ask another one for guidance, even though they don't formally report to one another. It also illustrates the company policy of having people cooperate as part of the same team.

It's impossible to actually have a candidate sit down and really complete a real, live software project. I’m sure this is true in most businesses. There's just not enough time, and it wouldn't be fair to the candidate. So, we focus on the sample code that is on the test, and the conversation develops from there. It allows me to see how much the candidate knows, how he reacts and how he thinks. I don't care if he finds the right solution or not. (Sometimes several solutions are possible, because this is real-life, not an academic exercise.)

What I'm Looking For
I have two main goals during my time with the candidate: to see which candidates are able to do the work, and to quickly see which ones are really interested in the work. When a candidate has opinions and ideas to express; when he can back them up with solid reasoning; and when he can work with me to explore ways to get the job done, I know I’ve got a solid contender on my hands. My challenge is to draw out the candidate.

Some candidates are not proficient in C (the programming language the test uses). That's okay, because I will let them pick any language they wish. The point is not to test for extensive knowledge about the language itself; it’s to see what they can accomplish with the information they have, using tools they are familiar with. Toward this same end, I instruct the candidates not to be afraid to ask any questions they may have, or for any information they need.

The test is mostly a way to get programmers talking about fragments of computer code. (In your business, you would be trying to get your candidate to talk about specific aspects of the job and about specific processes, problems and tasks.) After a short period where anxiety dominates, candidates open up. It becomes quite easy for an experienced engineer to evaluate a candidate’s experience and to explore how he or she would fit with the rest of the software team.

Some candidates are purely analytical and very careful; others are more daring and propose plenty of ideas. Some of them don't see even the most obvious traps (there are always some), which is a sign that they may not have much experience coding significant projects. My objective is always to make it easy for the candidate to reveal both the breadth and depth of his problem-solving abilities by giving him a natural task to tackle.

Ask a programmer for his strengths and weaknesses, and you usually won't get any answer. This is why conventional interviews are often fruitless: a lot of programmers are introverted or are not very comfortable with communicating. The test is good way of having a conversation in which they are free to actually share opinions which may be of interest to the company. In my experience, the most introverted engineer can talk for hours on topics such as "Unix vs. DOS" or "C vs. C++". And, I learn a lot about the candidate's capabilities while doing so.

[Due to the stress created by the interview process, a great many candidates — regardless of their line of work — clam up and find it difficult to communicate their best ideas or to demonstrate their most important abilities. That’s why this approach, which helps candidates focus on what they’re best at — their work — can be a powerful tool for almost any hiring manager. It isn’t just for "introverted programmers". — N.C.]

Some Rules
Even though the test is not tailored for it, one can tell quickly whether candidates come from a PC or a Unix background by the solutions they propose. (This distinction is critical to our work.) Therefore it's important that the engineer giving the test have as broad a base of knowledge as possible. The test would be unproductive if the interviewer were to force his prejudices on the candidate.

[In your business, you probably want to know what a candidate’s biases are with regard to how he approaches his work, whether it’s marketing, sales, customer support, accounting or whatever. The point is to encourage the candidate to reveal his fundamental work habits, not to impose your own expectations. Remember: you’re trying to figure out what kind of baggage the candidate is bringing along. — N.C.]

For the same reasons, the person conducting the test should be very proficient in what he is testing. If a junior engineer were to do the testing, he might actually reject a candidate because the engineer knows less than the candidate!

The manager (or staff member) doing the interviewing should have a broad knowledge of all the jobs carried out in his group, so that he can recommend the most appropriate position for the candidate. Different jobs have different requirements and a candidate may be a perfect fit for one position but a very poor fit for another. This is where the interview becomes a creative kind of "resource allocation", and it’s why only a skilled company employee with broad knowledge of the business should do the interviewing. Otherwise, you might waste a perfectly good candidate.

Because each candidate is a complex mixture of knowledge and weaknesses, the testing should not be used to come up with some kind of "objective score". That's why it's best to let the same interviewer perform all the tests, so that he can give an opinion based on comparisons for a given position. For example, "John was way better than Peter for firmware coding, but Bob looked like he would do great in algorithms."

This approach is a lot of work. It can burn a manager out. In my case, after one year of conducting these interviews (more than one every two weeks), I was feeling very tired of it. We finally assigned someone else to do them.

When transitioning, one has to be very careful to pass this duty on to a senior staff member who has enough experience. The challenge is to convince experienced staff members that it's an important task. When I transmitted the knowledge to the next engineer, that's when I noticed how much I had learned from this method or interviewing. While it's designed to last 30 minutes, I could talk for more than two hours about all the ways candidates would answer, and what to think of it.

The mistake that’s easy to make with this interview method is to administer "the test" as if it were a test. It's not! It's a way to stimulate a directed discussion about the work itself. In the hands of Human Resources or junior staff members, there would be a temptation to turn it into a quick, rote quiz that would result in nothing but quick, meaningless results. I believe that to apply this method effectively, it must be done by a seasoned manager (or staff member) who really understands the work and the business of the department that’s doing the hiring. That’s why I believe it’s critical, when a person burns out from doing this, that the next person who takes on these candidate evaluation duties is really prepared.

This interview method has really helped me sort between skilled programmers and ones with limited abilities. While I'm not saying it's a perfect solution, I honestly think this method has been quite successful at determining who was going to be really good at software engineering and who had just added some keywords to his resume to make it look good. In other words, there’s no faking it!

In retrospect, it looks very much to me as if I’ve been applying Nick's Ask The Headhunter approach to get the candidate to "do the job" in the interview. Of course, you can't expect a candidate to do a full design if you’re in the software business (it would take weeks), or to complete an entire business project in your own company. But, this "test" is the best way I know to evaluate a person's ability to actually do the job. I hope you find these ideas helpful in your own hiring.

Alain Raynaud is a senior engineer with one of the leading electronic CAD companies. He is based in Paris, France.


The contents of this site are Copyright (c) 1995-2015 North Bridge Group LLC.
All rights reserved. This material is for personal use only. Republication and redissemination, including posting to news groups, is expressly prohibited without prior written consent. Ask The Headhunter, Fearless Job Hunting, the ATH logo and other ATH titles are trademarks or registered trademarks of North Bridge Group LLC and Nick A. Corcodilos.

User agreement, legal information and disclaimer.

Visit the Ask The Headhunter Blog and sign up for your free subscription to the weekly Ask The Headhunter Newsletter.

We welcome comments and
suggestions. Please email to
Ask The Headhunter.