Question
Bloomberg just published an article about a problem I’m having interviewing and selecting job candidates: Job Hunters Are Using AI to Cheat in Interviews, and Failing at the Office. I’ve suspected some applicants are misusing A.I. and Bloomberg confirms it. From the article:
“In retrospect, the interview had gone too well. The candidate’s answers were polished, if a little scripted. He seemed more than capable for the… job, so a New York City [organization] hired him. Within a month, his boss realized he couldn’t do the project he had interviewed for. He froze on basic decisions and was far less nimble than he’d seemed from behind a screen.
“The manager, who asked not to be identified discussing personnel matters — and because, she says, it’s embarrassing to admit you got duped by artificial intelligence — now feels almost certain the candidate had been covertly consulting a chatbot like ChatGPT during the virtual interviews.”
A manager is more likely to encounter this in a Zoom meeting than in person, but the reality is we do most of our interviews online now. Is there any way around it?
Nick’s Reply
This is so easy, As long as managers keep asking questions that can be gamed by applicants using A.I., they’ll keep hiring the wrong people. It’s easy because all a manager needs to do is focus the meeting on doing the work vs. doing the job interview!
Are your candidates using chat-bots to cheat in interviews?
Try this. Ask the candidate to use the “white board” in Zoom (or whatever meeting tool you’re using). Ask them to outline their understanding of your department.
What are the department’s 3 main functions or objectives, based on what you know about our operation? Give the person some leeway here. They may be able to talk only very generally. That’s okay.
Then explain, Here are a couple of the problems or challenges we are facing right now. If we hired you, how would you help tackle these? Don’t be hypothetical. Be specific. Ask the candidate to sketch out a simple diagram describing their approach. It’s good to get a picture, no matter how crude.
Then, Could you please outline on the board a brief plan of how you would do the job? In other words, what tasks would you do day 1, week 1, during the course of a month, 3 months and 6 months? What tools would you need? Who else in the company would you need to talk with in order to get the job done?
Finally: Please take a stab at this for me. Knowing what you do about our business, about the work you would be doing, and how it might affect other departments, how much do you guess you might contribute to our profitability?
For more ideas, see Much Smarter Hiring: Rules managers keep forgetting.
Ask questions A.I can’t answer
A legit candidate should be able to broadly explain and justify their guess. Don’t judge their number. Listen for thoughtful questions and their logic (however weak!). Listen for how interesting they find the question, even if they stumble. Most important, help them along!
The point is not to embarrass the candidate or to make their palms sweat. It’s to see how they would work with you. Don’t quibble with their details. Instead, ask questions about the work that chat-bots can’t answer and follow the logic of the candidate’s approach.
- Can they talk about profitability?
- Can they reasonably defend their estimate?
- What questions do they ask?
- How smart are they?
Ask questions about doing the work
Unlike dopey behavioral interviews, which ask how a person did something at their last job, these questions focus on something no one can fake: How will you do this job here?
No one can fake a discussion about the work that needs to be done. Either they can demonstrate motivation, enthusiasm and ability, or they can’t. To me, that’s what a job interview is all about. Of course, if your company is itself using A.I. tricks to game job interviews, you can’t blame candidates for responding with A.I. — that’s not interviewing!
Make it hard to cheat in interviews
While the candidate on the other side of the Zoom might be able to quickly use an A.I. to gin up answers, the more specific you make your questions about the actual work the harder it will be to cheat. The more you make this about rolling up your sleeves and talking about doing the work together, the more readily you will recognize the best candidates.
So, ask the candidate about doing the work. I think you’ll quickly pick up on signs someone is cheating. However, this approach also reveals a really good candidate’s enthusiasm and ability to think fast on their feet — even if their answers aren’t perfect.
And, those relying on A.I. will reveal they can’t work their way out of a paper bag.
The interviewer’s job is to hire someone who can do the work, not the interview.
And one last thing
Consider doing all your interviews in person. No matter the cost, it’s worth it.
How could a manager conduct interviews to avoid this problem? What questions are best to ask? For candidates, have you used A.I. tricks to game job interviews? Did it work or backfire?
: :
Cheating in interviews using AI doesn’t surprise me. I just want to bring up that people have been padding their resumes forever. Same type of cheating. I noticed this at my most recent jobs. The person hired was always a master at self-promotion and couldn’t do the job in reality plus was very attractive (man or woman). Then their workload fell to the lower staff in their department.
Many people feel justified in making themselves falsely better skilled or using whatever it takes to get a good job. The online application game has become so frustrating.
“Interviewing well” is an underrated skill.
Glad to hear that overreliance on technology is finally starting to cost employers real money. I suspect that this is only the tip of the A.I. iceberg.
They have been having problems with their job boards. It is fairly trivial to set up an AI bot to automatically apply for roles.
Exactly, I am very glad to hear it too.
Didn’t Ford just have to rehire some engineers after laying them off thinking that AI could do their jobs?
LOL!
Similar situation occurred almost 15-20 years ago when some airlines (and other companies) outsourced their customer service overseas – big disaster. It was so bad for one airline, within several months (likely one fiscal quarter) they returned to domestic employees.
@Chris S: I thought of the same thing. A “great idea” (outsourcing, automated hiring) goes south because no one bothers to play out the implications and costs/benefits and upside/downside. It all sounds great.
Before LLMs, having someone use a text or chat to “provide answers.”
One way to get around this is in-person interviews.
I know, I know…not efficient. After all, this is only about keeping the business viable and competitive. Why would we *NOT* cut corners?
@Gregory: Exactly! Put in the time and hang up the cheap shortcuts!
The one use for remote interviews is when the job requirements will turn many candidates off. I had one where I was flown out to Cape Canaveral for the interview and found out the job required 3roughly month long cruises each year (on a WWII surplus freighter off the Aleutian Islands using a radar to study Soviet satellite launches). Cheap way to screen out people who would not take the job.
“…those relying on A.I. will reveal they can’t work their way out of a paper bag.”
Imagine that, sooooo predictable.
“Consider doing all your interviews in person. No matter the cost, it’s worth it.”
This would have been my first sentence in the article.
Yep, remote has its place but as we’ve all witnessed, there are significant pitfalls.
Too much fallout in too many areas occur in many jobs with this short cut interview modality that ends up pissing off already hard working staff that is then expected to mop up the mess.
I mean, I’m an accountant by trade and I couldn’t tell you how much I contribute to profitability. That’s such a sales-guy question.
@Rich: Do you really think so? I don’t think anyone can really answer the question. But the ensuing discussion can reveal a lot. I think it’s one of the most engaging, telling questions to ask. Getting to a number is not the point. What the candidate and manager talk about is.
Heyyyy Ladyyyy !!!! (Jerry Lewis style)
Here’s a thought.
Stop keepin’ up with the AI corporate Joneses down the street and PADDLE YER OWN CANOE, ya I-wanna-fit-in HR jackass.
We THRIVED IN and built this Country on face2face interviews til, maybe, ~10 years ago … or less.
How, oh how, OH HOW, did we survive and build for ~240 years THE GREATEST COUNTRY ON THE FACE OF THE EARTH without AI, ZOOM interviewing, and shiftless HR’ers dumping their work on imperfect algorithms ?!?!
DO YOUR JOB … FACE2FACE !!!!
And isn’t it so ironic such a failure at HR is demanding so much from Candidates ?!?!
Kinda like Benedict Arnold demanding George Washington prove and justify his Patriotism.
DAMN, is hiring in this Country soooo eff’ed up.
I’m having a hard time finding sympathy for this self-created problem.
I understand remote screening, but why in the world would you not do a solid technical interview in person? At least she has the sense to be embarrassed, but not enough to come to a conclusion that involves doing her job. Maybe AI already gave her that advice and she didn’t like it.
Also, companies have been using automation for decades on the hiring side, up to and including AI. Now that they’ve fully broken the system, they complain about automation without a hint of irony?
And how many times have companies used AI to do interviews? A few years ago, there were discussions here about one-way video/computer interviews, with the candidate never interacting with a human being. And now employers are worried that candidates are using AI? I’m sorry, but I don’t feel sorry for these employers. They’re the ones who have so automated and out-sourced hiring that getting hired feels like it is one big game of who can get through the minefields, and now they’re not pleased that candidates are treating them the same way?
I can understand having a phone/Zoom first interview, especially if the candidate lives out of state. If this first interview goes well, then insist that the second interview be in person. And if the candidate is that promising (to warrant a second interview), then pay for him to come to the in-person interview.