In the January 21, 2014 Ask The Headhunter Newsletter, Nick asks readers for help with an upcoming TV news interview:

There’s no question from a reader this week. Instead, I’m asking all of you readers a question. May I have your help?

I’ve been asked to appear on a TV news show to discuss how HR is using Big Data to watch you at work — and to process your job application without interviewing you. I’d like your input on the topic so I can frame my comments with your interests in mind. I’ll share a link to the program after it airs, and we can discuss it further then.

[UPDATE: Here’s the link that includes video from the TV program: Big HR Data: Why Internet Explorer users aren’t worth hiring]

Nick’s Question for You

Big-Data-KittyAre you frustrated because employers reject your job application out of hand without even talking to you? Tired of online application forms kicking you out of consideration because you took too long to answer questions, or because you failed to disclose your salary history?

Wait — America’s employment system is getting even more automated and algorithm-ized. According to a new report in The Atlantic, the vice president of recruiting at Xerox Services warns that:

“We’re getting to the point where some of our hiring managers don’t even want to interview anymore.” According to the article, “they just want to hire the people with the highest scores.”

The subtitle of that Atlantic column (They’re Watching You At Work by Don Peck) reads: “The emerging practice of ‘people analytics’ is already transforming how employers hire, fire, and promote.”

Does that worry you?

If all goes according to plan (hey, this is TV — all schedules are subject to change), Atlantic columnist Don Peck and I will talk about the rise of Big Data in the service of HR — and I want your input in advance, because I’m worried about the conclusions Peck draws in his article. It’s a very long one (8,600+ words), but it illuminates some of the technology that’s frustrating your job search. Please have a look at it, and post your suggestions to help me frame my comments for this TV program.

Here are the Big Problems I see with this Big Data approach to assessing people for jobs and on the job:

The metrics are indirect.

The vendors behind these “tools” don’t directly assess whether a person can do a job. Instead, they look at other things — indirect assessments of a person’s fit to a job. For example, they have you play a game and they measure your response times. From this, they try to predict success on the job. That determines whether you get interviewed.

The problem is that we’ve known for decades that this approach doesn’t work. Wharton researcher Peter Cappelli throws cold water on indirect assessments:

“Nothing in the science of prediction and selection beats observing actual performance in an equivalent role.”

All that’s being thrown into the mix by these “assessment” vendors is Big Data. But more data doesn’t change anything. In fact, it makes things worse if the data are not valid predictors of success. It’s worse because indirect assessment leads to false negatives (employers reject potentially good candidates) and to false positives (they hire the wrong people for the wrong reasons).

The conclusions are based on correlations.

These tools predict success based on whether certain characteristics of a person are similar to characteristics of a target sample of people. For example, Peck’s article says that “one solid predictor of strong coding [programming] is an affinity for a particular Japanese manga site.” (Manga are Japanese comics.)

Gild, the company behind this claim, says it’s just one correlation of many. But Gild admits there’s “no causal relationship” between all the Big Data it gathers about you and how you perform on the job.

In what can only be called a scientific non sequitur, Gild’s “chief scientist” says “the correlation, even if inexplicable, is quite clear.”

The problem: A basic tenet of empirical research is that a correlation does not imply causality, or even an explanation of anything. Data tell us that people die in hospitals, and that correlates highly with the presence of doctors in hospitals. All jokes aside, that correlation doesn’t mean doctors kill people. Except, perhaps, in the world of Big HR Data: If you’re selling “people analytics,” then playing a game a certain way means you’ll work a certain way.

When we pile specious correlations on top of indirect assessments (What animal would you be if you could be any animal?), we wind up with no good reasons to make hiring decisions, and with no basis for judgments of employees.


INTERMISSION: There’s a hidden lesson for recruiters in Big Data.

Hanging out at a manga site doesn’t improve anyone’s ability to write good code — nor does it predict their success at work. But, it might mean that a recruiter can find some good coders on that manga site — the one reasonable conclusion and recruiting tactic that none of the people Peck interviewed seem to have thought of!


I don’t think Peck wrote this article to promote “people analytics” as the solution to the challenges that American companies face when hiring, but he does seem to think the Kool-Aid tastes pretty good. I think Peck over-reaches when he confuses useful data that employers collect about employee behavior to improve that behavior, with predictions based on silly Big Data assumptions.

To entice you to read the article and post your comments, I’ll share a couple of highlights in the article that kinda blinded me. Well, the assumptions behind them were blinding, anyway:

Spying tells us a lot.

In further support of indirect assessments of employees and job applicants, Peck cites the work of MIT researcher Sandy Pentland, who’s been putting electronic badges on employees to gather data about their daily interactions. In other words, Pentland follows them around electronically to see what they do.

“The badges capture all sorts of information about formal and informal conversations: their length; the tone of voice and gestures of the people involved; how much those people talk, listen, and interrupt; the degree to which they demonstrate empathy and extroversion; and more. Each badge generates about 100 data points a minute.”

Peck notes that these badges are not in routine use at any company.

It’s just a game.

A lot of the “breakthroughs” Peck writes about come from start-up test vendors like an outfit called Knack, which creates games “to suss out human potential.” Knack continues to seek venture funding, and the only Knack client mentioned in the article is Palo Alto High School, which is using Knack games to help students think about careers.

“Play one of [Knack’s games] for just 20 minutes, says Guy Halfteck, Knack’s founder, and you’ll generate several megabytes of data, exponentially more than what’s collected by the SAT or a personality test.”

The big dbig-dataata gathered, writes Peck,

“are used to analyze your creativity, your persistence, your capacity to learn quickly from mistakes, your ability to prioritize, and even your social intelligence and personality. The end result, Halfteck says, is a high-resolution portrait of your psyche and intellect, and an assessment of your potential as a leader or an innovator.”

Let’s draw a comparison in the world of medicine; it’s an easy and apt one: If more megabytes of game data can be used to generate more correlations, could doctors diagnose patients more effectively by collecting bigger urine samples? Because that’s the logic.

No sale.

I don’t buy it. I want to know, can you do the job?

Some Big Data about employee behavior can be analyzed to good effect. For example, Peck reports that Microsoft employees with mentors are less likely to leave their jobs, so Microsoft gets mentors for them. But he seems to easily confuse legitimate metrics with goofy games of correlation. And the start-up companies he profiles don’t seem to be on any leading edge — they’re mostly trying to sell the idea that Big Data in the service of questionable correlations makes those correlations worth money.

(To learn the ins and outs of legitimate employment testing, see Erica Klein’s excellent book, Employment Tests: Get The Edge.)

Big Deal.

We know that what Peter Cappelli says about the science of prediction is correct. But I think Arnold Glass, a leading researcher in cognitive psychology at Rutgers University, says it best:

“It has been known since Alfred Binet and Victor Henri constructed the original IQ test in 1905 that the best predictor of job (or academic) performance is a test composed of the tasks that will be performed on the job. Therefore, the idea that collecting tons of extraneous facts about a person (Big Data!) and including them in some monster regression equation will improve its predictive value is laughable.”

It seems to me that HR should be putting its money into teaching HR workers and hiring managers to hang out where the people they want to hire hang out, and into teaching them how to get to know these people — and how good they are at their work.

In the meantime, is it any surprise to any job seeker today that employers mostly suck at recruiting the right people and at conducting effective interviews?

If you have questions or thoughts you’d like me to raise in this forthcoming TV program, please post them. I’ll try to use the best of the bunch. I wish I could tell you that hanging out on my blog causes employers to hire you. Thanks!

[UPDATE: Here’s the link that includes video from the TV program: Big HR Data: Why Internet Explorer users aren’t worth hiring]

: :

72 Comments
  1. Nick,

    One thing that jumped at me when I read the Atlantic piece is that the reference to Moneyball is somewhat misplaced.

    While the sabermetrics, or crazy math that was instituted under Billy Beane, measures actual baseball performance (e.g. BABIP is batting average on balls in play) and extends orthodox metrics such as BA, RBI, etc, traditional ways to scout use indirect means like an athlete’s physical ‘makeup’.

    For example, while being a 6’4″ chiseled, square jawed All-American athlete with raw physical strength plays a large part in a traditional scout’s assessment, sabermetric scouts just care about extended metrics of actual baseball performance hitting and catching and striking out, etc. An example of an unsuccessful player of the former mold is Billy Beane himself (a failed, but highly touted prospect), an example of an untraditional sabermetric player is Kevin Youkilis (the Greek God of Walks), and an example of a player whose performance was well predicted by both traditional and sabermetric methods is Nick Swisher.

  2. P.S. re-reading my comment…perhaps ‘misplaced’ was not the correct word… I just find it odd that advanced metrics of actual ‘on the job’/’on the field’ performance is lumped together with predictive but indirect analtyics (personality tests, etc) in that article.

    –just contrast the indirect Big Data of which you speak and with which you take issue in this article with measurement of on the job performance.

  3. Nick,

    I don’t understand several things about this topic. How is the data gathered? How is it processed?

    From what I’ve read so far, I’m reminded about the story, “Here I am, one foot in boiling water and one foot in ICY water… On the average, I feel fine.”

    Thanks,

    George…

  4. After a 12 month search, I recently started a new job for a company very well known for its technology and Big Data capabilities among other things. I can tell you that I have been networking to get a job with them for the past 9 or 10 months. I asked for a lot of introductions, informational meetings, coffees, and finally got to interviews, but I never filled out the long online application. I networked up to a VP and he told me it wasn’t necessary to apply online at that point.
    HR tries to eliminate candidates (I learned that from you). People like personal connections. They trust them.
    You probably don’t want to work for a company that is looking for a widget employee. Keep looking.

  5. I am astonished and appalled that “Big Data” plays any part in hiring/firing/promotion decisions. Why does a company even need an HR dept and/or hiring managers if this is what they use. Really, what are these people doing and why do they get paid so much to do it? Obviously a software program can take care of the whole process including shooting off the “decision” emails.

  6. Hi Nick.

    Thanks for posing your question, and for providing the link to the thought-provoking article in the Atlantic about the role of “Big Data” in the hiring process.

    The idea that Big Data can assist large organizations in identifying appropriate talent has SOME (limited) merit, but I would argue that tools such as these be employed VERY cautiously, and that the ultimate decision to hire someone should be based on the overall capability of the whole individual. It is frankly unfathomable to me how sophisticated algorithms that collect data points as we wander around – both in the real world and on the net – could ever accurately judge qualities like integrity, compassion, and on-the-fly creative deal-making that are but a small sample of what any specific job may require.

    In your past posts, you have made a great case for the “Serendipity” of the search process. As we go along and try new things and approach problems from a new angle, we are often surprised to come across a solution that is better than what we could have surmised at the start. Big Data seems to be geared towards identifying some basic measurable unit of conformity from what we ALREADY know. The results of data collection can only tell us what we have asked to find.

    And, to stir the pot, after seeing first hand how ineffective many HR folks are now, I shudder to think how much more “empowered” they will be to wreak havoc on an organization when they have their hands on these “proven, scientific” tools.

    Yikes!

  7. Hi, Nick –

    Excellent article. I think the biggest flaw in the “exclusively digital” approach that is described is that it assumes infallibility on two fronts: the algorithm or test is assumed to be correct – think about this the next time your laptop freezes or crashes if you want some idea of how laughable that concept is – AND it assumes that people with no backgrounds in psychographics will be able to read and determine the results accurately. Think of the boneheaded decisions and comments you hear every day at the office and then try imagining these same people making significant hiring decisions based on test data they know nothing about.

    To cite just one example of how crazy this idea is, I’ll give you a personal example. Many tests contain bias detection questions, i.e., questions designed to determine whether the test-taker is being honest or trying to beat the test. In my wife’s case, one such question she encountered on a recent job application test was, “True or False: Stereotypes are sometimes true.” As she put it, “That was a no-win question. If I answered false, they’d conclude I was dishonest and trying to beat he test; but if I said true, someone in HR could just as easily call me a racist.” Her claim is flawed only in that it assumes someone would bother looking at particular responses rather than just a raw score; but I think the point is a valid one if one considers that HR’s job is to eliminate as many applicants as possible, and there would probably be no quicker way for an HR admin assistant tasked with the job than to pick a few questions of personal importance to use as determinants.

    Sadly, I think this reflects a trend that will in all likelihood continue. And much like the endless advertisements that are posted not for hiring purposes but simply to collect resumes to populate databases, this will only make things harder for applicants. (But hey, that’s okay – as Cappelli himself points out, employers will just use this as another excuse to blame the educational system for their own failures to do what is necessary to hire properly.)

    Keep up the fight!

  8. Hi Nick,

    The back of the American worker seems to be broken, and far too many will do anything to anybody to get or keep a job now.

    There is just one solution: we must start saying no. No to the pointless interview, no to sticking it to the other guy, and no to outsourcing other peoples’ jobs to keep our own.

    After losing our careers over the last decade, my husband and I have worked for various companies that still have a shred of integrity left. Our jobs have been humorous, frustrating, boring and pay much less than our old careers; we have had erratic schedules, Neanderthal management and about zip for benefits, but we have not “stuck it” to people all day to earn an income. We’ve had unemployed periods (with no unemployment check), been on and off COBRA, and have readjusted our lifestyle to the survival level and have saved everything we can, still helping others we know along the way.

    Time and again, those who will do anything for that dollar will be summarily thrown under the bus, eventually.

    It’s time to think about coming generations, and about the system of work as a whole. Nothing will change as long as people accept the low ball standards and behave that way.

    Keep telling it like it is, Nick.

    And as for big data: this too, shall pass; we have already seen how well everything runs when people with solid experience are routinely dumped out of companies: they are a mess. Big data is another way to hack jobs and crown someone King for doing it even more efficiently. Job boards and automated systems allow employers to keep cutting jobs and lowering pay, when they see how desperate others are for work.

    Keep the faith, folks. Despite what you may be told at work, we really ARE all in this together!

  9. The only pre-employment assessments that I know of that have been proven of any value are:
    1. Personality fit tests (ethics)
    2. Tests designed to gauge actual skillsets (e.g. computer networking), but then employers can use other indicators in place of those (e.g. certifications).

    All of this other over-automating of employee assessment is borderline useless.

  10. Nick,

    NPR’s Planet Money recently did a podcast on this very issue, looking at which applicant-screening questions predict success in call-center jobs. The findings are surprising:
    http://www.npr.org/blogs/money/2014/01/15/262789258/episode-509-will-a-computer-decide-whether-you-get-your-next-job

  11. Data is hiring isn’t working for two reasons. The inability of past to predict the future is certainly a big part of it. The biggest issue to me is the “data” they are collecting. Key word searches are a joke as they can be beaten by the candidate who cares more about breaking the system than being the best candidate. I see more and more using screening questions on the application which I think is good but they need to be the right questions and stop the yes/no answers. I recently had one where it asked a question that was really three parts and the way it was asked I had to say no, but the reality is I had the experience they needed. Within five minutes the computer sent me a letter saying after in depth analysis they would not be speaking to me……analyze bad data and get bad answers

  12. I’m a scientist and engineer by trade, and two ideas struck me immediately while I was reading the recent Atlantic’s cover story:

    1. The metrics are all indirect, so they are utterly unlike sabermetrics (as mentioned by Some Guy).

    2. Even the name ascribed to the field – Big Data – suggests extreme statistical complexity.

    Both of these ideas can combine to make an extremely profitable business model preying on business managers’ frequent infatuation with (and in general their utter lack of understanding of) science and technology, especially when it can be applied to relieve them of the responsibility of decision-making.

    I expect that Big Data will follow a similar path as did Scientific Management at the turn of the century. While Taylor laid the foundation that – after considerable evolution – ultimately led to the far more useful systems of Six Sigma, lean manufacturing, and TPS, there were no shortage of hucksters (especially in the earliest days) seeking to exploit managers’ ignorance and desire for easy results by selling expensive and ineffectual services.

    In its current incarnation, Big Data will probably not generate much real world success for anyone other that the firms selling Big Data services. Unfortunately, by the time real utility is discovered in this field, I will probably have aged out of the workforce.

    Just as parts have been made interchangeable for many decades, large wealthy companies are spending incalculable sums of money to learn how to make people interchangeable as well. To a typical manager, interchangeable is also disposable.

    I’m done with it.

    For my part, I’ve been in my field for nearly twenty years, and have grown more and more dissatisfied. A year ago, I decided to stop fretting about my career, just do my job, and focus my creativity and my best efforts on starting a sort of retirement business – running a small brewery.

    A few extra dollars is not worth my soul. Or yours.

  13. Let’s take a look at some fields where individual employee performance is clearly correlated with organizational success.

    Sports: Scouts,game films and seasonal stats are cornerstones. Height, weight, performance in key measures are also considered. All these relate DIRECTLY to the job the candidate will perform. Character is also considered, this is also important for longevity and team success.

    Entertainment: Earlier performances, demo tape, portfolio, audition. Again, these all show how the candidate will perform in the new position.

    Can anybody point to a movie or play which was cast through Monster.com? You would think that with the huge sums at stake these producers would be eager to adopt the most powerful and up-to-date tools available.

  14. Nick,

    Perhaps this would work for some basic jobs, but we are supposedly in the era of the “creative class” and creativity and curiosity are highly desired traits. Creativity researchers universally conclude that creativity cannot be measured through standardized testing. This was tried, in earnest, for twenty years post WWII as the US feverishly sought the best minds for defense research during the Cold War. However, there has been found no test that can accurately gauge creative potential and curiosity/interest. Simple human observation is far more reliable. I don’t see how automated hiring processes could help companies acquire the creative employees they say they need.

  15. I don’t know, but I’m suspicious that some types of data gathering (like electronic monitoring) would be thrown off by different temperament types. As well, I think that big data would have a difficult time assessing a person’s virtue, because character is something that can only be assessed over time, and in how someone responds to actual situations (beyond “doing the work” in the interview). This leads into the biggest problem I see with big data — when given primary importance, it reduces people to mere test scores, cogs in the machine. Even if “effective”, I fear it has profound negative effects on culture and society. Big data can be a useful tool, but its users need to know how to use it responsibly, towards the betterment of each human person.

  16. The whole thing seems odd. Why don’t you want to meet someone face to face ? You will be dealing with them face to face daily if you hire them ! I am a new to hiring manager young in my career and have only filled 3 positions to date. I can tell you in all 3 the person i liked best “on paper” was not the person i ultimately hired after interviewing. I know that’s not a scientific data set but for me, its all the proof i need that data gives you an idea of the candidate but not the whole picture. And lets remember, data can be manipulated quite easily by the person putting it together or the person interpretting it. How are we sure that the integrity of the HR Big Data is intact ?

    My sister is trying to break into a new field and is going up against the Big Data machine. She has identified a company she wants to work for but its not necessarily one of those companies that is very forthcoming about doing informational interviews or sharing who the hiring manager is for jobs, etc. I continue to send her your advice and she slowly plugs along and tries to network. I really do feel for her though and all those others caught up in the HR data game. Good Luck !

  17. • I do not believe that the data collection of this type is necessarily effective.
    • I do not want to work for such an ignorant company, or with the employees selected by its system.
    • I work within industrial sectors that require the results that I produce.

  18. This reminds me of a lesson I learned many years ago: it’s a lot easier to be a manager than a leader. A manager likes the data approach because it appears to be cut and dried. It makes sense – on the surface at least – and using it takes none of the special attributes leaders must have.

    Leaders know that good people usually won’t be found through analyzing data. Successful businesses don’t run on data. People can’t be defined by data. Sure, there has to be competence in the workplace, but success also requires clear communication, integrity, mutual respect, trust, and esprit de corps – all those things for which there is no box on a spreadsheet. Things that leaders have to judge, demonstrate, and nurture, not depend on outsiders to define or mine for them.

    That’s my problem with the data approach: I doubt it can consistently identify what’s really important and I know it can’t indicate whether a specific person is the right fit for a specific position or team. It will never replace a good hiring manager with a vision for the position and the skills and patience to fill it with the ideal candidate.

  19. I agree will all your points, which are (of course) consistent with what you have been saying for years. I don’t, however, hold out much hope for you to successfully “speak truth to power”.

  20. Ah, Big Data. Just another buzz word for something we already have done/tried.

    I would argue that we already have “Big Data” already available in this space, and it has been an utter failure. One has to look no farther than the job boards and LinkedIn where supposedly we have direct data points on millions of people. Yet they only account for a small percentage of actual hires. Oh and companies already have applicant tracking systems, so they have already collected a ton of info of possible job leads themselves. Again, this data collection has been a epic failure.

    I could also see these correlations targets for discrimination. For example, Computer Science enrollments at Colleges are heavily male. So, is the algorithm going to target males for software jobs? I am not sexist, but if you want a complete analysis, this is a correlation.

    This is just another play for some software companies to sell more snake oil. The best way to find competent works will always be to go out and meet them and actually talk to them. Only then, can you make a determination of their qualifications. A resume or indirect correlations is only a crutch and will take us so far. As an aside, I have read places where the average recruiter spends 6 seconds reading your resume and makes a determination.

  21. Having worked professionally in the field of data analytics for more than two decades and spent the early part of my career in HR information systems, I’m resisting the urge to write an essay on this topic. Instead, here are a few essential points:

    First, the term “big data” deserves its own inside back cover in Consumer Reports as an example of “selling it” with over-the-top aspirational hype.

    Second, the historical technology phrase, “garbage in, garbage out” still applies.

    Third, data always needs a matching context for correlation to be useful. The more inference and bias in the matches, the greater the potential for syllogistic fallacies.

    Fourth, data quality problems are a fact of life in almost every system, whether public or corporate. Many data brokers disclaim liability for data accuracy and will resist disclosing an audit trail for how they obtained the data they sell. Poor data quality evidence is everywhere: the billions of $$ in revenues of CRM data quality vendors; the $18.6M damage award against Equifax for failing to correct data errors; the databases of the so-called “employment verification” services. (The data broker business is one of “passing the buck.”) Detecting data errors is expensive, correcting them even more expensive, so in the absence of penalty or regulation, data errors persist.

    Fifth, in most organizations, HR is a cost-center. They are measured and rewarded for doing their work at the lowest cost, even if they shift cost to other parts of the organization.

    In summary, using algorithms for data correlation to screen, select, and hire employees is naive at best. In the spectrum of worse outcomes, the naiveté could expose an organization to discrimination risk and millions of $$ wasted on yet another IT debacle.

  22. Nick, as much as I am wary of Orwellian trends in hiring, the following quote from the Don Peck article did jump out at me: ” . . . It’s good news for people who’ve fallen off the career ladder through no fault of their own (older workers laid off in a recession, for instance) and who’ve acquired a sort of professional stink that is likely undeserved.” I hope so, as the deck does seem stacked against workers age 50+. The closing quote of the article was telling as well: “There’s just some things you need to find out in person.”

  23. Whenever I hear about the “Quants” taking over the world as we know it I always go back to the Wall Street example of a couple of very bright Nobel prize winning economist who hired all their smartest Quant friends and started Long Term Capital Mgmt. They applied all kinds of sophisticated advanced math to corner the market and did very well until circumstances changed that thier smarty pants analytics hadn’t accounted for and they damn near took down the global economy! Advanced math has its place but will never run the world nor should they be hiring & firing!

  24. I am humbled to read some of these readership responses thus far. And, good for you, Nick, and I know you’ll make the very best of the TV opportunity. Here’s my take on this question:
    – I used to work for the US Census Bureau. Statistics can always be manipulated to lie.
    – I recently interviewed with a ‘Leader’ (better word for her: ‘Manager’ – reference ‘Russell’s comment above) who began our session with “I hate HR”. Her attitude typified the frustrated mindset of wanting to ‘fix’ her HR issues by way of cut and dried methods (and at present, her company is esperiencing 120% turnover!)
    -Recruitment/hiring is an art, NOT a science. While quantifiable tools have their uses, those things are still mere tools.
    – Nothing – NOTHING – substitutes the human element of decision-making during the hire process.
    – I am concerned that some of the tests in the article may not pass as altogether legal tests, if they were ever considered by an administrative law judge. Something to think about, Nick.

  25. Quantitative tests will never predict how an executive will succeed or fail in dealing with discontinuous events. Success in global markets requires the ability to handle multiple abstractions, business cases, and in different markets, cultures, and time zones. The only reliable predictor is prior experience.

  26. I have worked for Shell, and taken multiple day-long assessment tests, and been placed in the “talent pipeline.” I was given a posting in a unit with no approved budget, constantly rotating managers, and asked to do things that were not my prime expertise. In an effort to escape that I found a position where I could split my time between a research center and business unit. HR never understood this hybrid role; it was far to innovative for them. Today many independent companies have built on concepts I began working on at Shell and they have surpassed Shell’s expertise in resource plays. Shell laid me off and recently wrote off a $1 Billion investment in an area where my current employer is doing well, in part because of my contribution. My point is, while Shell talks a good game about their creativity and open-mindedness, they are as blind to innovation as IBM was to the potential of software in 1980 when they contracted out to Microsoft. None of Shell’s ‘big data’ for personnel is useful to them in my experience and they suffer from an inability to match skills with jobs. I turned down Shell when they made me an offer to come back.

    Recently I was invited, expenses paid, to Paris to interview for Total. My first day was spent with a testing services contractor even though I am now mid-career. Because I was a non-traditional student in the US and paid my own way through some very expensive colleges, my school records looked peculiar to a Frenchwoman who was accustomed to the government-funded time-controlled education system of France (I actually began working at Shell before I graduated). This caused them real concern, since I had taken more years (while running a company) to complete my degrees in my mid 30’s than they considered proper. The idea that I owned my own company before I bothered to get a college degree was wildly beyond their experience. The tests were in French (and I do not speak French) and at the end they remarked that I had a perfect score on my answers to a multi-dimensional problem set, but that I had not completed enough work. I had failed by being proficient. I felt the interviews went quite well, but this test seemed to be a problem. Total lost an opportunity to have a professional with specialized knowledge of cutting-edge emerging concepts based on a comparison of a seasoned worker to new college graduates from the French university system. I still correspond with those interviewers, but now would not consider working for a company with this sort of screening process. The proponents of this ‘big data’ approach fail to realize that many highly paid consultants are the indirect product of unintuitive HR departments.

    My conclusion is that ‘big data’ is likely to create many more talented self-employed consultants who have the freedom (and the burden) to market their skills as they choose. Real talent will be more likely to fail the test because the test is not measuring what adds to the bottom line. HR is assuring the steady employment of many consultants because it is often the only way a hiring manager can bring in the skill set needed through the morass of HR.

  27. Nick,
    Very thought-provoking, the concept is essentially institutionalized laziness. Big Data in HR is the new “silver bullet” that will make recruiting simple, quick, and make every hire a 100% fit. Nothing that HR has introduced for this has worked yet, and Big Data won’t either.
    -ed

  28. Using remotely gaterhed data to make hiring decisions is ill advised, for all the reasons you point out —

    More worrying, if a hiring manager REALLY doesn’t want to interview candidates, and they still have the gig . . . . .

    Staffing is, if not the most important responsibility a manager has, certainly in the top two or three. Giving up the ability to make good decisions means giving up the responsibility — when a candidate tanks in teh first three months,”hey, I hore the best score” shouldn’t be an excuse for poor management decision-making.

    The best thing I ever did, as a hiring manager, was get HR as far out fo the recruiting process as possible.

  29. I would like to know how many talented people are choosing not to apply for those kinds of jobs. My skill set is in demand. Why would I waste my time filling out a fourteen page questionnaire?

  30. I applied last year for a role that I was well suited for and was referred to the hiring manager directly by a former co-worker. HR contacted me via email and told me to take 2 tests, one personality and one skills. As I was taking the skills test, I kept saying out load to myself “what does ‘two trains are speeding towards each other…’ have to do with marketing? Apparently I didn’t pass the test as I got a “thanks but no thanks” email. Never even spoke to anyone at the company other than to the person who referred me. This happened probably a year ago. I still see postings for the job. I guess their “big data” is causing them to hire wrong. Their loss. My lesson is that I will refuse those kinds of tests until after at least a phone screen, preferably after an in-person.

  31. I work with data analysts and have to understand, as a marketer and communicator, the concepts behind ‘big data’. ‘Big data’ is defined as the gathering of unrelated points of information to create a profile. However, anyone who barely survived Statistics 101 knows that correlation is not causality!

    People who are expert in it (like Mitch above) are still in the early stages of understanding its use and successfully applying it to basic business problems. Its use in HR is a classic case of misuse of a tool by the marginally (or in)competent. Add to that the ‘Survivor’ mentality of hiring–why so many smart and highly competent people are still looking. (FYI the latest trick is to NOT put a key requirement in the job description or the screening, so that the interview with the hiring manager is a ‘surprise’ on a skill set you perhaps haven’t refreshed in mind. So the conversation winds up being pointless and/or embarrassing for the candidate.)

    @Tom, given your treatment by the Total screener, no surprise that French business is stuck on ‘fail’ along with its government.

  32. Since any company does not need to be “The Best” to be successful, the old joke about the two hikers and the bear applies to companies that use “Big Data” just as much as it applies to job seekers who use job boards: For the employers and job seekers who prefer meeting and talking about the work face-to-face, we don’t need to be perfect (faster than the bear), we just need to be better than the other yahoos (faster than you).

    The two income opportunities to come out of the use of “Big Data” for hiring are 1) the companies that use human interaction to assess candidates will earn more money (as noted above) and 2) entrepreneurs can set up tutoring services to “teach the test” for hiring questionnaires in the same way that high stakes testing in government schools has led to the flourishing of SAT-ACT-GRE tutoring services.

    Sheesh, haven’t these folks ever heard of “Google-bombing”?

  33. Nick:

    Here are some thoughts:

    First: The HR people who are championing these tests at their companies should be the first to take the tests themselves to see if THEY should continue or be fired immediately. I bet you’d get no takers!

    Second: I know a corporate flight department which gets over 6000 qualified pilot resumes for every one opening there is. I asked the hiring manager what edges one candidate over the other since they are all pretty much qualified: His response: “…it comes down to whether you can sit next to a person for 15+ hours in the cockpit and not want to poke your eyes out with a fork.” Where in this Big Data process does the human quality get evaluated.

    Thanks

  34. Indirect assessment and use of correlation is a very powerful tool if used properly. In fact, it’s being used on all of us quite successfully whenever we receive a piece of marketing material, pop-up ad on the computer, or a diagnosis of many major diseases (often tests for diseases actually look for proteins and other antibodies that correlated highly with the disease since it is hard to actually spot the disease.

    The question as to whether employers are using it well depends on if they are using it to make a final decision or simply to vet a pool of candidates.

    If it is the former, and the employer is using correlation to pick the person who will fill the position, then they are confusing causation and correlation and are subject to the problems that you raised.

    However, if they are just trying to take a pool of 1000 applications to boil it down to 20-30 that they can look at closer (by a human), they are using it properly. Like or not, the correlations (if they were based on a large enough data set) will do a pretty good job of finding people who are likely to be good candidates. They will also miss some people who will be outstanding candidates and will let through some total duds. But, for the employer, it’s a simple risk/reward decision. Given that there is no job for which there is only one person in entire world who is qualified, from an employer’s perspective, it’s probably ok to miss a few qualified people. That sucks from a job seeker perspective but it’s actually a lot more efficient from an employer perspective. If you can automatically cull 80-90% of candidates and get a pool that has a 90% (or higher) probability of having a qualified and successful candidate, it saves a lot of time and money – enough to justify not seeing 100% of the perfect candidates.

    The article also seems to assume that the alternative, recruiters, act rationally when screening resumes. There is plenty of research that shows that recruiters can also be improperly swayed by someone’s school, prior employment, major, or crazier unexpected variables like the weight of the clipboard upon which the resume was placed.

    Given the research in behavioral science and how irrational our decision making can be, I would be much more comfortable knowing that my chances of getting an interview were based on correlations rather than the opinion of a human whose just looked at 450 other resumes, is tired, and is subject to a lot of irrational bias. I read one article that says the typical recruiter spends about six to ten seconds looking at a resume. That’s not a lot of time. From an algorithmic perspective, they are probably using rules that are considerably less accurate, more primitive, and more subject to bias.

  35. The biggest mistake people make is confusing correlation and causality as you pointed out. However, the second biggest mistake is discounting the predictive value of correlation. If you know that two things tend to occur at the same time (or if one is rarely seen with the other) you can use that information to make predictions.

    Insurance companies underwrite you based on your credit score. Certainly having a good credit score doesn’t cause you to be a good driver. However, the fact that good drivers often have good credit scores has made it easier for insurance companies to predict your risk. That’s the issue with recruiting. If they are doing it right, they aren’t making a definitive assessment of the candidate. They are making a prediction as to which people are more or less likely to be suited for the role. Big data and correlation works especially well for doing that even if it can’t make the final determination.

    In the case of the hospital, it’s true that death and doctors seem to show up a lot in the same place. And, as you said, it would be ridiculous to assume that the doctors are causing the deaths. However, if you were blindfolded and brought into a building and the only piece of data you were given that was that you were surrounded by a lot of doctors, it would be a pretty safe bet to assume that you were in a hospital. The bet would get even safer if they told you that you were also surrounded by nurses, sick people, and ambulance drivers – all other roles/things that correlate to being in a hospital. None of those cause people to be sick or die but they are all predictors of the place you’ll be.

    If you know that certain behaviors, preferences, or habits correlate with success at a job, it’s no more irrational to use that information to “guess” that a potential candidate might be successful and worth a further look that than it is to use the doctor, nurse, and ambulance driver data to help you “guess” that you are in a hospital.

    However, in both cases, you need some additional information to CONFIRM whether you guess is correct. In the case of recruiting, that comes from the actual interview which can now be more robust since you have a smaller pool of people to worry about.

  36. @Some Guy: Good points about the baseball analogy, and the difference between metrics about performance and square jaws!

    @George R. Goffe: Another good analogy. Adding up and averaging multiple inherently invalid metrics doesn’t produce a valid measure of anything!

  37. @Brad Kolar

    You raise some interesting points. But don’t we already have that sort of filtering now, like Taleo? And people now are screaming that there is a talent shortage, based on “simple” screening criteria?

  38. Nick

    I read your weekly emails all the time and I felt I finally had something to contribute back to you. I work in the so called big data space, I design those very systems everyone else talks about it. I have even made systems that are used by HR for hiring and selecting who gets to work on what (btw, that system was not designed for that purpose).

    Big data is a lot of hype and wow buzz by the media. Would I use a system to hire? No. And this from a guy who runs big data teams, data science teams and data visualization teams along with a strong understanding of the various machine learning tools and algorithms that are out there. I know the limits and I know that if I really want to know who I’m hiring and why, I still have to talk to them.

    The issues have not changed, hiring managers for the most part, don’t know what they want. I know because I get calls all the time by companies that want someone like me to build them these systems. They don’t understand what they want and if they don’t know that, then the algorithm is nothing more than a fancy guessing machine. Here’s an example: I’m a retailer and you buy a TV from me, I know what you bought, how you bought it, when you bought it and even where you bought it. I can collect all that data on you. But 99% of the big data systems will not tell me WHY you bought it. What was your intent? Was it for you, is it a replacement TV a brand new 4K upgrading, is it a gift or perhaps you bought it for your office because you are a small business owner? I don’t know and without that knowledge, I’m guessing as to what you will do with that TV. And that’s the danger. If I don’t know the intent of the action, I really don’t know anything.

    Same goes with hiring, if most of the people I have hired have MBA’s should I exclude those that don’t? What is my basis for this reasoning? Is it because I like them? Is it because I have an MBA and therefore I like to validate that I made the right choice all those years ago? Does the data show MBA’s do a better job for the role I’m hiring? If the answer is the last one then, how was the data scrubbed? What other data shows correlation? Is the data set large enough to make a decision off of? What type of algorithm was employed in the analysis and was the right one to use? Most hiring managers have no clue how to answer those question and certainly not most HR people. So the person making the decision is not qualified to make the decision based on big data. There in lies the problems.

    All these vendors out there selling the tools have some really nice presentations where everything always works just perfect. They fail to tell you that, sure you can use social mining but realize that about 1/10th of a percent will normally be useful. And yes, you can grab all kinds of data sets but most of it will be corrupt and you will need to clean it up which takes weeks, you are better off calling the candidate then, it will cost you less and save you a lot of time.

    Big data is not the answer to hiring, the algorithms are not good enough to really do the job. I still do it the old fashion way, understand what I want and why and then match the resumes that have what I want and then call the candidate. Algorithms are a set of assumptions someone made up, I wouldn’t hire or fire anyone based on those assumptions especially if I didn’t come up with them. The bias nature of the developer always shines in their algorithm. If they decide to score MBA’s lower because they don’t like MBA’s, how are 99% of HR and hiring mangers going to know this is in the formula? They won’t, they will just assume it “works.” Unless you know how to dive into and analyze algorithms, don’t even play in this space. If any hiring manager came to me and said we should use such a tool for hiring or firing, I’d recommend they be the first to get fired. The best computer on the planet can only keep up with the human brain’s processing abilities for a few seconds. Why hand over the decision making abilities to a tool that is rather stupid compared to the one between your ears.

  39. Hi Nick,

    I was planning a Huff Po piece on this because it’s a vitally important discussion. Having read all 6,000 words; I saw the argument as frightening. First, because it was so well written and presented. Arguments where you see the opponents’ points are the scary and the tough ones. I respect this writer,

    But the other reason it’s frightening is that it, as you already know, seductively beckons us down a dangerous path. One where the tools, and not the people, are building the houses where we all live. Like hammers floating through the air detached from human hands.

    So, a TV conversation would be a much better way to beat back the Data Monster Poltergeists. You’re the perfect knight to slay this dragon—I’m a fan—so rather than me writing a static piece for Huff Po, I’ll share my main points so you can use them (or not) in action at your discretion as the conversation unfolds.

    1. A data dump is different than a story. The absolute first thing ANY good analyst will say when faced with a data set is, “Now, let me tell you the STORY being communicated. A PERSON needs to do that. Data leaves out the nuances, tones and emotional narrative that make us human. To turn the keys to human judgment over to data is to lose the story of the candidate and their connection to the role.

    2. Data is not “Fit.” The talent of determining fit is what separates the grown-ups for the HR weenies. Fit is what’s between the lines in the job spec. Fit is what the client really wants. Not what he says he wants. Fit is like music. NOT the math in the structure of the music,

    3. Data can’t show community building. I call it communitizing in my book. It’s the work of really building community. It’s a fluid narrative. Shown as data, it’s worthless because communitizing is always in motion while data is a snapshot of a picture in time.

    4. Practicing Stewardship—How will big data show a candidates grasp of something larger than themselves? Like for example, the employer.

    5. The presumption of sameness. Every person’s story is different. If you don’t see the difference, you haven’t asked enough questions. By compressing the human soul and individual story into data points, you lose something. And when you lose something, your judgement of fit is skewed.

    Finally, I’d go easy in attacking the notion of correlation. (Disclaimer—I’m a former Gallup Consultant) While you are of course right that they do not imply cause, they can be a powerful hammer.

    But only if someone is holding that hammer!

    Best of luck and if there is anything else I can do to help you make our case, do not hesitate to give me a shout.

    Take Care,

    Roger Wright
    http://www.findingwork.org
    Read Roger Wright in The Huffington Post
    Rogerwright1@sbcglobal.net

  40. Most organizations are badly run. Was anyone else surprised by this?

    We KNOW enough about management science to know that culture trumps strategy, managing to adapt to change is essential, hiring Nick’s way works better, ‘loose-tight’ works, etc, etc. These are all very simple.

    Unfortunately it’s not easy to get them all right, and missing even one element can cause failure.
    So well run companies are all similar; badly run companies are badly run in many different ways.

    You’d think that well run companies would out-compete badly run companies, but once an organization gets big enough, it partly escapes the market and becomes resistant to extinction.

    Clausewitz- “Everything in war is very simple, but the simplest thing is very difficult.”
    Tolstoy — “All happy families are alike; each unhappy family is unhappy in its own way.”
    See Vladimir Arnold’s book “Catastrophe Theory”

  41. I subscribe to your emails and find them enlightening as well as disheartening.

    My question is, what is a job seeker to do to get a new job, especially when I want to switch careers?

    Because I have been a teacher for over a decade i do not think employers give my resume a chance because I haven’t done the “specific job” they are looking for. BUT I have skills and talents and I mention those on my resumes, which I tailor to each job I apply for. I want to move into curriculum development or research.

    I know I can do these jobs I apply for but how do I get in the door when I am rejected before even meeting me?

    I look forward to your tv appearance.

    thanks

  42. Let’s not forget the ease of cheating on e-skills tests, etc. Unless the candidate is right in front of your face you have no idea who is answering questions or how. The company I work for relies on these tests and the number of people who last even 3 months is not high. Then what does the boss do? She changes the questions on the tests!
    Companies just want asses in seats and hope for the best.

  43. @CAT: The big secret is that HR departments today are largely driven by the HR consulting firms they hire. And most of those are in turn run by failed HR executives who realized they could milk the system with virtually any “scientific sounding” nonsense they can dish up. Beats living on a salary any day.

    @Eric: There’s nothing like amplifying ineffective HR organizations with the power of math and science (aka, Big Data). Big Data can be very useful in HR – but that’s now how it’s used in most quarters. HR wants magic. It wants to surrender accountability. It wants excuses.

    @John Franklin: What you’re pointing out is that HR is determined to process as quickly as possible the masses of inappropriate applicants (along with the few appropriate ones) that it spends billions to solicit via job boards and automated means. Any “objective” method is as good as any other, and I don’t know many HR execs who understand how to vet the methodologies they buy and apply to the process. HR execs are sitting ducks.

  44. To start: This approach to hiring seems lazy and irresponsible…but maybe I’m just an old guy. I am clear, though, that I’d have never learned to be a software developer had the bank I was doing temp work for at the time used anything like these tests.

    The article: I think that the last paragraph in the story is telling. How do the Oakland A’s scouts find out the really good info? They talk to the players and their families. Wow, how high tech!

    One thing that’s not mentioned in the article is that in baseball, the player stats are generated in the course of playing a game which is the work of the professional ball player. The ball players they might hire are already playing and their performance is being tracked statistically. This seems appropriate and valid. They’re not tracking whether the players hang out at the water cooler or which brand of bottled water they prefer, they’re tracking fielding, hitting, and situational stats about their actual playing.

    Nonetheless, it’s worth noting that Mr. Beane and company have not won a World Series using this approach.

  45. @Survivor: Did you notice that most of the Big Data “firms” cited in the article are start-ups reliant on funding? Even Peck acknowledges that they may not be around long. The holes in the story they’re selling are big enough to drive the Dept of Labor through. :-)

  46. I think this is just a clear sign that millions of managers world-wide, have lost clear touch with reality, common sense, and leadership skills.

    I see so many management books at book stores, which indicate to me, that managers are mostly incompetent at best.

    Yet they keep being hired.

    Which truly indicates that there is a systematic failure in the culture of business that has lost touch with reality.

    If they can not determine who is worthy of being hired and who is not, and because of potential law suits, unwilling to take any chances. This only empowers the even more incompetent and unethical people in HR, to take over that power, and make the employment in the world completely chaotic as it is.

    I have read many of your mailing letters, Nick, and they and you impress me. But sadly I do not have much respect for most of business leaders in this world.

    We created the concept of corporations decades ago, to provide protection against personal assets against being sued if the company got sued for whatever reason.

    However there is a cultural consequences to that protection, it encourages the idea and feeling that there is no consequences to misbehavior, bad management, etc.

    People are so protected from consequences of their incompetency and lack of ethics, that is no surprise how screwed up the employment process has become.

    Until managers become leaders with values, and stand by them, and stand by the people they hire, this will never change.

    We need massive tort reform, so that leaders can hire who they think is the best person for the job, and fire who they think is not doing their job.

    We should encourage good decision making and good leadership, not penalize it.

  47. Re hiring:

    What are the parameters built into the software/algorithm? I believe it would reveal illegal discrimination of all flavors, including age. Ironic, as “old” corporate white men drive this behavior. Do we have an Edward Snowden who can help reveal this??

  48. @Christopher: Thanks for saying it. Big Data is the new Scientific Management and Time and Motion. Peck alluded to this, too, but didn’t follow through.

    @Nick F: Score 10 points for the best insight so far! Why hasn’t Hollywood adopted Monster.com to find its talent? Uh…

    @Ian R: Few people realize that one of the main reasons the US was able to win WWII was surprisingly quick deployment of enormous numbers of soldiers. (Hitler was shocked.) And key to that success was the IQ test, which enabled the armed services to assign soldiers to jobs they were likely to be able to handle. So, yes, there are ways to use tests to decide who does what job. But this example is an extreme one – the US was sending people out to do jobs and to die. We don’t see much of that in business today, do we?

    @michael mell: I’m not trying to “speak truth to power.” I’m just trying to make as loud a sound as I can, and most of the volume comes from the comments on these columns. I can’t tell you how proud I am of the incredible standard of discourse on these comment threads. A million thanks to everyone for the thoughtfulness behind your comments.

  49. @Mitch: I’d welcome an essay. If you write it, I’d love to discuss publishing it. The trouble with the companies in this space, from Monster to LinkedIn to those in the article we’re talking about is that they’re driven by database technology in search of an application. I see it all the time — look what we can do with these algorithms and with these pattern-matching techniques. Now, how can we sell it? What problem does it solve? Every employment issue looks like a nail to these guys with the database hammer in hand.

  50. @Mayor Bongo: Point taken, but I’d hire someone with a good business plan over someone with prior experience ALMOST any day. If I could talk to them a whole bunch first.

    @Michael: I love your “two income opportunities” challenge!

    Rich N: HR should take the tests first, to earn their jobs back. And you earn 10 points for another great insight! But now I’m wondering whether I want to be on a plane whose pilot is willing to sit next to a person for 15+ hours without poking their eyes out with a fork… Who wants THAT kind of a pilot???

    @Brad Kolar: Good point about comparing this to marketing. The problem is, if Big Data goes awry on a marketing pitch, the company loses some sales, tweaks the plan, and moves on, none the worse for wear. But if Big Data goes awry on hiring a bunch of wrong people, the company is stuck with huge costs, massive risks, and legal problems. But more to the point (based on your comments), the “intelligent gambling” approach to hiring that you describe is right on — except that today employers arguably have more applicants than ever in history — yet they cry there’s a talent shortage. Job boards — which rely on the culling process you describe — fill only about 10% of all jobs.

    So there’s a problem here that this approach is not dealing with. What is it?

    (Regarding recruiters: That’s another story. Most are not very good at all at selecting hires. But that failure isn’t an argument for an alternative that hasn’t in fact been very productive.)

  51. Hi Nick,

    Yes, I am frustrated by the current hiring practices. I work on an application, research the company, tailor my resume, and do not receive any response. Rarely can I get the name of a hiring manager, and if I do, I have to leave voice mail message which is not responded to. I find the requirement to disclose salary invasive and impractical – that is what negotiations are for, and if I am the right candidate then I would want the job and they would want me. I am turned off by the atmosphere of automation that does not value the whole candidate enough to determine if they are worthy of face time. I find it unbalanced and unfair – all the power is in the employer’s hands and they are taking full advantage of that fact. I do not mind proving that I can do the job, I do not mind discussing my experience and how I would handle various challenges. I am happy to sit and be scrutinized – by real people who know what the open position requires. The current process is dehumanizing; it negates the value of all I have accomplished.

    Other perceptions:
    1) This is a case of a product being invented, capitalizing on a current trend. Marketing kicks in to create a need for the product. Articles are published, the machine kicks into gear and the product becomes necessary.
    2) This is the relentless march of technology as the answer, regardless if it works. Technology that cuts costs, at any cost.
    3) Our economy is dependent on consumers spending money. The current scenario is: too many unemployed or under-employed, too little disposable income to purchase products = too little growth. Except in the technology sector, which is busy inventing products and creating needs.
    4) People Analytics is an absurd concept if it is used beyond a few tried-and-true areas. Data is not the key to predict human behavior. Human beings are complex and behave differently depending on environment, stress, motivation, rewards, etc. Humans interact, and interactions and relationships determine to some extent what we do. No employee works in a vacuum. The employer creates the work environment; the employer is half the equation.
    5) Who are these HR staffers who want to hire, fire, and promote based solely on analytics? Where were they educated, what is their work experience, their life experience? What is driving this trend? Is top management telling them to hurry the hire, to cut costs? Sure these HR staffers overwhelmed by the number of applicants, and unwilling or unable to figure out a way to weed out the unqualified or unwanted applicants that get through the filters. Are they just not capable of doing the work that is their job? Are these HR staffers held to standards; is their performance measured through the long-term productivity and contributions of their hires?
    6) Analytics depends on good criteria – the results you get depend on how you build the algorithm – what exactly are you trying to analyze, what steps are built into the calculation? I would not trust any outside firm to analyze my firm’s data without my input. Garbage in, garbage out.
    7) I would have a problem if I was expected to wear a device that recorded my words and actions, and all the data gathered was put into a computer which then assessed my performance. I want to be judged on my output – my work, and the relationships I build. I have hired, fired, reviewed, and disciplined people who reported to me. I took the time to know their performance level and to discuss any issues with them. I took the time to write their reviews, and I expect that respect from my supervisor. I do not believe that any data analysis can fairly review a human’s performance – certainly not if soft skills and relationships that benefit the company are taken into account.
    8) What kind of world do we want to live and work in? That is part of this discussion. I will not be part of a company that solely measures data. Years of research have proven that people need to feel they belong to a community at work, and that their work and character are valued. Humans are motivated by relationships and rewards, not data analysis. Do we have to learn this all over again?
    9) Businesses must be flexible – able to respond quickly to changes in the market and the world. Those who rely on interpreting data only run the risk of being too slow, or too wrong. I want leadership, vision, and solid implementation. Creative, innovative, yet practical goals reached through well-utilized human potential. These are the attributes and values that build good companies.
    10) Problem: too many people looking for jobs and not enough jobs. Too many qualified or overqualified for the low-wage service sector jobs that are the norm. Let’s stay focused on that. Yet the education/industrial complex wants us to believe that we all need advanced degrees….

    Go get ‘em, Nick!

  52. the premise of big data is that under a large pile of s#!t you will find a pony.

  53. Nick,
    Since the inception of Online Application System began in the mid-1990’s I question how the EEOC has allowed this black hole of tracking applications to continue. It is well documented that these systems have blanked out numerous qualified candidates for the thousands of job openings in the U.S.
    I was an HR person in the 1970’s and believe me it was difficult to review scores of resumes and applications to determine who would make the cut. I almost didn’t make the cut because I didn’t have a BA but the hiring supervisor felt that I was a far better fit than the BA candidates who did turn down the job because the job title was Personnel Records Clerk (moving hard copy personnel records to computerized data historical system).
    Big Data has been evolving since the layoffs of multi-degreed math folks since the mid-1980’s due to the decrease of US contracts for NASA and related industries. The US financial corporations hired these number crunchers to launch financial products to circumvent the laws enacted during the depression to insure the financial crisis of then would be regulated to prevent another one. HA!
    With the increase of math structured analysis of the financial markets, then this analysis was move to marketing as to how consumers spent their paychecks from groceries to homes, then onto evolution of BIG DATA came to Online Application System to aid in hiring and reduce costs and headcount in HR/Talent Acquisition Departments on the heels of TQM. What a fraud!
    Yes, there are some measures of human behavior that may be a produce a ‘true’ assessment of outcomes, however, BIG DATA Analytics has a major flaw. The human being is very changeable and will always be an enigma to metrics.
    It is time for the CEO’s and their board of directors to recognize that we, human beings, are REAL and it is time to re-direct their HR departments become more human not worried if their HR teams have multiple acronyms for certifications. It is time to put HUMAN back into Human Resources from actually reading the resumes responding to the job boards. The CEO and board needs to remember job seekers are consumers of their end product (from auto sales, CVS retail stores, MetLife, Banks, grocery stores, energy/utility companies).
    Some of these job boards show hundreds of open positions, but many jobs are the same position with new posting #’s. Often there is no information regarding the start and end dates of these posting – just a merry go round to confuse the potential employee and perhaps their shareholders and investors.
    And often once you do apply either to the in-house online application system or through an agency who is hired to find a temp/contractor until the in-house HR fills the permanent position – you invariably are notified that you made the short list but it has been placed on hold, or cancelled or the position has been off-shored by both ‘hiring units’.
    One wonders if these companies aren’t using all of the available resumes to great a huge data pool just for Tax Credits and subsidies.
    And one of things I’ve noticed once I have secured an in-person interview, no one seems to have a business card or willing to share either their email address or phone #, but as an interviewee I’m suppose to send a thank you note within 24 hours. The snail mail ones have been returned due to improper addresses, and a tad of annoyance that I figured out how to contact by email — we live in such a strange, strange world. Hiring managers feeling inconvenience by someone who would help make their workday easier. Go figure.
    and what is this new thing – gamification???? I just wish I had a rich uncle who was being forced to finance so of my wild ideas!

  54. The phrase ‘big data’ seems to have become associated with ‘big brother.’ In fact, ‘big data’ is a meaningless piece of jargon now being applied to a diverse range of situations.

    As a frequent victim of job discrimination for my sex, age and other irrelevant features, the responsible collection of data that can be shown relevant sounds valuable. Like auditioning orchestra musicians behind the screen. That’s not necessarily ‘big data.’ I’d like to see more ‘good data.’

    Also, you may find the NY Times article from last April valuable: http://www.nytimes.com/2013/04/21/technology/big-data-trying-to-build-better-workers.html.

  55. Big data, as it’s used by large HR departments, is big bullshit. Web portals and applicant tracking systems probably have more to do with why nobody gets hired anymore than anyone wants to admit. I bet if we dumped those systems in favor of newspaper ads, phone calls and faxes the unemployment rate would decrease fairly quickly; that is, if the jobs advertised were real,not chum.

  56. Nick,

    There may be a place for Big Data in business decision making, even hiring and firing. However,unlike MLB, there are no companies that are in industries that are pure monopolies. In MLB, the exact same performance metrics apply to all teams, and each position. Even in MLB, you have to be able to analyze, interpret and apply learnings from ‘Big Data’. This is where companies will fall down.

    Companies using ‘Big Data’ hiring and firing are doing so to avoid making mistakes. It vastly underestimates the company’s work culture on individual and collective performance. It devalues intangibles of performing in a particular environment and it will limit a company’s need to change or renew itself, as anyone that doesn’t fit the mold, doesn’t even get considered. Big Data is part of the problem because there does not seem to be the real ability to properly use it.

  57. @Nick Congratulations on this opportunity for representing many voices that may not otherwise be heard. I had a feeling that this day would come for you.

    Big Data/Spying/Illegally interpreting people is everywhere, and even start ups on angel.co are bragging about human resource assessment modules that they have programmed for small businesses to utilize as a service to “find” the best people for their companies. These modules are touted proudly by these psychologists and programmer heads from these startups with some hailing from MIT and other top school names making it seem as if they have “leading edge” technology that is flowing with the progressive times. Phooey. They are just (in my opinion) jumping on this Big Data bandwagon following status quo instead of creating something totally revolutionary, let’s say, “a cut to the grit guide” based on proven tried and true human interaction practices for hiring that they can write in an e-book and sell on Amazon. Instead, they want a piece of the “big data” economical pie,touting a computerized algorithm that is taking the most valuable thing away from the equation: human to human interaction and connection.

    I have no problem with getting paid for creative efforts that people could pay for to use but these people do not come off as intelligent as they lead on to be. I see it for what it is, jumping on a bandwagon with their spin on it. Not impressed, even if they do have a Stanford or MIT programmer on board behind its inception or have a PHD from Columbia on psychological behavior as the assessment tool creator.

    @Katherine Fullerton you get it and see the forest over the trees… Your comments are DEAD ON! But I continue…

    There is absolutely, and I mean abso-freakin’-lutely no substitute than to vet out the best people for your team based on sound psychological face-to-face practices, interaction (email, phone or otherwise), and talking to potential solid candidates for employment than leaving it to a computer or computer program to do it for you. Not nurturing true connection and interaction (which is a vital component of our human existence)may lead us down a road that may regress us from the one thing that gives us the satisfaction of making a good decision based on the bridging of our intellect and soul: our intuition. Killing it, may also confidence.

    @Nick, tell it like it is. Of course have some solid research as you have found from Peter Capelli, but if you can connect with him to ask if he knows other researchers who support what you and the rest of us know about the hiring game and what it s outcome really inspires (or not inspires).

    Smart comments like the ones from @Katherine Fullerton, @MDH @Ray Stoddard and others to bring in as real people interpretations to quote to let “them” know that truly ingenious minds see through this Tom Foolery. A mix of backing up what you know with solid research and out of the box interpretations from your readers is enough to hit it out of the park.

    Good luck and know that we support you. Make us proud!!!

  58. The question, of course, is does this junk work. You would think that Google has gotten pretty good at analyzing tons of data, but on my computer at work, they think I’m a 45-54 year old white female. They got the white thing right, but I’m 65 and male. (Go to http://www.google.com/settings/ads/onweb to find out what Google thinks of you)

  59. Analytics is fine but you have to have some testing to show your analytics are correct. Lots of data means nothing if you can’t demonstrate relationships.

    By definition, Big Data is more than 1 terabyte of data. What is called Big Data analysis is nothing more than statistical analysis.

  60. Five years ago, I fell into clinical depression because after I lost my job, I sensed that because of the cybertrends in hiring, I would forever be excluded from the labor pool. I didn’t explain the details of these fears to my shrink, because I didn’t think that she would believe me.

    Five years later, the problem is only beginning to see light.

    The last few paragraphs of Mr. Peck’s article actually give me hope–the human touch in recruiting is making a slow comeback in the high tech companies, and the recruiting net is slowly being cast over accomplished but uncredentialed people such as myself.

    Hopefully, the discussion on TV will accelerate the fall of algorithms, and assist the resurgence of person-to-person hiring.

  61. The first thing about these online applications that really gets me is they ask for (and require) a date of birth. I thought this was illegal. These so-called psychological questions about hypothetical job situations really annoy me. Many of these questions I feel have gray areas. Yes I understand that if you see someone stealing, you report it, you never “clock-in or out” for your buddy because of some supposed personal emergency and other common sense violations of any company’s rules.

    But what do you do if a fellow worker needs help with something and you have your own work to do, and there’s no one around to help him? What if the co-worker is relatively new to the company and doesn’t know how to handle the problem they’re facing? These are actual questions I have been asked on on-line applications. The choices were something like: don’t help the co-worker, send them to the supervisor and continue your own work; ignore your work to help out the co-worker until their work was finished; tell them “I don’t know” and go about your business. To me I’ll help for a few minutes to get them on the right track, but there is never any gray area answers. If I could speak to a human(ha-ha)
    I could could give them a perfectly sound, logical reason for my decision and actions, but machines don’t wan’t to know about such things.

  62. OK – I have been thinking about this issue for a few days now, and I have re-read the article. Here are a few additional comments I have:

    Most of the comments on the blog are negative. Mine was – the idea of being judged by a machine raised my hackles. As I step back and consider the contents of the article objectively, I think there may be possibilities for good in these new products, if users are very careful not to overreach, be unethical, or abuse power, and if meeting the candidates is still the ultimate factor. We shall see how it plays out, and the conversation and scrutiny must continue.

    There is a difference between what Evolv is doing and the applicant-screening software that parses on-line applications and resumes, and misses good candidates. The real debacle is how arbitrary the hiring practices are now.

    We can agree that employers need some way of filtering applicants in today’s employment market – there are too many applicants for each open position. We can agree that most applicant-screening software in use is failing; or, at best, it is a numbers game – it whittles the pool down to manageable size in the hopes that in that pool will be the perfect candidate. It appears to be based on key-words and education (I do not have real knowledge of the parameters). This system becomes an impediment to job-seekers, and it does not require the best HR methods of matching an applicant to a position. I have visions of inexperienced HR staffers spending 10 seconds on each application that gets through, and making determinations based on who knows what. What a ridiculous scenario that is – applicant spends hours on a cover letter and tailored resume after researching the company, and gets 10 seconds.

    Some possible uses for people analytics include: applicants can all take the test as part of the application process. This would be fairer than testing only those who survive the first cut.

    Additional opportunities for good include matching candidates with career paths or employers, as a tool for job-seekers to understand their proficiencies, as a tool to teach managers and staffers how to improve soft skills as well as productivity, as a tool to eliminate (or at least reduce) bias in hiring.

    If we take what these new statistical analysis programs in the article show, we can see that college degrees and previous matching experience are not good indicators of performance. If companies actually learn these facts and change their hiring behavior, that is a good thing. I perceive that ageism, racism, elitism are rampant in the job market since the Great Recession and that has to change if this country is to prosper.

    I need to be convinced that these programs are created to avoid bias and sloppy correlation/causality relationships. I need to be convinced that those candidates who can do the job, and fit into the company culture, are given an interview. I strongly believe that people respond to questions differently depending on the circumstances. A test, where success or failure depends on answering correctly, is a stress situation. I also believe that when sitting down to complete a pre-employment test, the applicant tries to answer what they believe will get them the job, not necessarily the answers they would give if there was no pressure to succeed or fail. As for wearing a badge that is recording all of a staffer’s interactions, I can see people quickly learning how to do what is expected or praised, whether genuine or not. I would like to know how these things are factored out.

    I have seen job postings in San Francisco for customer service paying $11 and requiring a Master Degree. Many employers will not hire an unemployed person. Employers require a candidate know how to use even relatively obscure software programs, instead of hiring a good fit who can learn quickly – they don’t want to train at all. All of these things make no sense.

    I have noticed in my job search that very few of the best practices in HR are currently being practiced – somehow, in this economy, companies have un-learned a great deal of what works, and it is showing. Entrenchment (positions that have no merit are defended), greed, ego, fear are all in evidence. Repeat something enough times and it will be believed; one can usually find some fact to back up a position, regardless of its efficacy or ethics. The concept of opportunity, of being able to work to support oneself and one’s family, are casualties. There is not enough being said about the true unemployment rate.

    We are in deep doo-doo.

  63. @Kate Fullerton: Thanks for your very thorough comments. But I think much of your analysis is predicated on a false premise:

    “We can agree that employers need some way of filtering applicants in today’s employment market – there are too many applicants for each open position.”

    I don’t agree with that at all.

    There are too many applicants only because HR solicits them and encourages mindless, automated applications to jobs “just because they’re there” and “because it’s a numbers game and I need to up my odds of success.”

    HR has created this false “need” for “some way of filtering applicants” because it uses the very tools that you suggest are now necessary — to stimulate those masses of inappropriate applications.

    The only way to stop the misuse of technology and algorithms when processing applicants is to stop using it to encourage people to apply for every job they see. Strip out the systems that generate all those applicants and — almost presto — HR has time to thoughtfully review small numbers of more appropriate applicants.

    What HR can’t figure out is how to recruit small numbers of really good applicants. The joke is, HR used to do it all the time. It forgot.

  64. Hello Nick,

    You misunderstand – I am not suggesting that these tools are necessary. I am suggesting that given the situation created by HR and the millions of unemployed, they are not likely to go away any time soon. My point is that we have to deal with the situation as it exists, and if the new ‘people analytics’ trend continues, there is a spectrum of possibilities – some of which have potential for good and some not.

    You have an inside perspective that I do not. My comments are responding to the matter as it stands now. I agree that the ‘mindless’ stimulation of masses of applicants is not the way to go – I do not endorse it and believe it does not work. If you can use your influence to change that, then we all shall rejoice.

    While employer’s on-line solicitation is out of line, there are indeed many more job-seekers than there are jobs. Many small companies do not have an HR department at all. I do not think we are going back to newspaper ads, and not all companies will use a recruiter.

    So, Nick, do your best to persuade and educate. If the message is repeated enough perhaps it will start to stick.

  65. Hi Nick,

    Seth Godin recently posted a blog titled “Measuring nothing (with great accuracy)”, saying that just because something *can* be measured does not mean that it should be. http://sethgodin.typepad.com/seths_blog/2014/01/measuring-nothing-with-great-accuracy.html

    The prevalence of Big Data, especially in this context, always reminds me of internet dating. Everyone knows or even has horror stories from such sites, and those far outnumber the happily ever afters. Ergo, Big Data will not find my Mister Right, be that a man or an employer.

    This is a glaring example of how technological ‘advances’ can do more harm than good, and how reverting to what is now considered old-fashioned will almost always prove much more promising.

  66. Had to relay this real time app rejection:

    Applied for position via email as requested in November 2013.

    Rejected as follows:

    “Thank you for submitting your resume for the Controller position. We were fortunate in the number of highly qualified individuals who applied for the position, and regret to inform you that we have filled the position. We would like to thank you again for your time and interest in T P Company.”

    Same job posted again 1/22/14. I wrote:

    “If you filled this position in November, why is it posted again on imatch through worksource Oregon?”

    I await their response w/bated breath.

  67. Nick,

    Thanks for the compliment! I will give more thought to the essay and let you know…

    In the meantime, Liz Ryan wrote about the topic in Forbes this week: http://www.forbes.com/sites/lizryan/2014/01/29/how-technology-killed-recruiting/

  68. I’m coming late to the discussion, but I’d wanted to thoroughly read and digest the lengthy article first.

    Nick, thanks for the letter and link. Job hunting is getting scarier and scarier. Tracking employees? All of this metadata? And employers are still crying about a talent shortage and skills gap? Whatever happened to using brains (the ones in human heads) and common sense when it comes to hiring?

    My fear is that with the push towards online everything and with employers refusing to engage with prospective employees, more and more control is being ceded to the machines. Computers cannot think, cannot make judgment calls.

    Where are the hiring managers in all of this? Surely there must be some hiring managers who have a modicum of common sense and don’t want to rely upon a computer or big HR data or cede all of their hiring authority to HR and computers? If not, then where are the boards of directors?

    It seems that whoever is making these decisions are merely attracted by and chasing the latest, newest, bright shiny thing without thinking it through or without thinking that the last technology hasn’t been helping getting vacancies filled, so what makes them think the newer shiny trinket will be different?

    Nick, I hope that you’re going to be on 60 minutes or 20/20 or some other show, and that you’ll have the whole hour to get into this topic. Technology is great, provided that it is used correctly, but it really can’t be a substitute for human judgment.

  69. I could see companies trying this as a supplement to the hiring process, but for replacing it is beyond crazy. Statistics and data mining are well known for bizarre and spurious correlations that are meaningless coincidences. Sort of like folks that try to find secret messages in the bible, or something similar, or play records backwards.

    Reminds me of several decades ago when it wasn’t that unusual for employment applications to have a box requesting a handwriting sample. I always thought this quite odd, and often wondered why. So I was astonished to learn many years later this was intended for graphoanalysis (handwriting analysis) to assess the candidate’s “character”. I was dumbstruck that even major name brand corporations would fall for such an unproven concept, which allegedly, was especially popular in Europe.

    Regarding Evolv et al, I was also disturbed that such apparently small non-random samples were used to generate this data, with no controls against selection bias. No wonder they get such weird correlations. To use such for screening would just enforce conformity to a very specific set of characteristics in a particular time and place, kind of like an eccentric hazing ritual for some secret club.

  70. What’s up, yes this paragraph is in fact good and I have learned
    lot of things from it concerning blogging. thanks.

  71. I’m an electronics technician, recently lost my job to a factory closing. Yes there are still factories in Maine, and we are still closing profitable factories to send the jobs offshore. It has been a long shutdown process. After looking around, I decided my best option is to use the federal retraining program to get into a new field. I am middle aged, my husband is close to retirement, so it would be foolish to try to relocate. One of the conditions of the program is job hunting using the department of labor website. It displays the percentage match of every listing based upon the resume you upload to the site. It amazed me that I had a 97% match for an engineer position that my education and experience in no way qualified me for. I would be completely incapable of doing any of the described duties, which were more electrical industrial applications than PCB assembly and test which is the environment I have worked in for 14 years. This is a perfect proof of your doubt that data can determine a good candidate.

  72. Nick, I’d be interested in a personal reply to the suggestion I made for Bella (rx’d today) I do very much agree on this “I don’t agree with that at all.

    There are too many applicants only because HR solicits them and encourages mindless, automated applications to jobs “just because they’re there” and “because it’s a numbers game and I need to up my odds of success.”

    HR has created this false “need” for “some way of filtering applicants” because it uses the very tools that you suggest are now necessary — to stimulate those masses of inappropriate applications.

    The only way to stop the misuse of technology and algorithms when processing applicants is to stop using it to encourage people to apply for every job they see. Strip out the systems that generate all those applicants and — almost presto — HR has time to thoughtfully review small numbers of more appropriate applicants.”

    I’ve always interviewed personally every applicant for a job I have advertised…I’m the person who has profoundly known what I want and what I don’t for the Company and I consider it an obligation, and an honour to help every unsuccessful candidate recover, improve their chances of success and give advice. My awareness of the ‘as long as it takes’ global conspiracy commencing in the 18th Century which is I think really undermining human respect and decency and made worse by techno-frenzy, fear and drastic environmental issues and overpopulation issues I hope hasn’t affected my judgement in giving advice. I’d appreciate your review and advice on my advice to Bella. Warm Regards Jack