Why I refuse to do take-home coding tests - and why I think you should not ask me to.

I wrote the majority of this post two years ago when I was actively interviewing for new roles. I never ended up publishing it, as I have not been actively looking for roles since then, but there has been quite a boom of new roles recently, and I receive quite a lot of job descriptions both via e-mail and LinkedIn. I noticed that a lot of companies still have take-home coding tests/exercises as part of their application process, which continues to baffle me. Therefore, my mind took me back to this piece, which I finally finished.

And this is the gem that inspired the post - a fun way to spend your weekend:

The exact proportion of companies that choose to include homework in their interview process is unknown to me, but based on the job specs I get send, I initially thought it could be anywhere between 20-40%. However, as a quick 20 minute search of a jobs board showed, the number can be as high as 50-60%. So why is it so popular? The “obvious” reason for using such a method is that it is cheap - you will find out what the candidate is capable of and you will not need to spend company time doing a technical interview with them - only to check their solution. However, I actually believe that both these premises are false. You will NOT find out what the candidate is capable of and you will end up spending more time than you save.

I will explain why I made this statement, but first, let me present my position from the candidate’s perspective. I have made the decision years ago to refuse any homework as part of an interview process. My first questions to recruiters are always about the salary and the interview process, as these are the two things that are easy to filter out without wasting 15-30 minutes for an intro call. If I find out that a take-home coding exercise is part of the process, I state immediately that I will not partake in any such tests. I sometimes offer to provide my coding portfolio as a compromise - I am happy to talk about any of my past or side projects, and some companies are okay with that. Let me explain why I made this decision.

First of all, coding “challenges” are rarely challenging, they just take a long time. Most of the take-home tests I’ve had to do were very similar - develop an API with basic functionality. Most of the time is spent on the boilerplate that is needed for the API to work independently, which can be very lightweight if you are using Node or Python, or pretty cumbersome if you are using Java or Scala, as well as things like setting up testing frameworks and CI pipelines, as most companies expect the app to be "production-ready". After doing a few of these challenges, I really don’t feel like doing them any more because it feels like a chore. I would rather work on something I enjoy or am interested in, rather than doing the same thing all over again just to prove myself to a company I haven't even met anyone from.

More importantly, I get nothing out of doing the coding test. If I do a live coding interview with the company, then I get to meet one of the engineers. That is valuable to me — it’s one of the reasons why I do the interview. The company doesn't just get to pick the best candidate, the candidates are also picking the best company for themselves. By doing the coding test, I invest several hours, or, potentially, even a weekend of my time for no benefit whatsoever and no guarantee of a job offer, interview, or even feedback. Yes, I did once spend multiple evenings working on a task and only got to know that I will not get an interview — with no explanation as to why. 

Let’s go back to whether this is actually an effective selection technique. I am sure I am not the only person who does not want to waste their time on coding challenges - again, I don’t know the exact number, but I doubt it’s small. In addition, anyone who simply does not have time to do those challenges, e.g. people with families or other commitments are also eliminated. Have we eliminated the worst candidates? Definitely not - only those who don’t have the time or desire to take part. 

“Isn’t it a good thing?” I can hear you say. “Surely if you are passionate and good at what you do, you can get this challenge out of the way quickly.”

Therein lies the problem. When you are reviewing the challenge, you have no idea how long they spent doing it, which is of considerable relevance. If I can do something in 2 hours that would take another candidate 10 hours, then I am 5 times more efficient. However, you are not going to know that when looking at our solutions. Not to mention you have absolutely no idea whether or not they had any help doing the challenge. 

I remember bringing it up with a CTO of a company in a recruitment event once. His response to me was that cheating in take-home tests is pointless, because you will get found out later on in the process. My counterpoint is that if you rely on later stages of the interview process, then what is the point of the coding test? Is it just an obstacle to filter out the people who will refuse to do it? I am willing to go out on a limb and say that you have eliminated some of the best candidates, who have other options and can afford to miss out on a potential job offer.

Another argument I heard in the past in favour of these tests is that it allows you to see how the candidate works without the time pressure of the interview situation. Afterwards, they can discuss the solution with the candidate to understand their design decisions. I respect this point of view, but what’s wrong with having them talk about a personal project that they have done in the past? Doesn’t it accomplish the same thing? Admittedly, some companies do substituted coding tasks for personal projects, but in my experience, they are in the minority.

So what are the alternatives?

Contrary to what you may believe, you do not need to bring someone in to interview them properly. I think if you are smart about the questions you ask, you can easily check someone’s understanding remotely. Ask a few conceptual questions in the initial phone call. Schedule a 1-hour remote programming session with the candidates you are most interested in. Have them share their screen and work on a problem. You will be able to see how they work and how they think without bringing them into the office. Yes, it will cost you more time than just having them do a homework exercise. But you will learn an awful lot more about them and the quality of the candidates you bring in for a face-to-face will undoubtedly increase. And, more importantly, the impression you make on candidates will also be a lot better and you will have fewer good applicants drop out at early stages because they don’t want to waste time doing homework.

TL;DR - take-home coding tests are not worth the candidate's time. And I don't think they help you hire good developers either.

Anyway, the point of this article is not just to put my opinion forward (or to put off up to 60% hiring managers from hiring me). I am genuinely interested in hearing some of the other perspectives on this as well:

• Engineers, who like take-home exercises - why? Have I missed anything?
• Hiring managers, who believe it is a good selection method - why? Does your experience show that it is actually the best method?

Perhaps at the end of the day, I will end up with a different perspective on this!

EDIT: I received a message privately, which raised some very sensible points. I should have clarified that this article applies to mid-level engineers and above. I can totally see the value of take-home exercises for internships / graduate positions where the candidates might feel more comfortable working on a problem in their own time and where there might be less incentive for the company to invest too much into interviewing every candidate.

I also see the point of using take-home exercises as a kind of “show that you want to work for us approach”. This perhaps works if you are a Silicon Valley company with thousands of applicants as a way to filter out the ones that are serious about applying, but I don't think it applies to the vast majority of companies. It especially does not apply when the company is approaching the candidate, which is increasingly the case these days.