Everything you wanted to know about candidate code challenges (but were afraid to ask)

Man smiling at camera on smartphone

4 mins, 29 secs read time

Code Challenges are used by hiring teams to assess the technical skills of a software engineer.

Used as part of the hiring flow, a code challenge can take various forms but essentially it’s a technical problem that a candidate is asked to solve to enable the hiring team to decide on their (technical) suitability for the role.

A code challenge can take many forms, from a multiple choice questionnaire to a detailed specification requiring the candidate to build a fully functioning application, pulling data from a company’s APIs.

There are a number of problem areas that you should consider when using code challenges:

  • Developers feel code challenges aren’t a proper reflection of their skills. This is particularly true if the challenge is an algorithm that they might have learned in school or in a training program, and have never used since.
  • Too time consuming. When applying for different roles, candidates are asked to take multiple code challenges with a lot of these taking hours to complete. This means it can be incredibly time consuming. This can often lead to them taking the wrong role because of the effort involved in completing too many challenges.
  • Lack of feedback from the hiring team. After spending hours (in some cases, days) completing a code challenge, candidates who receive a curt response of, “Sorry, you didn’t pass the challenge,” will inevitably have a negative impression of your hiring process and employer brand.

Most of these problem areas can be overcome with a small amount of effort.

Here are some top tips to think about when using code challenges in your recruiting process:

  • Think about the challenge itself. Create real world challenges, not brain-numbing algorithms.
    • Why not create a challenge that uses your brand? If you are an e-commerce business, why not ask the candidate to build a shopping basket and checkout flow. Perhaps even use some of your product data and brand assets to add to the authenticity. Data shows these drive more engagement and increase the return rate of your challenges.
  • Provide feedback. Especially if you are not going to move the candidate forward in the hiring process. Why not let them know where they went wrong so that they can skill up in time for their next interview? They will walk away with a positive feeling about your business and more than likely spread enthusiastic, and very valuable word of mouth about you and your team.
  • Think about how long it will take the candidate to complete your challenge. Asking a candidate to spend all weekend on your challenge is a big ask. Limit challenges to a few hours. Anything less and you probably won’t get any meaningful insight.
  • Think about the environment they use. Lots of automated platforms use a browser based Integrated Development Environment (IDE). While this allows all sorts of fancy features such as video playback, it is alien to the user. They are likely much more familiar with their own IDE. Why not allow them to work in this own IDE and then copy and paste their code into the browser?

Different types of code challenge

Automated vs peer reviewed

Automated code challenges use machines to review the candidate’s submission. These are quick, giving results almost immediately. They are also inexpensive to operate as the per unit cost is minimal (although annual subscriptions may be applied).

As the name suggests, peer reviewed challenges are reviewed by humans. With this approach,the feedback is much more detailed and provides insight to both the candidate and the review team about ‘how’ the candidate codes as well as whether they solved the solution.

Time-limited or open-ended

There are two schools of thought on whether challenges should be time-constrained or open-ended.

Some argue, and they do have a point, that restricting someone to two hours to complete a task is not something that you have to do during your day job (aside from fixing production bugs - but no one has production bugs, do they!).

At Geektastic, we have seen a lower return rate where the challenge is left open for a long period of time. There are a couple of reasons for this. Firstly, there is the fear that other candidates in the process will spend wildly different amounts of time on the task. If you are given seven days to complete the challenge, how do you know other people trying out for the role won’t take longer than the suggested time.

One of our client’s code challenges is a 10-day task. Suggested time to complete is 4 hours over this 10-day period. The average time to complete the code challenge is 11 hours - with the longest being 40 hours (yes, 40 hours - that’s an entire working week!).

Our suggestion is to create a time-constrained challenge, with one caveat. Give the user ample time to complete it. If it’s a two-hour challenge, set it at 3 hours to remove the pressure. An important part of the hiring process is to level the playing field by ensuring all candidates are receiving the same amount of information and are working within the same parameters to complete a take-home task or code challenge.

After all the time and effort your talent acquisition team puts into sourcing top candidates, it's worth evaluating how you're delivering code challenges to prospective technical employees. The outcome: you'll help your company receive more submissions and improve the experience for everyone involved, from your candidates to every member of the hiring team.