I’ve both taken and created a lot of coding interviews throughout the years. And I’ve learned there are many ways to screw it up. The worst tests feel like a way to make the interviewer feel smart by probing for obscure knowledge deep in the bowels of a technology.
“Wrong! Due to type coercion it’s evaluated as Hex!”
[Insert raucous laughter and pointing here]
This exact exchange actually happened to me years ago. I remain scarred. Despite my failure on the gotcha, they offered me the job. I politely declined. Interactions like these are a double failure. They turn off the candidate, and they don’t give the interviewer the information they really need.
The best coding tests provide the candidate and the interviewer a glimpse at what it would be like to work together. So professional coding interviews should be characterized by these three traits:
1. Foster Conversation
A good coding interview question should be open ended and broad. Why? We want to foster a conversation about the developer’s preferences, style, and knowledge. The goal isn’t trivia. The goal is to get a sense of how the candidate solves problems. For reasons that you’ll see below, specific technical questions don’t belong in a coding interview. They belong in the subsequent technical interview. The coding interview is about solving a relevant problem so we can discuss your solution. After the coding interview is complete, the technical interview is a natural place to discuss specific technical areas.
The solution created in a coding interview guides the conversation in the subsequent technical interview. The conversation that stems from the coding test is where the real value lies. Why? Because you’re looking for a true fan.
Okay, but let’s not overdo it.
See, I don’t know jack about Major League Baseball. Yet if you give me a test, I can tell you about strikes, balls, and foul lines. You may think I know baseball because a simple test of rules doesn’t foster our conversation afterward. Once you start talking about baseball like a true fan, as someone who truly follows the game, I’ll be exposed in about 20 seconds. If you discuss specific players and their performance on certain teams, you will see clearly I don’t actually follow baseball. I only know some of the rules.
Coding interviews are the same way. If your knowledge is only surface deep, that becomes apparent very quickly. Not because I asked you tricky questions, but because once we discuss your creative solution, we can all smell fellow fans. Now a candidate saying “I don’t know” occasionally is totally acceptable (and certainly preferable to confident wrong answers!). However, a candidate who can’t chat about the merits of their solution with some excitement isn’t a true fan. That’s why the test must be designed to build a framework for our conversation, not to measure mere head knowledge.
2. Make it Real
Allow the candidate to select the technology and approach. Their choices speak volumes. If you’re looking for a JavaScript developer and they build a server rendered solution, that’s worth a conversation. Allow the candidate to work in their IDE of choice. Let them Google for things like they would in real life. Make it real. Coding on a whiteboard or via pencil isn’t real. It elevates stress and risks alienating otherwise great candidates.
This is my IDE for the next hour? Why? That’s not real.
Instead, consider pair programming with the candidate on an actual problem and you’ll actually learn more. You don’t need to try to stump her. Instead, treat her as a co-worker who is helping you solve a problem. Along the way you’ll see how she uses the IDE, what she searches for, and where she struggles.
If you create a standardized test, make sure it is relevant to the job. For instance, if you’re building web apps for the medical industry, the coding test should solve a medical related problem using a web app. A good coding test is a small preview of what a day on the job is like. Both the candidate and the interviewer should walk away with a good sense of whether they’d enjoy being co-workers.
3. No Right Answer
It’s easy to create a simple coding test that asks the candidate to reverse strings, calculate a fibonacci number, or print fizzbuzz. But as you can see from these links, they’re worthless questions. They don’t foster conversation, they don’t represent real problems we solve on the job, and they’re trivial to Google.
Sure, you can forbid searching the web during the test, but remember, a good coding test strives to mimic real life. Banning Google during a coding interview is like interviewing a cook and not letting him open a recipe book. Silly. Instead, a good coding interview should be broad enough that is has no “right answer”. This way the solution isn’t easily found online, and the merits of the approach selected can be discussed.
That’s it. Not so hard.
What did I forget? Have a coding interview approach or horror story to share? Chime in on Reddit.
These are some of the same conclusions I’ve come to on coding interviews recently, but this is the first time I’ve seen them crystallized into a blog post and it’s great to see independent confirmation of my thoughts. I will probably be doing a coding interview soon and will be following this advice. Great post.
I’ve also done plenty of interviews over the years. Besides getting bored from asking a lot of academic questions, “what is the 3 pillars of object oriented technology”.. etc. The candidates have studied these and typical questions and are armed with the answer. Now I usually start with some features in C#, or just .NET get them going and see how far their knowledge goes. Then I get them doing some hands on coding in Visual Studio.
I know you mentioned that reverse a string is not worth it, but a lot of developers have struggled on those simple examples. I do like the idea of letting the candidate choose how they would solve a problem, and let them decide. That is really what I want to see.
Thanks!
I don’t think the FizzBuzz or other “tests” you mentioned actually qualify as coding tests. They’re just negative filters to get rid of the people with lackluster skills. If you actually want to measure the knowledge and skill level of potential candidates, you do that with actual coding challenges. Most software companies either have some kind of coding test platform or use a third-party service like testdome.com
Although they’re just one step in the longer screening process, the usefulness of these coding tests can’t be ignored.