How to Freaking Find Great Developers By Having Them Read Code

When hiring developers, there are many things we are looking for, but over the years I have found that raw coding ability is easily the most important quality to look for.   I can quickly train a person to have knowledge in some domain, but I’ve never seen raw coding ability come from anything other than personal commitment to extensive and deep practice.   Because of this, I have found that some methods work better than others to discover talent.

The Old Way

A typical coding interview starts out something like this:  “Write a function that reverses letter order of words in a string.”  Then for the next half hour or more, the candidate scratches out something on the white board (or a shared text document if they are lucky).    This approach is weak for a number of reasons:

• The same questions get passed around and candidates often memorize answers.   Are you testing their skill or their ability to memorize answers?
• The problems are often “trick” problems, requiring some kind of deep insight to come up with the O(log(n)) solution.   True insight almost never comes to even the smartest candidates in the timeframe of an interview.
• It tilts the power balance in favor of the interviewer.  Who likes having to awkwardly produce code in front of a judge who will determine your professional prospects for the next several years?
• Writing code on a white board or even a text document is unnatural and slow.  Nobody sketches code on a white board or notepad as part of their daily work.   What people really do is sketch code in an IDE with a heavy dose of Google on the side.

A Better Way

So instead of writing code, consider instead having the candidate read existing code and talk about what it does and how it works. This offers some powerful advantages:

1. Reading probes the most fundamental skills.   Reading code is probably 95% of what a developer does as part of their job.  Whether a developer is writing new code, fixing bugs, or creating documentation, they are constantly reading.
   
   What abilities does a coder need to read code well?   There are two important ones: 1)  The ability to remember variables and stack locations and 2) The ability to generalize a piece of code once they understand it.   I can memorize interview coding questions, but I can’t prepare for being plopped into some random code (other than just writing and reading code all the time). It is essentially impossible to fake these skills.
   
2. Reading code is way more efficient than writing.  A candidate can tell you a lot about their programming skill in the first five minutes of reading because reading is easily an order of magnitude faster than writing. In a reading interview, I can skip around half a dozen essential topics in the time it would take the same person to write code to reverse characters in a string.
   
3. Reading puts candidates at ease compared to writing code.  As an interviewer, stress is your enemy because it raises adrenaline which lowers IQ by several points, causing you to miss good candidates.   Candidates prefer reading partly because they are relieved to not have to write code, but also because the interviewer can easily adjust the reading questions to accommodate for the candidate’s skill.   (This adjustment can include writing code if it seems like the candidate wants to.)

How I do this in practice

Here’s some practical points on how I actually do this in interviews:  

• For each fresh interview cycle, I create a set of predict-the-output exercises that start really easy, then get harder.  My current set starts off with a basic function call, then multi-level function calls, then recursion, then side-effects.  These are generally “pretend” functions that are meant to give the candidate some quick success and give me some clues on how the rest of the interview will go.   For more advanced questions, I pull code from stuff I’ve written.   My current “hard” questions explore ability to make abstractions while reading and also following asynchronous operations.   (Some other things that make good reading are unlabeled routines that execute well-known algorithms such as sorts or tree traversals, and finding bugs from error output.)
• Before I use my questions with candidates, I calibrate my questions against people I work with so I have a realistic expectation for how to gauge the skill of the candidates.  Calibrating the questions also helps me refine them and weed out confusing parts.
• At the start of the interview, I explain:
  • I’m NOT testing knowledge of syntax.  Treat me like an AI-enabled google and I will just tell you what some function or operator does.
  • I don’t expect you to finish because nobody does. We will stop after 20 minutes.
  • I don’t expect you to get answers correct.  If the answer is wrong, I would love to see you go back and debug your thinking.  This is just as valuable to me as anything else.
• The test goes like this: 
  • I show a commented line of code that will call some function and return an output. 
  • The candidate reads the code and predicts the output
  • I uncomment the line and run the program so they can see the answer.   
  • If the answer is different than their prediction, they go back and explain why. 
• I give the candidate 20 minutes to get as far as they can.  This gives me extra time to ask follow-on questions.    In the interview report, I write how far they got and what strengths and weaknesses they showed.  

Obviously, these special coding skills are not the only thing to probe for in interviews.   Domain knowledge and cultural fit are important, however I find that reading does a great job weeding out candidates who do not fit where it counts the most.

Some of you might want to know how to develop your skills to do well in such an interview.   My answer is simple:  write lots and lots of code because there is no substitute for regular practice. How to get practice?  The easiest way is to crank up some non-trivial side projects that you care about.   A game, a web site, an app, … whatever.    Target 4-8 hours per week of getting into code you care about and make it something you like to use and are proud to show off.    (Also, it helps to host it somewhere and put the source on github so that future employers can see your activity and get a sense of how you work. ) 

Hope this helps