After replying to this tweet from Heba, I have a strong urge to write this blog because I feel that job interviews for software engineer positions can be improved. I am sharing some of my experiences, and observations.
Puzzles
In almost every interview, candidates are required to solve programming puzzles, and write code for them. This part of the interview process is typically the most difficult one. In order to do well in this area, candidates can engage professional coaches and instructors. Some people spend many hours/weeks/months on solving problems listed in online programming platforms such leetcode.
Coaching and training are needed because these puzzles are tricky and they require a lot of thinking. During the interviews, the candidates are given limited time to find good solutions for these puzzles. Hence, it can be extremely stressful, and they do not perform well. In my opinion, the ability to solve these puzzles does not tell us if the candidates are good programmers or not. A candidate may be able to solve the puzzle because (s)he has encountered the same kind of puzzle in the past, or (s)he is coached by professional instructors. Therefore, the candidate may not be a good programmer.
I am not against solving tricky puzzles during interviews. Actually, I am for it. The interview can be structured in a way that the interviewer and interviewee work together to solve the puzzle. The interviewer provides hints if needed, and asks questions around the puzzle. For example, what is the complexity of the problem? what about using dynamic programming? what happen if we run out of heap space? Once the candidate has the solution (it is alright if interviewer has to drop a few hints which lead to the solution), candidate writes the code and tests. As an interviewer, I look for good coding practices and adequate tests. The candidate’s ability to solve the tricky puzzle is secondary to me.
Many years ago, I was a member of an interviewing panel. And, team members were sending out programming puzzles. I found that one of the puzzle is extremely hard. we could solve and implement it in 30 – 45 minutes. The person who shared the puzzle, mentioned that it was supposed to be hard so that we can test the candidate. I do not understand why one asks a puzzle which one cannot even solve.
Domain Knowledge
It is important to understand the domain knowledge that the candidates have. It can be research projects that the candidates are involved in, or code contributing to open source projects. Typically, the candidates can explain in details on their involvements in these projects. There are red flags if candidates are unable to do so. For example, in an interview, I asked a candidate why numpy and pandas are popular in Data Science. The candidate mentioned that he used these open source libraries in the past, and he could not elaborate on their strengths and limitations.
From the resumes, we (as interviewer) can spot topics for discussions during the interviews. It can be topics that we know or do not know. We can educate ourselves on these topics before the interviews and have intelligent discussions with the candidates. Then we determine if the discussions are good, and the depth of the discussions after the interviews are over. I learn much from these discussions, and I am gladful to learn new things from the candidates.
Software Architecture
We have software architecture questions for senior engineers. It is usually solution based. For example, how can we process massive live data?, or what is a cost effective solution for elastic demands?. Most of the time, there are many solutions to this kind of questions. Typically, candidates are required to know how to design, deploy and monitor semi-complex to complex solutions. They must explore different options (with the understandings of their strengths and limitations). When I was interviewing at Microsoft last year, the interviewer and I filled the entire whiteboard with the technologies that we know, from UI, backend services to orchestration and persistence layer. That one hour is the best interviewing experience that I have.
Observations
It is impossible to determine if the candidate fits the position through job interviews. Hiring managers gather all the information and make guesses. Some guesses are good, while others are not. We often label the latter as hiring mistakes or bad hiring decisions. There are traits of a good software engineer that we are unable to uncover during the interview process(es).
Hiring is always not easy, especially in the Bay Area. Hiring managers have to work with recruiters to find the right candidates, pre-screen the candidates, form a interviewing panel, consolidate assessments and decide on the hire. In order to reduce hiring mistakes, Internship, and Employee Referral programs are good ways to find good engineers. Another way is to look at the open source contributions that Heba has alluded to.
Dennis Seah's Blog 