Why are live interviews higher signal than public GitHub code in hiring?

For most of the hiring funnel you want to evaluate everyone the same way so that you can compare each candidate apples-to-apples and so the hiring team builds expertise in the interview process. This means a standard set of interview questions, coding challenges or take-home projects. Even a 20 line coding challenge can have a lot of variations and after seeing it solved dozens of times you get better at evaluating each candidate's approach.

On the other hand most people's GitHub profile is a mishmash of forks and projects that are either trivial or blank and mostly useless for hiring. Or, maybe the candidate has a serious GitHub presence in which case it might take hours of questioning to even understand what they've done, let alone if they're any good at it. Their activity might be mostly in a natural language you don't understand, and how could you evaluate that fairly?

Also GitHub activity is skewed toward people who have time and energy to create a portfolio for free. Their GitHub portfolio or lack of it might say less about their skill and more about the rest of their life which is not relevant for hiring.

That said if a candidate provides their GitHub or other portfolio then I think you should consider it just like any other part of their resume, not to replace the coding challenges but as part of the chat about past experience.

I have to interact with a person who does a lot more than produce code

GitHub is often used when it's there to build expectations. Interviewers will get to see a candidates resume and I always looked at it before interviews myself.

If the interviewing is inconclusive it's sometimes directly consulted by the decision makers.

More pragmatically. How do you know they wrote the code on their github? Do you expect people who are already working for a company to write even more code when they're not working? How will candidates get a feel for the company themselves? How do you ensure some level of consistency? How would you generate documentation that could be used in any kind of discrimination case to protect yourself? What kind of systemic biases do you think GitHub/portfolio review style interviewing would create. How do you understand a person's disposition without adversity?

I can't say how many candidates I should have passed but didn't, but pretty much every candidate I said yes to performed the job itself pretty proportional to how they performed in the interview.

It might be interesting to estimate how much smaller the candidate pool would be if you tried to evaluate people that way. For what subset of the developer population would any significant quantity of professional work be visible on their personal github account?

Because there's a lot more to being a professional software developer than just writing code.

In my experience, most people involved in hiring are looking for negative signals, not positive ones. One "red flag" outweighs ten "green flags". GitHub allows you to collect green flags, but does not reveal your red flags (only the most anal retentive would red flag you for your personal pet projects' poor code quality). An interviewer's bad feeling in their stomach is to them more important than GitHub.

All other issues aside, you can't expect people's github repos to resemble their professional work.

It's common for people's hobby projects to use libraries and tools they wouldn't normally use, to not bother with tests or documentation they would create for professional work, and so on. They're just sharing something they thought others might find useful or interesting.

Timing. You can take an eternity to produce commits for Github and intermediate work may not be shown.

A live scenario will force time constraints.

How do they know you wrote this code. And how do they know how much you copied, and how much you understand from what you copied.

because they don't want to pay for your experience, and because engineers interviewing engineers quickly results in some kind of "who is smarter" competition, which eventually drives down the cost of hiring... and unfortunately it seems to make it easy for companies to streamline the hiring process even though the end result is that the "wrong" type of engineer is hired.

I have almost 20 years of IP development behind me, none of it is on github, because I paid for it out of my own pocket. I have products in the market. companies i've interviewed with don't want to take any of this into account BUT they would sure love to get their hands on my work, and try to extract tidbits of IP out of me during interviews (even though they do not want to let me talk about my work as an alternative to their ridiculous and irrelevant leetcode questions).

The more senior the candidate is, the less important is how they code. Important things I consider in order:

- can they communicate? Can we talk about tech stuff in a way that we actually understand each other?

- what kind of domain expertise they have? E-commerce? Banking? Cloud? Compilers? ...

- how good they are at building abstractions

- ... a lot of stuff like: not being an asshole, being a team-player, being positive (or realistic), honest...

- how good they code

I do understand that me looking at your github is faster and easier for you than me getting you to problem-solve live, and also potentially more representative of the kind of work you'd actually be doing once employed. From my POV, if I'm hiring someone to solve problems, I'd like to see how they solve a problem; and certainly your github is a record of you solving some problems.

Nevertheless.

If it's a toy problem, I'd rather it was my pick of toy problem than your pick of toy problem. If it's a hard problem or a significant open source contribution, it will take much more time to fairly evaluate than a toy problem, and so there needs to be some form of screening of candidates first because there is only so much time and there are more candidates than that.

In order to evaluate your open source contribution on github, I need to invest considerable time understanding the codebase you are working on and the problems your changes are solving before I can even begin to properly think about your changesets, then prepare myself as I would for a live code review in order to give you a fair hearing; otherwise it's just Elon Musk's "send me your best line of code" all over again. I could enter the interview without the preparation and just let you talk me through things from cold, but then what I am really evaluating is your communication skills, not your problem-solving skills; which may be appropriate for some positions, but does not help me much if what I need is someone to problem-solve.

When there are more candidates than available positions, it then takes even more investment after the interviews to compare candidates to each other, and the results are relatively subjective.

Such a process may be appropriate for some senior or specialised positions with relatively few applicants, but in general does not scale well during We Just Got New Budget So Let's Recruit All The People month.

The live interview with simple questions and toy problems, for all its faults, has the (admittedly dubious) advantage that it is self-contained and that multiple candidates get similar or identical questions, making it easier not only to agree on a shortlist but also to spot potential problems with the interview process itself. One might then go look at github repositories as well as scheduling longer meet-and-greets for the most interesting people to make the final selection.

This isn't great, but choosing whose codebases to properly look at based just on the candidates' CVs is even worse, and that (or even worse methods! - they exist!) is what we are left with if we exclude the phone screen / live interview as a tool from our hiring pipeline.

(Note that writers, when submitting their novels to publishers, also don't throw the entire novel at the publisher from cold (or if they do, they aren't likely to get very far) - they generally have a cover letter with a brief summary, and supply a representative sample for the reviewer to read; different publishers have different rules for these, but generally no more than a chapter. Moreover, many publishers don't accept direct applications, or only accept them during specific, brief, time periods - by requiring the author to get an agent to agree to represent them, they are effectively outsourcing that initial screening step)