It could be adding a feature to a side project, starting something new, or contributing to open source. (Or, we have some stock ideas if people need inspiration.) Rather than asking them to solve a problem they had never heard of before or use a codebase they're not familiar with, I get so much more out of watching them work in their own environment.
I can ask them questions about why they're doing something, and they tend to have much more detailed answers because they've been thinking about it for weeks. They're solving a problem they care about in a codebase they know, which mimics how working with them will be a few months in.
We can talk about tradeoffs and design decision, and I get a real sense for how they think. Plus, I've found most people are so much more comfortable and excited.
(If you're interested, I wrote a blog post about all the ways I've designed how we interview, to give everyone the chance to show off the best version of themselves: https://blog.readme.com/designing-a-candidate-focused-interv... And of course, I'm hiring!)
You can find all my past articles[1], and I'm happy to find you specific ones. I think, given that you can't switch away from algorithmic interviews, at least make the interviews more collaborative, define your expectations up front, and generally find ways to help the candidate show their strengths.
[0] https://hiringfor.tech/2020/02/10/false-positives-and-false-...
That means that regardless of the format, interviews are always going to be inherently unfair. Even the slightest hint of incompetence will nuke your chances. Assessing a person you’ve never met is just an inherently noisy process, and there’s always going to be false negatives. And almost every company in the world would much rather be safe than fair. You probably are having a bad day or misunderstood the question, but employers can’t afford to take that risk.
For the interviewees part, the process is just plain soul draining. You’re literally subjecting your life’s work to the harsh, quick and unfair judgement of people you’ve never met. You’re putting yourself in a position of extreme vulnerability, and it’s easy to feel like a piece of meat.
I don’t really see how twiddling with the format will change the fundamental dynamics. The only thing I can say is that both sides should have more understanding and sympathy for the other.
Candidates, don’t take any single rejection too personally. Remember the interviewer is a person, who also can be having a shitty day or make mistakes. Interviewers be kind and positive and give constructive feedback, even for the clear no’s.
If you really hate interviewing, then spend more time networking. Hiring personal contacts and recommendations is much less noisy/risky, and therefore an overall less adversarial process.
We provide a candidate a piece of production code with some slight tweaks made that will break it in (common) edge cases. Further, we give a test harness that already supplies some basic tests for which it will work. What we're looking for is to see if the candidate can grok the provided code (< 30 lines for a 2hr interview) and reason through ways it could break. Writing the tests insures that the candidate can _actually_ write the code and shows any areas they might be less focused on or miss.
Compared to our old procedure of writing the (provided) code snippet this has been a far more successful approach. In 2 hours I learn more about an applicant than through the entirety of the previous process. An added benefit: this workflow is representative of real work - you'll need to track down code you likely didn't write and implement new functionality, or fix something that's broken. Doing this in an interview is a near direct translation to the job (minus understanding the whole codebase).
I'd strongly recommend this unit testing based process and it can definitely be tweaked to fit your own scenario!
Provide candidates a small sample "project" of 50-100 lines, a Flask webapp perhaps. Work with them to get it running on whatever interview machine they're using. Have them extend it with some fairly trivial feature, around FizzBuzz complexity but with a bit of a design component. These questions should be carefully designed and standardized across interviews.
Key point: completely strip out identifying information from their solution and pass it to someone else for grading. The person "giving" the question is a proctor, not a judge, and they're officially on the candidate's side. Possibly add a side dish of talky conceptual, design, and behavioral questions, with as much blinding in the assessment of the answers as possible, though that's harder if it's a dialogue... Hmm.
A good interview is fun. You get a nice bite-sized problem to solve with someone to bounce ideas off of, and maybe learn something in the process. I'd suggest almost a game level-design approach to coding problems. Make interviewing fun again.
All excuses for why this won't work can be answered with: then make a better test.
As a species, across all cultures, we tell ourselves that the human we're after -- the student, the employee, the promotee -- can't possibly be selected by test alone.
When in fact, if we can't write a test to select the qualified humans, then either we're too lazy to write one, or, more likely, we actually want to leave plenty of room for human bias to do the actual selecting.
And this is ok because we have a special power: we can judge the value of every human, and its future likelihood for success, with a single conversation. If we weren't in a tech company, we could make a very good living reading palms.
We never hire people who don't pass our wonderfully fuzzy exams, so we have no evidence that we're selecting the best people or not. No worries, though, our palm reading is very, very accurate.
The way we look at it is like this. We make a test no one can pass. We always have one more question or one more "level" that there's not enough time for. Because when all candidates fail, we have to fall back on our palm reading, which is just how we like it.
Power and privilege are precious resources to us. We give them out to those most likely to reciprocate. That's what we're poking for with our "culture fit".
In the future, students of our culture will look back on our hiring practices and say, "It was illegal to hire based on race, age, sex, and a million other things, but not beauty??? They didn't start with beauty? And they never realized that beauty needed to be in the mix? I don't understand."
But we understand. It makes perfect sense.
My ideal interview questions are as follows:
* No trick question. Reaching a solution shouldn't involve obscure knowledge or an "aha!" moment. * Only basic data structures and algorithms, the kind that you will realistically use in most day jobs. * Start with a very basic problem, and ask follow up questions with increasing complexity. * Extend the problem in different dimension to probe the candidate in different axis. For example, ask the candidate to design an API for the code, or to run it at massive scale.
It's hard to come up with problems that cover all these points, but you can get close. I find these kind of questions allow me to assess candidate without making the process unfair or overly stressful for them.
Technical talks are never about coding, but more a conversation about previous project and perhaps a little side track on how to tackle large projects.
The style of interviews discused here on HN show an alarmingly low level of trust in the people you want to hire. That’s a bad way to start a relationship.
Writing some algorithm No why should I? If I need an algorithm that must be fast I will look up a peace of code that is used and tested by other people.
No I will not live google in front of you watching me. How is this a real world scenario? How came up with this?
Like stupid questions like: How many log entries should a service have each hour? Yes this was a real question in a job interview.
IT Job interviews are broken in so many ways.
I refuse to live code or have this you now have X minutes to solve a problem stuff. Again this is not how I work and also I have never seen people work like that in the real world. Only in shitty teams with shitty organization and where they were thinking that they were agile because the sprint ends in 5 minutes...
Here is the only way of interview I tolerate these days: 1) (optional) Have a call with HR
2) Talk with people from the team in an open way. Talk about real challenges and how the applicant can help in the challenges of the team. Check if the applicant will fit your team! Good team chemistry eats technical knowledge by 1000x. If you only take people because they know some algorithm you will most likely end up with something in Germany they call "fachidioten". They will solve the problem on a technical level but will bring with them 1000 other social, product and long term problems with them.
3) If you still not convinced that this person is good enough for your team then. Give that person a small challenge he/she can do at at any time she/he wants.
4) Let the applicant send you the solution and if you think it is okay then talk about the solution of the applicant and see how he/she reacts to criticism. Again a team that has good social skills and likes each other will eat every team were you have rock stars that think they are the best.
I have hired dozens of engineers and have never failed do find great hires.
What I mostly do is just a conversation, which I call “geek conversation”, just like when you meet people at meet-ups or tech conventions.
With a simple talk I can first make the person relax, this allows me to have a great impression about her/his personality, passions, interests, personal roadmap, attitude towards problem solving and etc. Whenever we ponder on a more technical subject I throw some questions that might help me measure their technical level a bit, but what I care to measure is mostly awareness on things, nowadays software engineers need to be, above all, great at researching, for most roles they must have a broader overall knowledge and be able to creatively join the dots with a ton of pragmatism. I don’t care if they are able to implementing Dijkstra's or to modify a Trie in front of me, although having awareness of what problem these things solve is a good indication of some seniority.
These interviews take a lot more from me, I have to properly learn about the candidate and the role before I enter the interview and that requires people skills and real interest in the product, and of course I am not able to interview 40 candidates to find just “the best one” (which is ridiculous and something I actually don’t believe actually exists).
Yes there are roles which require the extreme algorithm coding skill, but those are a minority and it is unfortunate if your company is treating every candidate as if they were going to do that.
My "FizzBuzz" coding question that I ask on phone screens is "Write a function that takes a string and returns a list containing the most frequent character." It has a few clarifications that are needed and I expect the person to ask about (What if there is no most frequent character like in an empty string (return an empty list), multiple characters equally frequent (each equally frequent character should be in the returned list once), etc) and has some basic logic requirements.
I am continuously astounded when people can't do this. I encourage them to use the language they're most comfortable with. I give them time to think about the problem, encourage them to ask questions etc. If they seem stuck I'll ask them to walk me through what they're thinking about, etc.
I've had multiple people with master's degrees in CS or with years of professional programming experience get stumped by this problem and it just makes zero sense to me. I took undergraduate CS courses and this would've been an easy problem in those classes. I was a TA in an undergraduate intro to programming course and I would've expected all of my students to be able to solve this problem in similar circumstances.
It seems to me like someone with a Master's degree in math has come in and I'm asking them to "solve for x" in the kind of equation you might give to an eighth grader and they're unable to do it.
For senior engineers I would say it's all about design/architectural problems and talking generally about concepts and taste to gauge fit.
Mid-level engineers I think the take-home is king. A scoped problem, 1-2 hours at most that can be followed up with an interview talking about an extending their solution is pretty bulletproof. You get to understand what tools that are comfortable with, practices around source-control, testing etc that you would expect from an experienced but not overly senior engineer.
Finally for entry level candidates I don't really know. Personally I hire entry level people on gut feel. Algorithms and datastructures just tell me if they a) went to university and b) paid attention. Seeing as I did neither of those things and still write IMO decent software that seems like a hypocritical measure. So I stick with how I feel about them, their level of enthusiasm, how well they researched the position, what stuff they have played with etc.
Hire them, give them a week to catch up, and their off; or wait months to find a perfect match? Any developer with a few years experience knows how to pick up the next piece of technology, the same way everybody else does: read docs, search blogs and forums, ask StackOverflow, etc.
Some people can. Some people can't.
The secret to a good hire is probably to exclude people from doing the interviewing who are essentially poor judges of character.
In other words let's stop trying to auto filter the candidates and instead let's filter the interviewers. Pick good interviewers.
Let those firms that play sunk cost Frat games get the sort of people who will work all hours for next to no pay. Those people always burn out quickly anyway. Instead treat the interviewees' time with great respect and you will get better long term people anyway.
I suspect saying that interviews are an hour's chat and then a decision will get you a set of outstanding interviewees who are just too busy to be bothered with anything else. And they will stand out from the crowd even on paper.
When hiring what works for me is looking at past experience. If someone says they are a cpp performance guy, that creates an expectation of what they should be able to chat about. If you're making it up, you will run out of stuff to say, and I'm patient enough to let people keep talking. Hopefully I'll learn something too, it happens often with top pros. But basically if you're experienced in a field it should be hard to come up with anything where you draw a total blank. If that's the case you're pretty good in my book, the main differentiator after that is whether you've done exactly what the firm needs or just something similar.
With entry level people it's hard. That's why they typically make less money, it's an Akerlof lemon cut due to the being a lot of people who just aren't going to be good at it. This isn't even necessarily a coding thing, but general life maturity. We had a guy who couldn't wake up on time to get to work, and he couldn't phone in when he was late either. I reckon there's no way to know about this kind of attitude problem until you hire them. This is also why you can expect a relatively high jump in pay after your first few years, it's proven that you aren't one of the lemons. The distribution is a large mass of hard to differentiate professionals, plus some duds of near zero value.
So for entry level, your best bet is simply finding someone you think is smart. You'll end up falling back to the same thing as everyone else who is hiring: the prestige of the uni they went to, and some pot luck questions to make sure they picked up something. However that's really not enough to know what you want to know, which is whether they are able and willing to learn what you need.
45 minutes - do code review of this extremely buggy code like you'd review a coworker's code
45 minutes - presentation on a technical topic to one or more team members
45 minutes - behavioral. Let's talk about failures, conflicts in your career. Who are your role models? What do they do well? Leadership abilities, career progression
45 minutes - troubleshooting. Here's a docker container. Make the service inside it work
If you were a team lead tomorrow, how would you go about hiring your coworkers?
Honestly, that should tell you everything you need to know about:
a) What kind of technical skills they value (i.e. will work to improve in themselves if necessary)
b) Their interaction style with coworkers.
Sure, this interview can probably be gamed, but IMO a lot less than your typical "reverse a linked list" style interview.
That works best for programmers with entry to mid-level. In Senior level you need to focus more on problem solving.
Most articles complain about the status quo but don't offer any practical advice. When they do, it is some approach that may not be an option on most cases - e.g. 40 minute interview followed by a 90 day probation period.
Live coding interview can be pretty tough for candidates, but we try to make a lit bit less difficult by:
- Giving them time and resources to prepare.
- Letting them use their preferred language and IDE/editor.
But since you have no control over format or time, maybe try to:
- Use reasonable problems - to assess skills people will actually need on the job. In our case it usually boils down to using comodity structures like a dictionary or map to do simple tasks efficiently.
- Use problems that start simple but can be extended. This is good because when candidates finish the first part, they get a boost in confidence. It is also nice because if they freeze, you can help them finish the current "level" and still have material to assess other skills.
- Set them up for success during interview:
- Reserve the first 5~10 minutes for introductions, ice-breakers or questions about the interview.
- Reserve the last 5~10 minutes for the candidate to ask any question about the company or the hiring process.
- Make clear that is OK to ask any question.
- Help them on small blockers.
- Be friendly.
Some links/references:
[1]: https://lawler.io/scrivings/erics-guide-to-hiring-software-d...
[2]: https://medium.com/@alexallain/what-ive-learned-interviewing...
[3]: https://www.holloway.com/g/technical-recruiting-hiring/about
Rather than saying how you are constrained, what freedom do you have?
Of course, this requires knowing what skills & knowledge are needed for the role - you may need to review the job description or push on the hiring manager to quantify them.
When asking problem solving or coding questions, work with the candidate as if you were two teammates solving the problem together - don't be adversarial, don't play "gotcha!", don't try to show off or prove yourself smarter.
If you have an awesome candidate, they will be smarter/more skilled/more experienced/more knowledgeable than you in some technical areas. That's a good thing!
- algorithms
- experience with a specific technology
- getting things done
- communication in "not being a dick" sense
- generally "being smart"
I want a person to score "B" in all of them and "A" in one.
It's important to realize that you don't need all A's: it's good if you manage to find an all-round A-student, but it's rarely possible and takes huge amount of time.
One "C" may be allowed, but then I'd probably want more than one "A".
So I keep my questions simple. For algorithms, for example, I ask to write a bubble sort. If a person can do it - it's a firm "B".
1. Hire based on the resume, gut check on the interview. Is it plausible that they played the role that they said they did? If you were a major contributor to an RPC framework in C, but you can't walk a linked list... Maybe your resume is too inflated. This means that the questions are basic because their purpose is not to rank candidates but to ensure the validity of the resume.
2. Focus on reading over writing. I would rather a candidate who can read code someone else wrote, articulate what it does, and maybe plan for how to extend it than someone who can pretend to invent something new.
I generally take a look at their resume and then do some research about the tools they've used in advance. Then, during the interview, I ask about what they like/don't like/find interesting about those tools. The goal is absolutely not to gotcha them, but instead to find out what they're interested in in that space. If I ask a question that it becomes clear they've lied/fabricated about on their resume, I say something to the effect of "No worries" and change the subject.
Depending on the role, you need more info than just what languages/frameworks they've used. For more senior roles, or roles that involve architecture/cloud functionality, I'd ask about how they've built systems in the past. If they call out AWS, I ask about what resources they've used, how, and why. If you've written down DynamoDB but cannot speak intelligently about access patterns or secondary indexing, it's kinda clear that you just used a system someone else defined. Whether or not that's a problem depends on what role they're applying for. If they can speak intelligently about how they got to a specific DynamoDB structure, they probably are being honest enough about their experience. Note, it needs to be clear that the candidate is not speaking in the abstract, but about things they've actually done. Googling stuff is easy, finding the weird parts of tech in practice is hard.
Ultimately I want them to feel comfortable enough to get chatty about development. Usually I find out enough about their skills while they're chatting - I think most would be surprised to find how clearly you can understand a person's abilities without directly asking about them. You just kinda have to spend some time up front learning pros/cons/common pitfalls of the tech on their resume.
There's always a ton of posts, but I have never had a pleasant or unbiased interview experience in my life.
It shows to the interviewer how a given person works in a programming job, how they researched solutions to the problems they were faced with in the past etc.; as opposed to the usual exercises that have nothing to do with the job and won't really show you how well they will perform.
It's also less confrontational which should put the interviewee at more ease.
So what I started was a local library program (pre-covid) that I would attend before / after work to be around people who are doing career switch to coding.
Then when students learn enough and have strong foundations that I look for, I recruit them to work on open source side projects that benefit new students who are learning.
For the students who have shown the maturity to build and maintain production features, I refer them to the company I work at and they interview and (hopefully get in).
In the past 2 years, I've referred 5 students successfully and they're all doing pretty really well (I secretly stalk their internal code contributions).
I hope that one day, I could have enough reputation at my company to hire engineers based on their open source contributions. I also hope that the stock market doesn't crash next year because I plan to use my RSUs to start a fund to hire junior engineers students so they don't worry about medical benefits while they contribute to open source projects and have a hard time finding a job.
If you mentor a student as they are learning, you can instill good engineering practices: like taking the time to communicate and write good documentation, write code and comments to make life easy for the next engineer to take over, how to handle tradeoffs between time and quality, etc. These are critical engineering skills that are very hard to test in 1 hours but are very effective if mentored early on.
I find it astonishing how many developers absolutely cannot think through a tree model. I deal with tree models absolutely every day in programming, management, and personal concerns. For example: the DOM, file system, company/employee organization, outlines, security models, business/customer needs, and so forth. All these things are tree graphs and I encounter many of these EVERYDAY.
Many developers simply cannot explain walking from one point on the tree to another point. I am not even talking about code. Even in casual conversation I am astonished at how many times I have encountered people like lawyers, QA, bosses, and others who can engage this sort of logic while some of my developer peers are just lost on the subject.
That is absolutely something I tested candidates for as a filter and will continue to do so. If a person even talking through such a real world common form of abstract logic how can I expect them to write code? It’s like they expect some magic wand to simply provide the logic for them and just sprinkle on a little bit of syntax.
1. Tell me briefly about your professional development, including challenges you have faced, mistakes you have made, how you overcame them and what you have learned. Have there been any critical mentors or resources? What motivated you in each position and how did that lead to your current application.
Opens a floodgate of possibilities for discussion, generally gives a sense of career pattern or path, potentially personality.
2. What are you learning right now and why?
Screens for curiosity and/or goal orientation.
3. Choose any recent technical or business idea you have had and explain why you considered it to be interesting.
Possible insight in to how prolific their idea generation may be, and whether it is circumscribed by professional domain or not.
With a bit of two-way questioning on these, you should have a basic sense of their capacity as a communicator as well as personality.
Important: Predominantly co-workers and direct managers should interview, not random HR departments, external contractors or disconnected tiers of management.
We work on fixing that, so we implement a process: founders pick some personality traits they find essential for a great hire, we prepare an interview playbook, and they can be certain that the new guy is actually what they are hoping for.
Interviewing for personality (cultural fit) is hard because we (humans) are hardwired to survive so we give answers that we think are in our favour - and not necessarily truthful. Key to our approach is that we use falsifiable questions, meaning that you can't guess what I believe is a right answer.
If this helps any fellow founders here, am happy to lend a hand.
Now pick some arbitrary rating system (5 stars, school grades, 1-10 points, whatever). It does not matter that much which one you pick. The hard part is to stay consistent about it over time.
Now, here is the way how to make it more pleasant for your candidates. After 3/4 of the time, write down your rating and discuss it with the candidate. E.g. "I gave you four stars for cultural fit. I get good vibes with you but you would be the only PhD in our startup, so you are probably more academic than everybody else." Give them a chance to correct your rating. "Oh, I forgot to mention I'm involved in my brothers construction startup. I feel fine in that non-academic world."
Giving such honest and clear feedback to candidates is a better experience than most companies.
A lot of interviewers get lost with these obscure algorithms, and forget what they're actually trying to test: the candidate's thought process.
A counterpoint on why I wouldn't ask anyone to implement something (such as a web server) using a framework: They're ever-changing. Furthermore, if I were to try and see if candidates understand building an API well, I'd rather ask them a mix of networking and open-ended, system design style questions.
How would you design/write xyz
What's wrong with this code [code] (bug, security issue, bad code)
What do you think about
Use multiple (2-4) recruiters paid on contingency. Receive about 5-20 resumes a week from them (total, not per recruiter!)
Filter these resumes to about 3 per week. Interview these three yourself for 30 mins each. Just ask them for more details about technical experience on their resumes to detect liars. If your recruiters are any good they will have filtered out people based on the soft skills / career questions.
Reject the liars and accept just one of the honest people (if there are any). Push the one forward to your team. Let your team loose on the candidate with any bizarre (but legal) questions they like. Give them the final say on the hire. Repeat until position is filled.
I never focus on algorithms or particular technologies. When I interview someone, I'm looking for two things:
- First and most important: we both are in the same wave. I want to make sure the applicant understands what's the goal of the position. For example, once a guy with over 10+ of experience applied for a junior position but demanding responsibilities and salary of a senior.
- Second, expose a general problem and work with the applicant to get an idea about what can we do. I don't expect a solution in 5 minutes, I'm looking for ideas of how we can tackle this problem and how we would start working on it.
2 what would be a moonshot feature for P? (If you had a perfect AI lib or unlimited processing power or devices spread all over the world)
3 if you had to find someone to take over your job (for you to manage the new one), what would you look for?
-Don't ask questions designed specifically to gauge how the candidate 'thinks'. If you do pose such a question have another interview with the candidate some days later so that the candidate can think though various solutions.
- Don't ask questions which are cliched like 1) Tell me about yourself 2) Tell me your strengths and weaknesses
-Don't continue interviewing a candidate if you have decided not to hire him/her.
What you can do:
- Ask them to look up an answers using google/search
- Paid assignments
You discuss different architecture options, feature trade offs...
It's easier for both parts to connect, it will give you a glimpse of how the candidate faces requirements from other teams and you give him a taste of the inner workings.
It was an easy assignment that covered a lot of concepts, and it was followed by a 2 hour techinical discussion on it and all things related and unrelated with my future colleagues.
Outside of that the interview stuff was just around personality and culture fit.
The algo interviews are partially to test can you do the stuff. If you have done similar and can describe it in sufficient detail, then you know it.
First couple of rules:
- quick turnaround. We want to give an answer to a candidate within a week they engage us. If we make an offer, we give them plenty time to sign the offer, even shop around, before they make the decision
- no step can be repeated or returned to. If a candidate progressed from step 2 to 3, our hiring managers are not allowed to go back to the previous step
- we disclose as much information as possible early so there are no surprises
Here is the process:
1) Initial 30 min call with one of our recruiters. We cover mostly the company, the compensation package for the role (usually range based on the level they fit in) and our expectations. We also cover the culture and share as much as we can so the candidate can make informed decision if this is a right fit. We ask some high level questions, but usually focus on the company itself.
2) Take home challenge. We have one for every type of position, from engineering to marketing. They are usually simple and fairly arbitrary problems that illustrate candidate's thinking about problem, how they analyze it, what tools they choose to solve it, etc. When we design the challenges, we are aiming for it to take around 30 minutes. We are not trying to get the person to any significant work for us, rather just figure out how would they solve problem if it was given to them at their position.
3) The candidate meets 4 people:
- their manager where they cover the role, the expectations, the growth at the company and anything related to their job
- random colleague. Essentially someone they will not work directly with. We use this person to help us to focus on the candidate themselves rather than their skills
- our COO, who covers with them more high level topics like what they did/didn't like about previous employments, what they care about in working environment and such
- our CEO (me). I usually focus on their individualities, especially passions and hobbies. I want to see that they care about something, anything. I also answer any unanswered questions from vision, company's cash position, customers, how we generally work, etc
As I mentioned, we try to get an offer to candidate's hands within a week. With this process we get around 95% offer acceptance rate.
As an interviewer, I don't really care if a candidate happened to know the answer to an algorithmic puzzle or has a deep knowledge of academic minutia. I want to know three things: can this person reliably and meaningfully solve technical problems, can they communicate their findings to the team, and finally – do they strive to learn the details or do they just skim/superficially solve problems? Obviously expertise in a job-relevant domain is great too - but I'd consider it a bonus; the difference between a senior and a junior/mid-level hire.
1a. Optional coding test at home (no time limit)
1b. Optional coding test in front of me (1hr time limit)
2. Very simple fizzbuzz-level problem. Essentially can you break down a problem with an if statement and while loop. This has the least bearing on the process.
3. System architecture/design "whiteboard". We ask a question that's essentially an entirely different problem space from out company but who's end system design, and lifecycle, looks almost exactly as the design our company took. Essentially if you `s/A/B/g` you'll have a 1:1 idea of what our company's infrastructure looks like by the end of the problem. There are absolutely no wrong answers, just answer how you would if you were given this ticket ("design X") and I'll ask from time to time "how many QPS can this handle", "any bottlenecks you can identify", "how much bandwidth/cpu/memory will this take", etc to see if they understand the implications of their decisions.
Surprising facts:
1. Most people can do the take home coding test "enough" but do horribly on the on the phone coding test. This has been shocking to me because I'm not someone who struggles with this aspect of the interview process (I'm horrible at explaining myself).
2. When we get into the system arch/design session 99% of people have no idea how to proceed. They've never sketched out the design for something in front of someone else and have often never thought about latency, memory, CPU, disk, etc. We had one engineer take it who couldn't even come up with a way to estimate how many QPS a service might receive in the worst case. Another engineer settled on a design that was going to cost >$1mill/month in AWS DB costs and could not imagine any way to reduce this bill and thought this was reasonable (just needed to put a cache node at an edge location).
3. People use buzz words. They use them a lot. If I had a working feature for every time I've heard someone just say "autoscale it" whenever they hit a bottleneck during the arch/design section I wouldn't need to hire an engineers. It shows a core lack of understanding of where the bottlenecks in our interview question are coming from.
4. We see some (small) percentage give up on doing the coding test. I'd say it's less than 10% of inbound leads. Obviously a significant portion of people don't want to code in front of someone and don't want to code at home. Unfortunately I can't advocate for hiring someone until I see code that they've written. We've had horrible experiences hiring "senior" engineers who actually don't know how to program.
5. I've had people freak out, and start cursing at me, when we get to an interview question they get stuck on. It's something I'd never have imagined happening in a professional setting but, at least, they've been filtered and won't be working with me in the future!
6. Resumes and pedigree have the worst correlation to quality of the engineer I've ever seen. I've seen undergrads working part time in the summer output 10x value to full time engineers with 5+ years of experience. "10 years of experience or 1 year 10 times" is the most accurate pattern to describe this.
It allows your company to learn, to get a feel for the person's ability to communicate, and to stay away from stupid things like tests.
It's also easily the best interviewing process I've ever experienced.
---
edit: Since it appears that multiple people have taken this to represent an hour+ long presentation, let me clarify here.
A better way to think about this would be a long-form discussion about a topic. The initial "talk" would be 5-10 minutes, with a Q&A afterwards to ask for clarifying questions.
I've been working in this field for over 25 years. I spent roughly 30 minutes of prep for the interview and it was easily the best interview process I've ever experienced.