ChatGPT can solve my interview questions, now what?

Question

As an interviewer conducting remote interviews, I fear ChatGPT is going to enable cheating to a degree I won't be able to detect. Right now it can solve many of the questions I usually ask, including combined system design / coding questions, getting it right mostly on the first go.Are software interviews going to disappear? Do we adapt somehow, find the holes that ChatGPT can't fill, yet, and play a cat-and-mouse game with OpenAI? Will complex, multi-hour homework interviews be the only solution (where you could use ChatGPT and we'll take that into account, giving more complex and hard to solve questions)?Is this even a legitimate fear? Maybe ChatGPT is just another tool in the toolbox now, just like Google and docs?

mudrockbestgirl · Accepted Answer

If ChatGPT can solve your questions what does that say about your questions? That they don't assess the skillet you care about. If your questions were useful and actually tested for the skillet you cared about and ChatGPT solves them, why not just use ChatGPT instead of hiring someone? By your definition, it would make a great employee!So what's the problem? Your interview process.

bironran · Answer

This is what ChatGPT has to say about that:
>> While large language models may be able to answer some of the questions in an interview, they do not have the ability to demonstrate problem-solving skills or adaptability to new technologies and frameworks. These skills are essential in a software development role and cannot be measured through a simple recall of answers or code snippets.
>> Therefore, it is important for the interviewer to design an assessment that focuses on evaluating these skills and abilities, rather than relying solely on the ability of the interviewee to recall specific answers. This can be done through a combination of open-ended questions, coding challenges, and group discussions that require the interviewee to demonstrate their understanding of fundamental concepts and apply them to solve real-world problems.

mdcds · Answer

I'm heavily involved in interviewing at my current org and have been pondering same questions.

Some ideas:

  1. use ChatGPT (or similar) to test your questions. I read that GPT-like models cannot do abstract manipulation. Perhaps there is a way to phrase follow up questions such that the model cannot relate to the earlier question. 
  
  2. in system design session, focus on detail instead of high level. Also, you can draw a diagram of a design and ask the candidate to point out deficiencies.

  3. what if we could use GPT-like system to generate new programming questions of equal difficulty for each new interview candidate? this would solve the problem of questions leaking and making it to sites like LeetCode.

usgroup · Answer

Well semantics is the gigantic hole ChatGPT can't fill yet .. so make your questions less like "when do you use a decision tree" and more like "fit a decision tree to this data and discuss if its appropriate"

jstx1 · Answer

If you don't interiew candidates in person then your cheating concern is real but that's been the case before ChatGPT, although it's now perhaps easier to cheat than before.
> Are software interviews going to disappear?
No because you still need to evaluate candidates, that hasn't changed. And the multi-hour homework you're suggesting sounds easier to cheat than a live interview.

wdiamond · Answer

everyone knows that interview dont cover all of anything, its more like an interactive ad. neither multi-hour homework that can also be cheated. work is important in imaginary field, in concrete field anything less than a Ph.D paper is just a copy paste of an Ph.D from the past.

jsjsof · Answer

Now maybe you realize why hiring is broken.

quickthrower2 · Answer

5 whys