HACKER Q&A
📣 darkLord19

How do you review and validate LLM generated code?


It's become very fast to generate code nowadays, but how do you review and test all of it?

Every time you ask a model to review some piece of code, it comes back with different "findings". So, you still have to filter through the noise of their findings to get to actual defects.

And if you try to do the full review manually, then it might take more time to just understand the code than writing it yourself from the beginning.

As for testing, you can ask the LLM to write the tests, but you can't be sure if the tests actually assert the expected product flow or just write pass/fail tests for a piece of logic.

I would like to know if you have any workflows, strategies, or tricks.


  👤 verdverm Accepted Answer ✓
1. I don't have the agents write the code handsfree, I'm involved in the process so I know what's actually there

2. review/validate, yes, this still requires you getting into the guts, ai does not write good code, HITL is very very important

3. test coverage reports help improve tests, separate test writing from code writing so they do not modify the code to pass the test or make writing them easier. This is a place where I believe Go really shines in the agentic era, not that it didn't shine to me before, it was designed in the service of software engineering and why it lacks fancy PL features (or lacked for so long)

4. Review, yeah, that's a thing, I actually think it good. You have to put work into your PR review agent. It should see all the prior commentary from the PR along with the diff. Tell it to summarize those which have remained unaddressed in a single comment, to ignore defects/comments which have already been resolved. Using different models and finding different defects is a good thing imo


👤 fuzzfactor
If you can't at least validate your code as "easily" and thoroughly as you could before, I would think you have problems that you never had before.