What's your experience with AI-based code review tools?
I've seen a lot of AI code review tools hitting the market, and I'm curious to hear from people who have tried them out.
Do they speed up reviews? How?
Do you still need human reviewers, or can you directly land code that the AI says passes?
Do they lead to higher quality code? Do they catch important enough things that the review is worth it, or is it mostly minor things?
A lot of value in code reviews comes from knowledge sharing and collaborating on code. It is close to a pair programming exercise that many people in the team can partake in asynchronously.
I’m sorry for not answering many of your questions, but I do think it’s not worth it, at least in a professional context.
For personal use, it might be different.
In my experience, automated AI review can definitely find and identify bad patterns or practices in code or popular SDKs, but our company's tech stack is highly distributed and there is a lot of business domain logic that the AI simply can't really predict (at least not with the current generation) how new code may interact with it on a macro level.
Which code review tools are people using?
I think it needs to know every conversation that happened in the company to provide meaningful feedback that is above generic coding suggestions. But then it makes onboarding new people and getting them familiar with the project much easier. Businesses will have a lot of incentives to do that.
It helps catch the most obvious bugs. Like the algorithmic bugs. If the comment string says something and the code does not do that, that's caught fine. But it is not so good at catching domain related bugs. I think that's to be expected because these AI-tools don't have the knowledge of the domain and the hidden assumptions that come from experience in a domain. It could improve in future if the AI-based tools are trained on a lot of design documents + code that solve domain problems.
It’s more fun to make your own ai linter then to buy a service.