If you stick to targeted problems with well-described prompts, acceptance criteria, and lots of linting, unit testing, and integration testing, you'll typically get what you want with code that looks okay. And when things start to stray, it's easy to get things back on track.
It's when you start trying to have LLMs write too much without a human review that you start getting unnecessary function chains, abstractions that aren't needed, code that doesn't really match the existing style, duplicate code, missing functionality, hallucinated functionality, tautological tests, etc.
It works best when there's regular feedback in the loop about what's good and what's not good. Testing and linting can fill in some of that, but we still need a human in the loop with "taste", so to speak.
I once had claude code write a python script that emulated the output of my training script, including prentending that loss is decreasing. Why? Because it was unable to install a python dependency.
Everytime I use a coding agent, I need to double check that it's not cutting corners, hard coding things that shouldn't be or straight up rewriting failing test cases.
What I need more is honesty.
I have found it very helpful to ask AI to review the latest changes - it often finds serious problems in review of code it just wrote.
For example, models repeatedly try and not use my Makefile's (build / run / test / test-file / check) and instead spend several cycles attempting to invoke using system tools and getting it wrong.
I've got to the point where I run most models with an extra path folder which overrides typically bad commands tells them off and redirects them back (more effective and portable than hooks). But then sometimes the model reads the makefile, discovers the workaround, and does the stupid anyway.
- Plenty of API hallucination happening on cutting edge Spark (4.0.0+) functionality, especially PySpark. Spark bares some blame here for broken and incomplete documentation. Takes a human in the loop to realize that the documentation is misleading or wrong or missing.
- API design. I’ve found that, unless specifically steered towards “good” API design (highly subjective), agents tend to just add another endpoint / function to satisfy the exact task at hand, with total disregard to how the rest of the API looks. (Pretty much exactly what a junior engineer would do…)It did the above twice in a row around different parts of the API.
The thought that there are almost certainly devs out there merging Claude PRs without the skills or volition to push back on its screwups is not comfortable.
But one big limit is the DX. Their choice of DX is usually abysmal - ironically, just like an average devs. They seem to lack the aesthetic instinct for code, so you have to really point them hard into the direction or provide a sample of the expected DX, for them to still fight against it at every turn.
While understandable in a way, as they are trained on average code and most code will now be written by the machines anyways making the DX "less relevant", it's also a giant code smell, as bad DX tends to point towards bad internals and wrong decisions along the way.
So not really a technical limit - they swallow anything you throw at them, even the most complex cases - but more of an aesthetic limit in terms of taste.
But the last few weeks Opus 4.6 seems to have got dumb again. Now it is making way more mistakes and forgetting useful things and recent context it used to manage.
I am guessing this is just Anthropic quietly dialling down the real effort as they either downsize or free up compute for someone or something else.
Proper escaping of layered syntaxes in Ansible on the first attempt
Writing bare-metal embedded Rust, although this was long ago, so not current models
The reason is that sometimes it spits out something or does a workflow that's pretty sophisticated, and sometimes it fails spectacularly in the most basic ways.
I don't think there is a complexity or domain knowledge limit as there would be with a human. Or at least not in the same sense. As long as it can repeat and remix patterns that it is trained on, then it will do its thing well. The same seems to be true for "reasoning" loops and workflows. It can spit out code that has been done N times before in a similar manner for a large N.
They can still break down because of very trivial issues and assumptions that happen to be baked in, go off the rails and get stuck long loops that are completely insane if you think of them as imitating human programmers.
When I use an agent, I always interview it first about the task. Ask how they would go about it, probe them, give them info that they lack.
Never go from prompt to action. Have them define their approach first, then split the approach into pieces, from gathering data to cleaning it up and so on. If applicable, front-load work that can be achieved with scripts, so you have testable and repeatable steps rather than let it go wild.
So the TLDR is: I think the limitation is simply that it's a non-deterministic token machine that produces useful results enough of the time so it appears to be reasonable.
I tend to re-enjoy handrolling code more. I delegate the stuff that annoys me.
We have a large multiplatform codebase, the issue seems to be more the time it takes to navigate the code and reason about it, rather than the size. Arguably the size is causing 'them' to be slower in that regards, but I haven't found the limit yet. And with compaction, it's even less of a problem.
my 5c.
I had a mistake in which a large back-up file deletion event happened during a robocopy. 600gb of files got 'deleted' (file headers toast etc). Trying to get the LLMs to understand the hunt parameters, what to focus on, what not to focus on - none of them could reasonably come close to doing file content recovery properly. I needed to build a custom solution because the available industry options couldn't do what was required and the LLMs were useless for that (including the latest versions of Claude, Gemini and GPT). They just went around in circles, capped by their apparently weak knowledge of file recovery as a field. That is, creativity was their limitation.
Also any situation where the context window is even remotely close to being full. At 80% the degradation is noticeable enough to make me start from scratch
Eventually I asked it "hey, are you sending two requests when you could send one?" Claude thought about it for a minute and said, "you're right! Let me fix that." The 429 errors stopped.
I've found it really is more like pair programming than having another fully independent developer. For Jenkins pipelines, I don't care about hardcoded string compares as much. For the core capability of the software, details are important.
There's also some types of code that I believe is often wrong in the training data that is almost always wrong in the LLM output as well. Typically anything that should have been a state machine, like auth flows, wizards, etc.
When all is said and done I think the main savings come from the high throughput of low-value generic solutions. I don't currently see this changing, and the reason is that high quality products cannot be generated without specifying a lot of details. Of course, we may not want quality.
I feel like a human would give up a lot quicker and start to learn where the limits are. Claude spins in circles convinced it's finally found a solution. Again. And again. And eventually gets back to where it started.