So here is the gist of it: Imagine a user-facing software that is out there in the world, doing its thing. Doesn't have to be anything fancy for this example, let's say an inventory management system called Foo, used by several hundred people a day.
Now imagine you set up a kind of "loop" that works like this:
Every 24 hours, a Coding Agent launches, with the following prompt:
"Here is the codebase for application Foo. Over there are all application and system logs that Foo produced over the past 24 hours. Over here are all emails that went to support@foo-app.com in the past 24 hours. Over here is all UI telemetry that Foo collected: where users clicked, where they scrolled to, etc. Over there is the current database snapshot of Foo. Here is the document that describes the business goals of Foo, and the basic limitations (legal, financial) in which it needs to operate.
This is your mission: analyze all these inputs, analyze the codebase — and then create Pull Requests with the three most pressing improvements that make Foo a better software for its users, based on your analysis."
Yes, there's a ton of details I'm glossing over, and yet: shouldn't something like this lead to some kind of fully automatic improvement of software Foo over time?
Software is used, produces telemetry, Agent analyzes telemetry, improves software according to what the telemetry says, new software version is used, produces telemetry...
I was working with it last night and it would create a function, and almost immediately delete it and things wouldn't work. (╯°□°)╯︵ ┻━┻
Like right there in your own code man you're calling a non existent function... how did you even "know" what parameters to send it?
Sure I can get a bit of a nice greenfield project going, and some focused changes if I can put blinders on it and hold AI's hand. But after that I'm prompting it to put back code it deletes for no known reason. It seems to lose focus and think it is working on something else ... forget initial guidelines.
Code also has built in assumptions that aren't apparent and AI does not seem to care. This is hard for humans, but also seemingly impossible for AI. "We handle our X,Y,Z like A,B,C for reasons." only seems to keep AI on the ball for a short bit of time. AI has no ability to UNDERSTAND "oh yeah that will blow up because L,M,N" like a human does, and no matter how foolish the prompt AI will do it.
If you did that today, what percentage of the time would the change improve the software? What percent of the time would it make it worse (security, usability, accuracy, or whatever)?
I think that today, this approach would work on some days. Some days it would fail, perhaps catastrophically. (I don't know the percentages, though. Does anyone have enough experience to comment?)