HACKER Q&A
📣 Edd1CC

Are there any Gen-AI products in the wild?


I went to a major conference and the AI hype was mad. All devs will be replaced, all testers will be replaced, code will be generated from a prompt each time, tests will be generated from a prompt each time.

Utterly absurd, in my cynical mind.

But, playing devils advocate, are there any popular tools or apps in the wild that are written in this way? Or even partly in this way?


  👤 ultrablue Accepted Answer ✓
I've been experimenting with basic (free, COTS) AIs to generate requirements and then code based on those requirements, typically python or php.

The results have been dismal. Fine-tuning prompts is a good exercise for requirements gathering, complete with the tedium that we all love. Sadly none of the AIs that I used were capable of remembering the current requirements with any degree of confidence. ChatGPT4 does a pretty good job until it forgets everything randomly. Both it and CoPilot failed to remember simple instructions like "Please don't send the updated requirements until I ask for them." I assume so that I can waste tokens?

For code creation, both AIs consistently failed to include features that were in the requirements and in pseudo code that they provided and that I approved.

I did enjoy the change of pace of prompt engineering. It's fun while it's happening and while the AI behaves. But it gets very old very quickly saying things like:

Me: We just added , but I don't see it in the most recent version of the requirements.

It: Sorry Dave, you're right. Here's the latest requirements:

Hallucinations abound, not just adding things at random, but forgetting things, sometimes with a great deal of resistance to incorporating the forgotten thing.

Also, once you do get to code, don't expect to fine tune it. If you see a bug or oversight in AI-provided code and point it out, it won't correct the code. Instead it appears to re-generate it from scratch. Will there be new, unasked for features? Could be. What about the correction -- is it fixed? Maybe. And maybe in a completely new way.

I have found some success with CoPilot and basic research. In essence it's a smarter search engine. So prompts like this can result in useful leads:

I've got some data that has .

Questions:

What is the correct term for ?

What python library would you suggest to help me analyze ?

It's quite possible that my prompt-fu is simply weak and ignorant. And using COTS free-ish AIs I'm getting what I pay for. But I would say that as far as using an AI as an all-purpose junior goes, we're not there yet.


👤 d3m0t3p
I think you could say that Github copilot is partly in this way ?