For instance, I'm trying to use AWS Amplify's Auth service to authenticate users, and when I sign them out it's trying to auto-log them in again.
This is a known bug and there are a few github issues about it. ChatGPT confidently tells me that the way to solve this problem is to use some methods which, when I try them, don't exist at all on Amplify Auth, and when I search for them there's no record of them whatsoever.
Similarly I tried to get its help in making an animation using the RN Animated API, and its suggestions were all from React Native Reanimated (a different project), although the code sample imported the Animated API from RN instead. When I corrected it, it gave some suggestions with methods that didn't exist for either.
These are just a few examples but I'm finding it fairly useless for coding help so far. Am I expecting too much from it? Am I using it wrong?
In many ways it is a glorified copy/paste machine that's faster (in some cases) than using traditional search. And for bigger projects you're going to get obliterated because at some point the model will either forget the conversation or it will start giving you some random suggestions, all of which lead to frustration and lost time.
But for small projects and snippets, I definitely use it a lot. JavaScript functions, CSS boilerplates and stuff like that - even if it's 50 lines long, I can just do some other stuff while it generates the basic outline and I am on my way.
Hard to say.
Am I using it wrong?
Maybe? I think we're all still trying to figure out the best way(s) to use this stuff right now.
Just for the sake of illustration, here's a recounting of my recent experience using ChatGPT for code:
I've only tried it once on a coding project of any substance, but it did work well enough that it definitely saved me a meaningful amount of time. Probably on the order of an hour or two.
The project was concerned with transferring a bunch of my Github repos en-masse from one organization to another, using the REST API. I could have code that up mostly in my sleep, but I decided to try ChatGPT as an experiment. My initial prompt was something like:
"Write me some Java code to transfer a Github repo from one organization to another, using the REST API. Please use Apache HttpClient."
The code I got was pretty close to what I needed, although I think I did have to prompt it to tweak the code once or twice. One was asking it to add code to use a Personal Access Token as an Auth header.
So in the end, it generated most of the code I needed, and it represented a meaningful time savings, just based on the time it would have taken me to type the code in, plus I'd have had to consult the GitHub REST API docs at least once or twice to lookup endpoints, parameters, etc.
Can it do much more than something like that? I have no idea. But for dealing with fairly generic, boiler-plate'ish code like that, I find it useful enough that I'll probably continue to experiment with it.
YMMV, of course.
(2) Huge amounts of fraud and misconduct have been widespread in the text generation field, in the RNN era I'd be trying to train a model to write fake clinical reports and get results like
"A 57-year old male was admitted to the hospital for a heart attack attack attack attack attack attack.."