HACKER Q&A
📣 nzach

What are some things ChatGPT isn't able to create?


I'm talking about things that should be within reach of ChatGPT, but it doesn't seem able to create.

My example would be a very specific Prometheus query to be used in an alert. The query needs to triggers the alert when the metric is zero(connected_users{} == 0) and also trigger when the metric is missing.

For this I would create a query like this:

(connected_users{} OR on() vector(0)) == 0

I've even tried to give it some hints on how to create this query but in the end he wasn't able to do it.

And that got me curious, in my mind this is something it should be able to create.

Do you have any similar examples?

  👤 type_Ben_struct Accepted Answer ✓
I’d imagine there’s lots of edge cases like this. It’s only regurgitating data it’s trained on, and if it’s missing from the training set then it won’t be able to produce results.

Sam Altman from OpenAI has been very clear that this shouldn’t be used for anything actually important. It can be useful if you have the knowledge to vet it’s output, but I’m worried that if used by someone uniformed it’s dangerous.

👤 bediger4000
I tried to get ChatGPT to write quines. Got one in Go, got rubbish for m4 and nroff quines. That matches with the theory that ChatGPT has to read something very similar to get ve good results - there's lots of Go quines, only one or two m4 quines, and no nroff quines I can find.
👤 hxugufjfjf
It does not know anything about Event IDs or Sysmon IDs. Just invents shit. Its weird considering how good it is at writng accurate code. This is just looking up a few tables on its archived internet.