We’re keeping it a secret.
"zero shot" = ask an LLM to do it with a prompt
"few shot" = show a model (maybe an LLM) a few examples; LLMs perform well with "in context learning" which means giving a prompt AND showing some examples
"many shot" = train a model with many (typically 1000s) of examples.
The more training examples you have, the better results you get. A lot of people are seduced by ChatGPT because it promises fast results without a lot of hard work, rigorous thinking, and such, but you get back what you put in.
My RSS reader and agent YOShInOn uses
to transform documents into vectors and then I apply classical ML techniques such as the support vector machine, logistic regression, k-means clustering and such. I used to do the same things with bag-of-words model, BERT-like models give a significant boost to the accuracy, are simple to implement, and run quickly. I can write a script that tests 1000s of alternative models a day.
The main classification YOShInOn does is "will I like this content?" which is a rather fuzzy problem that won't retest perfectly. I tried applying a fine-tuned model to this problem and after a few days of trying different things I developed a procedure that took 30 minutes to make a model about as good as my classical ML model take took more like 30 seconds to train. If my problem wasn't so fuzzy I'd benefit more from the fine tuning and someday I might apply YOShInOn to make a training set for a better defined problem but I am delighted with the system I have now because it does things that I've dreamed of for 20 years.
The whole "prompting" model is dangerously seductive for various reasons but the low-down is that language is treacherous. This classic book
https://www.amazon.com/G%C3%B6del-Escher-Bach-Eternal-Golden...
is not such an easy read but it contains some parables that explain why making a chatbot do what people would like a chatbot will be like endlessly pushing a bubble under the rug and these problems are not about the technology behind the chatbot but about the problem that they are trying to solve.
Using ChatGPT in Serbian is like talking to a really bad... bot from the 90s. It mostly feels like two bad machine translation steps to and from English in between.
So, there is a whole NLP world out there that's badly covered by OpenAI, which means lots of options for advances in ML.
Not that I find there aren't plenty of improvements to be made to ChatGPT English models too.