Will there be a "LLM Prompt Engineer" Job in FAANG next year?

I think not.

I am seeing a lot of papers on arXiv where people do "prompt engineering" experiments, that is they set up maybe 1000 or 2000 sample problems, run them through the LLM, and see how it performs.

The results are usually sobering, frequently you find that the models get things right 70-80% of the time which is acceptable for a "chatbot" that is ultimately animated by the end user, but put it in a process where there are 10 steps that are 80% correct and you are talking about a process that works 10%. You can ask an LLM to "open the pod bay doors" but not expect it to fly the spacecraft to Jupiter.

The difference between organizations that apply ML successfully and those that don't comes down to two things: (i) luck, (ii) picking well-defined problems and developing appropriate training data. Luck comes in because once in a while you get handed an easy problem. If you don't have that you have to do the very difficult work of convincing management to invest in making a training set of 1000s if not 10,000s of examples. People are about as happy to see the ontologist as they are to see the oncologist because the ontologist makes them think rigorously and you will certainly have phone calls where you've figured it all out and then they call you back and they want to mess it up again and spend another few years pushing bubbles around under the rug. You've got to push back with tough love. People think the "new AI" eliminates the need for ontology but no, training is a "garbage in, garbage out" situation if your problem is not well defined you'll never get past a certain point in accuracy.

If you can master the black art of whipping real life problems into shape and the herding cats problems that get involved, that's a lot more valuable than being another statistic in the Kaggle leaderboard.

Nobody knows about future jobs for Prompt Engineers, but I suppose that such a job should require reading the new techniques and papers about LLM and be able to discover what type of context is better, so categorizing, clustering, A/B testing and some math skills could be good to have for such roles. I

If I was to interview someone for Prompt Engineering at least the candidate should be able to look for information in the web with the least effort, that is you should be able to collect important information with the minimum number of queries.

No, as prompt enginneering skill is a subset of anyone working with a LLM (not necessairly limited to DS/ML) and the current cases of hiring a prompt engineer are effectively PR stunts.