Why are we not seeing “Large X models”?

They exist. People have trained BERT-like and other transformer models for specialist domains, particularly medicine. I “see” them all the time on arXiv.org.

Models like that perform well if you are doing classification, information extraction, etc.

For text generation though they have the same bullshitting problem as ChatGPT however, remember that science chatbot that Facebook released that they had to take down?