Is it time to run the LLM engines on the CPU?

You always could. It's just slower and less efficient because most of what a model query needs to do is matrix multiplication and the GPU is optimized for that.

It depends, how much coal are you comfortable burning?