How big is your LLM compute budget?

What the title says.

I am trying to figure out how much people expect to spend when training their own LLM on their own data from scratch. I’m also wondering if there are techniques you can use to get that cost down (data/model parallelism. Pipelines etc.) and what libraries implement these.

Thanks!