PyTorch or Jax for Making an LLM?

You mean for finetuning an LLM?

PyTorch + QLORA is very affordable.

Otherwise JAX on TPUs is a reasonable way to finetune on a large dataset. I'm not sure about the exact cost these days.