How bad is the GPU shortage for startups/researchers?

Question

Curious how folks are faring with GPU access for training / inference? (those of us outside of big contracts / commits with the big three)I've seen folks try to address the shortage through a number of mechanisms: - Co-op pool for recipients of a specific AI investment fund - Collating a transparent price book (Shadeform) - People putting a rack under their desks?(Full disclosure, I am thinking about renting out GPUs, trying to get a pulse on the market)

steerex · Accepted Answer

I haven't actually needed access so haven't tested this yet, but a month ago an AWS rep told me that priority went to folks who were willing to make up-front commitments; the longer-term, the better. He implied that there was a serious shortage.

quickthrower2 · Answer

For hobby stuff: no problem. VastAI. I imagine the issue is when you need banks of 8xA100s machines.

rkwasny · Answer

If you need 8xA100 you can get them cheaply on GCP, just not in any region.If you need 8xH100 only option is AWS, p5.48xlarge - $98/hourI recommend skypilot for finding a good region