Why is it hard to rent GPUs?

Question

I contacted Google Cloud and Linode. The latter told me that they need to start a sales process to prevent fraud.Doesn&rsquo;t most fraud happen with CPU&rsquo;s? Spam, piracy, etc?What&rsquo;s behind this? A shortage doesn&rsquo;t make sense because a first come first serve model works well. Is Nvidia pulling the strings?

ayende · Accepted Answer

They mean fraud against them.Assume you rent a GPU to do PoW mining, using a stolen credit card, they can't charge you, but you have your coins and can disappear

antiherovelo · Answer

For a DC operator, GPUs have high runtime costs: think about all amps drawn and heat spewed by a box full of GPUs.
The fraud thing has been a factor, for sure. You need some guarantees that your client is legit before you spend money on them.
Running big clusters of GPUs is a more challenging technical exercise than ordinary Hypervisors/VMs/K8s/whatever. Doubly so if you are doing multi-tenancy and containerization. It takes a lot of effort to rig that stuff up properly and integrate it with the rest of you management stack, and folks tend to be protective of it once it’s fired up.

mejutoco · Answer

Consumer GPUs (rtx3090, etc) are not allowed to use in a datacenter according to nvidia&rsquo;s terms, so companies renting them need to buy the much more expensive GPUs. I think this might be a big factor.

politelemon · Answer

By rent GPUs do you mean creating an instance/VM with a GPU attached? I've not seen this on AWS, assuming their instance types are an equivalent of what you're referring to.https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html

freemint · Answer

Try vast.ai . They are a market place where people rent out mostly consumer GPUs at low prices.

phire · Answer

Crypto miners ruin everything.

joshxyz · Answer

It costs to run and it costs a lot

Why is it hard to rent GPUs?

They mean fraud against them.
Assume you rent a GPU to do PoW mining, using a stolen credit card, they can't charge you, but you have your coins and can disappear

Consumer GPUs (rtx3090, etc) are not allowed to use in a datacenter according to nvidia’s terms, so companies renting them need to buy the much more expensive GPUs. I think this might be a big factor.

By rent GPUs do you mean creating an instance/VM with a GPU attached? I've not seen this on AWS, assuming their instance types are an equivalent of what you're referring to.
https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html

Try vast.ai . They are a market place where people rent out mostly consumer GPUs at low prices.

Crypto miners ruin everything.

It costs to run and it costs a lot