Does Nvidia have any credible competition in the AI/DL space?

Some are here - some of these are things people are using today, some are available but don't have much user adoption, some are technically available but very hard to purchase or rent/use, and some aren't yet available:

* Software: OpenAI's Triton (you might've noticed it mentioned in some of "TheBloke" model releases and as an option in the oobabooga text-generation-webui), Modular's Mojo (on top of MLIR), OctoML (from the creators of TVM), geohot's tiny corp, CUDA porting efforts, PyTorch as a way of reducing reliance on CUDA

* Hardware: TPUs, Amazon Inferentia, Cloud companies working on chips (Microsoft Project Athena, AWS Tranium, TPU v5), chip startups (Cerebras, Tenstorrent), AMD's MI300A and MI300X, Tesla Dojo and D1, Meta's MTIA, Habana Gaudi, LLM ASICs

I'm in the process of writing about this (I'll probably post in the hiring freelancer thread tomorrow, might like to find a freelancer to help me research and write it).

The A/H100 with infiniband are still the most common request for startups doing LLM training though, I did some research and a writeup on that.

If I'm missing or miscategorizing any above, please let me know.

I think AMD is at the very least, credible.

https://wccftech.com/amd-instinct-mi250-boosted-ai-performan...

I think they are better than people give them credit for. They seem to have a stigma against them.

Nobody wants to make a large deployment of AMD hardware because there are of course risks involved and nobody wants to be the one to blame if it doesn't work out. I can't blame them, but that doesn't mean it wont potentially still work out.

All it takes is for a big successful deployment I think.

Currently there’s lockin due to most things calling upon cuda on nvidia gpus. There are alternatives to cuda but they’re up and coming: triton, tvm, openvino, improved rocm, etc.

My employer is getting closer: https://www.hpcwire.com/2023/06/28/mlperf-training-3-0-showc...

For inference, yes. For training, no.

Cerebrus wafer

Google TPU

Not that these are for the average buyer