Are Nvidia H100s that good?

Question

I was playing around with some different GPUs yesterday and put all of the results here: https://www.tensordock.com/benchmarksI tried a vLLM and Resnet training workload. The H100 outperforms the A100 about 45% to 80% consistently, but it isn&rsquo;t that much faster&hellip;What workloads would see the most speedup, because I&rsquo;m really not seeing 3x+ any on vLLM or simple training workloads?

zer00eyz · Accepted Answer

You're doing inference, not training.https://lambdalabs.com/gpu-benchmarks

jonathanlei · Answer

Oops, just noticed the link isn&rsquo;t clickable, here you go! https://tensordock.com/benchmarks

Are Nvidia H100s that good?

You're doing inference, not training.
https://lambdalabs.com/gpu-benchmarks

Oops, just noticed the link isn’t clickable, here you go! https://tensordock.com/benchmarks