Are there any ready-to-use Docker images for running LLMs locally?

I've been having a lot of trouble spinning up the various stacks for running open LLMs like Alpaca or Vicuna because they often require specific CUDA versions, specific gcc toolchains, etc...

Has anyone got a dockerfile or published container image that "just works" and can run 4-bit quantized models on CPUs and/or GPUs? Ideally something that will run StableLM.

I've tried to build such a thing myself, but I found that the vague instructions in blogs aren't sufficient for reproducible build. Too many instances of "clone this (every changing) Git repo" or "just curl & execute this", leading to very rapid bit-rot where even instructions from a month ago can't be reproduced!