Has anyone used llama.cpp in production?

Question

I&rsquo;ve found ggerganovs work on llama.cpp to be amazing, and I&rsquo;ve loved playing around with it. However, has anyone used it in production? I&rsquo;m sure there are some really cool use cases, but I haven&rsquo;t seen any yet.

runjake · Accepted Answer

Some people are.https://old.reddit.com/r/localllama

linusg789 · Answer

4chan /g/ has a small community of people playing around with local LLMs, maybe check there.

Has anyone used llama.cpp in production?

I’ve found ggerganovs work on llama.cpp to be amazing, and I’ve loved playing around with it. However, has anyone used it in production? I’m sure there are some really cool use cases, but I haven’t seen any yet.

Some people are.
https://old.reddit.com/r/localllama

4chan /g/ has a small community of people playing around with local LLMs, maybe check there.