HACKER Q&A
📣 alphasonic

Are you using a local LLM? If yes, what’s the use case?


Are you using a local LLM? If yes, what’s the use case?

  👤 accrual Accepted Answer ✓
I run one. I built an iMessage-like frontend to it using plain JS and a Python websocket backend. I use it mostly for curiosity and playing with different prompts. I only have 16GB of RAM to dedicate to it, so I use an 13B parameter model which is enough for fun and chitchat, but I don't find it good enough to replace ChatGPT. In the future I would like to upgrade to Llama2 and observe the difference.

https://github.com/nomic-ai/gpt4all