Can I download GPT / ChatGPT to my desktop?

Question

I would love to be able to run GPT / ChatGPT on my desktop and remove some of the length limits on text.How can I do that, and where can I download it from?

5e92cb50239222b · Accepted Answer

OpenAI is 'open' in the name only, so no. I don't think they have any plans of opening full access to the public either, considering that their previous model (that ChatGPT builds upon) was sold to Microsoft for exclusive use:https://en.wikipedia.org/wiki/GPT-3

turkeygizzard · Answer

I'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home.Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able to manage context/understanding beyond this window length

htns · Answer

I don't much follow AI news beyond what I randomly happen to see on HN, but this might still be the largest open source model: https://github.com/yandex/YaLM-100B . There's discussion of it here: https://old.reddit.com/r/MachineLearning/comments/vpn0r1/d_h... - at the bottom of that page is a comment from someone who actually ran it in the cloud.

Sharlin · Answer

Even if it were freely available, there's no way to run GPT3 or ChatGPT on any existing desktop hardware. The exact hardware requirements aren't public either (yes, very "open") but a full 175-billion-parameter GPT3 instance requires hundreds of gigabytes of GPU memory, and even though ChatGPT is "smaller and better", when it comes to conversational dialogue, there's no way to fit it in current consumer hardware.

navjack27 · Answer

You can do gpt-jhttps://gist.github.com/navjack/32197772df1c0a8dbb8628676bc4...I mean yeah after you set it up like this you still have to prompt engineer to get it to behave like a chat but I mean it's better than GPT - 2

fred967 · Answer

You can download his ancestor here:https://winworldpc.com/product/dr-sbaitso/2x

mellosouls · Answer

It's not possible post GPT2 for the reasons given by others.
Open communities with potential for involving yourself include Hugging Face and EleutherAI, the former perhaps more accessible, the latter an active Discord.
It's been a while since I spent time looking at them, I'm not sure if there is something you can easily get up and running with.
https://huggingface.co/
https://www.eleuther.ai/

dragonwriter · Answer

There are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”.
You probably won't be able to run (or especially train) them on typical desktops, though.
[0] https://www.eleuther.ai/projects/gpt-neox/

chamwislothe2nd · Answer

https://github.com/bigscience-workshop/petalsSince my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large models. Good luck!

trilbyglens · Answer

My feeling is that even if it were available to download, the compute requirements to run it at production speeds would likely be blistering.

PlotCitizen · Answer

It&rsquo;s not possible currently but there&rsquo;s another story on the HN front page with an open source alternative which I haven&rsquo;t tried

luckyme123 · Answer

Luckily, no. Otherwise you (others) could hack the safeties and ask it how to cheaply kill a lot of people or so .. better not make bad people too intelligent! (Obviously not talking about you)

Can I download GPT / ChatGPT to my desktop?

I would love to be able to run GPT / ChatGPT on my desktop and remove some of the length limits on text.
How can I do that, and where can I download it from?

OpenAI is 'open' in the name only, so no. I don't think they have any plans of opening full access to the public either, considering that their previous model (that ChatGPT builds upon) was sold to Microsoft for exclusive use:
https://en.wikipedia.org/wiki/GPT-3

You can do gpt-j
https://gist.github.com/navjack/32197772df1c0a8dbb8628676bc4...
I mean yeah after you set it up like this you still have to prompt engineer to get it to behave like a chat but I mean it's better than GPT - 2

You can download his ancestor here:
https://winworldpc.com/product/dr-sbaitso/2x

https://github.com/bigscience-workshop/petals
Since my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large models. Good luck!

My feeling is that even if it were available to download, the compute requirements to run it at production speeds would likely be blistering.

It’s not possible currently but there’s another story on the HN front page with an open source alternative which I haven’t tried

Luckily, no. Otherwise you (others) could hack the safeties and ask it how to cheaply kill a lot of people or so .. better not make bad people too intelligent! (Obviously not talking about you)