What language models can I fine-tune at home?

Question

Are there any interesting language models, that can be fine-tuned on desktop hardware?

theCrowing · Accepted Answer

The most promising are probably one of the smaller versions of bloom https://huggingface.co/docs/transformers/model_doc/bloom or gpt-j https://github.com/kingoflolz/mesh-transformer-jax