HACKER Q&A
📣 wseqyrku

SEO for GPT


I imagine if GPT were to recommend your website or product, the home page is going to be crucial in communicating what your company does.

Does it make sense to have something akin to `robots.txt` for GPT trainers? or is it better to let it infer the product for itself?

  👤 verdverm Accepted Answer ✓
robots.txt does nothing for helping search engines understand what a page is about, just how to crawl. How do you see this aiding GPT?

You're probably thinking of JSON-LD, which obviously already exists, but is not some golden truth. Search engines have been using machine learning to understand the content and topics of sites for a long time.

GPT is token prediction, you still want to do all the same things for your website to aid discovery and understanding.

👤 smoldesu
If you're lucky enough to ever get mentioned in a training dataset, the last thing you'd want to do is something suspicious enough to get removed.