How do you know which model is hidden behind the API?

Question

We know that Llama weights are not for commercial use, but suppose somebody served Llama(or any other model) via Web interface or API, how do you know which model it is? Does Facebook have magic prompts that returns an ID? What happens when the model is finetuned or a delta is applied to the weights?Now that there is an explosion of apps building on various LLMs, the copyright of LLMs doesn't seem to be very well defined, will LLMs have DRM in them in the future?

quanticle · Accepted Answer

Currently? You don't know for sure. It might be possible to make some guesses by examining how the model responds to various prompts, and checking the output of the model against the output from the same prompt against known models. But that will at best, give you a good guess, not certainty. This is why the FLI's proposal for AI regulation [1] suggests that AI models be both watermarked and that the output from AI models be clearly identified as such. In a world where most people use regulated models, this would enable you to identify which model generated a certain piece of content.
As for applying a delta to the weights, that would likely break the model. It would be like randomly scrambling bytes in a compressed file and then expecting the file to decompress properly.
[1]: https://futureoflife.org/wp-content/uploads/2023/04/FLI_Poli...