Do you think ML model weights are copyrightable?

1. No. The weights are not a work of authorship. Nobody takes any intentional action to make the weights be what they are. Only works of authorship are copyrightable. And a copyright comes into being as the property of an author... which, as I said, models don't have.

The closest thing to authorship that happens is curation of the training data, but that's too tenuous to support a copyright. For one thing, the curation choices aren't directly targeted to shape the weights themselves, only the output when a separate program runs inference on those weights. And even for the output, there's probably not enough direct control there to support much of a claim of authorship.

And the people building the models had better hope that's true, because if that curation is strong enough to support a copyright, then it's really hard to claim that the contributions from the training data aren't strong enough to make the model a derivative work of every single training item.

2. No. The last thing we need is more ways for people with a bit of capital to get vast sweeping IPR claims.

Also, isn't about time HN got a real Markdown engine?

The idea copyright is to protect the author’s specific personality in the expression of an idea while, at the same time, allowing for the free circulation of the idea itself.

An ML model has no personality. It is not a legal entity. Also, its outputs are in no way the direct expression of the personality of those who trained it.

It is precisely this what differentiates ML from regular code: With regular code, it’s the programmer who solves the problem in a way that is similarly personal as someone writing a novel or an easy. But with ML, the specific implementation is found by the software itself.

You can’t have it both ways. Both not having to write the solution itself AND claim personal ownership of the results.

(But of course, we’ll see what the courts actually say.)

In 2016, we asked an actual IP lawyer the other question that has been fervently debated of late, “does the IP status of the training data taint the weights?”

He said “we have no idea, and we won’t until someone with a boatload of cash burns it in a lawsuit.”

So we might find out soon!

All the model's weights are just a compressed representation of data it was trained on. Keep in mind it's open data, created by millions of people for free and placed either on Stackexchange or Reddit or Facebook or on Wikipedia. Thus, it is a nonsense to speak about any kind of copyright wrt neural networks.

Not a lawyer. 1. EU yes / US not so easy. Under EU law, content of a database can be copyrighted. I argue the weights of ML model are like the content in a database. Under US law the arrangement of data needs to be creative / original. Creative / original assumes a human expressing himself. Maybe the design of the ML model, selection of training material and selection of hyperparameter justify a copyright claim on a database of model weights. 2. Yes as resources were used to create it.

I think this has interesting overlap with the concept of illegal numbers [1].

[1] https://en.wikipedia.org/wiki/Illegal_number

Curious why it would be copyright vs patent. Weights are in essence part of an invention - I think the specific numbers would be pretty weird to copyright, but patenting the system that includes the model architecture and training process would offer a much more robust protection that's more along the lines of past physical inventions.

I'd be interested to look and see if there are any old analog computer patents, and how they are worded? Like if the specify list of resistor values, or focus more on architecture and tuning. I think that's a possible precedent to look to.

How much wiggle room is there with weights? Tweak a single weight, is that copyright infringement or a new weight set?

> Do you think model weights can actually be copyrighted?

Under US law, the answer is clearly "no." US requires there to be, at minimum, a "spark of creativity." (Also, as far as legal definition is concerned, machines cannot possess any creativity.) If you try to start doing some creative legal argument to work around that, you're going to run into the issue that the same lines of legal thinking will conclude that you are infringing all of your training set... and trying to argue that infringing more or less everything ever made is somehow fair use is a truly uphill task.

You might be able to make a case for it under jurisdictions that follow the sweat-of-the-brow doctrine, but I know too little about jurisprudence in such jurisdictions to make any confident predictions.

The answer to (1) is plainly yes. Copyright is a human process. Things you think shouldn’t be copyrighted get copyright and vice versa.

Lawyers will consider how experts make decisions that affects and shapes the (sometimes far-) downstream representation of a valuable thing: thus an mp3 recording, a compiled executable, and, yes, a neural network’s model weights are all going to be in scope of copyright discussions.

As for “should”…I don’t know. I feel like the model I train to synthesise my own voice should be copyrightable. Meanwhile I’m wary of titanic foundation models destroying entire markets in part because of a copyright moat.

An argument can be made that model weights are a "creative work", in the same way that a list of pixel intensities and RGB values is a creative work. Who knows whether the argument will hold up or not.

As with all IP, it's one thing to be granted intellectual property rights, and a completely different thing to actually go about enforcing those rights. Without access to infringer's source code, it's challenging to make the case.

I really love this question. I think most of use will answer seeing "through the glass darkly" as it were.

I can't answer (1) as my opinion is just noise until it works through the legal system. I suspect they will be.

For (2), I think they should. You can't copyright the algorithm (recipe) but you can copyright the output (expression) of the recipe. On that note, this allows for open sourcing of models as well.

My opinion is that they should not be if they were created using public data that is itself not owned by the creator/trainer. Otherwise we have a new model of for profit piracy where huge companies exploit their capital advantage to dragnet the collective work of humanity and then claim ownership of the result.

If a model is trained on data that one owns the copyright to, it should be copyrightable.

1. If synthetically crafted DNA can be patented, then why wouldn't weights be eligible for the same protection? Both are kind of similar in function, I think…

2. Definitely no. Copyright should only apply to things entirely crafted by hand IMO.

Any information can be represented as a single number (albeit possibly a very large number). You can turn a movie into a number. Model weights are just numbers too. So yes copyright laws should apply.

1. No

2. NO

Absolutely not. Software should not be copyrightable because math equations are not copyrightable. But besides that, ML Models end up as polynomials at the end of the day, and that literally cannot be copyrighted.

hprotagonist probably has the most correct answer, but IMO no to both. Mostly because I’m looking at it from the spirit of copyright law, not the rule. The spirit is to encourage the creation of art/books/etc by allowing a artist/writer to make a living from their craft.

Not to mechanically create an index of facts about other art/writing, no matter its usefulness.

The weights don't do much without the model right? So shouldn't weights + model be considered together?

I'd say that we'd need some more law to be certain, either some precedent ruling or explicit new legislation, but as of now some of the relevant and irrelevant arguments would be:

1) Purely mechanistic transformations aren't copyrightable, so if a model is determined to be that (it may be debated if it is or isn't), then it has no independent protection from copyright law other than any potential restrictions imposed by fragments of other copyrighted works included in that particular model.

2) On the other hand, even the tiniest bit of creativity is sufficient to make something copyrightable. Some work on ML definitely has some creativity, however is the "creative part" fully within the code used for training and, or is part of that creativity embedded also in the weights output by that code?

3) Facts about a copyrighted work; e.g. the word frequency statistics of a copyrighted book do not violate the work's copyright and also are not copyrightable themselves, so at least some language models - the very simplest word n-gram models, which were used both not that long ago for statistical machine translation and also long, long ago in pre-computer dictionary work - aren't covered by copyright, just as any other collection of facts. Databases may have specific ('sui generis') extra protection granted by law, but then that's a separate debate than general copyright.

4) In general, there are different things which often are conflated but in this complex situation the difference may be meaningful and even critical - e.g. (a) being a derived work of the training data; (b) violating the (limited, explicitly enumerated) exclusive rights of the author of that training data; and (c) being copyrightable itself - those are three separate things that might have different answers for the same type of model.

5) "sweat of the brow" doctrine asserts that (at least in USA) it doesn't matter how much effort and money it took to make something, so the effort/expense of training a model is not relevant to it 'deserving' protection and being copyrightable.

6) Not all creative creation is protected by copyright - the law lists very many protected types, but the rest are not protected unless courts get convinced that they fall under the umbrella of something explicitly listed in copyright law; for example, copyright applies to software because (and only because) program code is considered to be a type of literary work, which is one of the things listed in https://www.copyright.gov/title17/92chap1.html#102 or equivalent legislation elsewhere. Are model weights a type of literary work? That's a tricky question, and my opinion on it doesn't really matter, we'd need to hear a precendent-setting judge assert if they are or not.

... and there's probably many more arguments that should be considered before even trying to answer the original questions. And I think that it's illustrative to see what all the megacorps with megapaid megalawyers are doing; it seems that they're writing all their legal documents from a perspective where models might not be copyrightable and trying to assert whatever conditions they want under some type of contract, since conditions properly agreed under contract law will apply no matter how the copyright law will be interpreted.