Why is A/B testing unacceptable for novels but encouraged for software?

A/B testing doesn't lead to coherent thinking, which is essential to success in creative works.

That is, you can run this test in precisely two places: In early planning(what kinds of story do people want) and when adding finishing touches(alternative versions of a scene).

Everything else has to come from self-feedback, a structural plan for what the novel "does": Does it explore certain themes? Does it go deep into a character's psyche? Is it a way to build up a world? Once you've set that plan, you have the tool you need to rubric yourself and determine what MUST be written(as in when you see a MUST in RFC specs) to satisfy the plan and what can be cut before it ever hits the page. And so most of the process never needs another person in the loop, it's just grinding through the technical details of how to make the story work given that spec. Seeking external advice is more counterproductive than not when you get into these details, because, as with the innards of coding, it's a spaghetti of dependencies that you can't get many "quick fixes" for.

When we look at a software product, there's still a great(and probably underserved) need for that kind of structured rubric, but most of the work is biased towards delivering a high quality of service according to specific metrics, which is something that can be evaluated on a feature-by-feature basis. Often, in fact, the metrics get in the way of addressing the structural problem by creating conflicts of interest, especially when profit motive appears. That leads you towards, e.g. productivity tools that dark-pattern in some ads so that you misclick them.

A lot of things that are commonplace in software are curiously absent in other areas.

How about just upgradable versions? Say an author puts out a novel, and then over the next year, does some further edits, better character development, whatever. Why not put out a version 2 of the novel?

Same question with music. Especially now, with so much music worked on in software rather than recorded to tape, it would be easy to spin out a version 2 of a single or an album.

We do see this in some other areas; textbooks, for example, routinely have multiple editions with various updates.

It seems really odd to me that we don't make updates to all sorts of media.

But then, lots of folks were perturbed at the updates to the original Star Wars movies...

Authors could be running a/b tests during the process. The book is kind of an end state.

Similar to software, you run a/b tests to figure out an optimal end state. Once you get the results you may not a/b test that part of the app ever again.

A/B testing just isn't the right tool for the job. The combinatorial explosion of things that can be tweaked with small variants, not to mention major changes, is for most practical purposes infinite. Readers also self select to a large degree, some changes to a novel can result in a work that appeals to completely different groups of readers!

What many authors do is have a group of 'early readers' that they know and trust to give them actionable qualitative feedback ("I didn't really feel that decision by the character was consistent with what we know about them", or "I get you're trying to establish the setting, but the action and conflict doesn't start until chapter 3, you need to bring it forward, or at least do some foreshadowing"). This is much more like agile software development that revolves around close collaboration with a customer, and sometimes an author's agent is also an early reader.

That said, publishers also send out hundreds or even thousands of early (usually edited but not always copy-edited) copies of a novel to a much wider group to solicit feedback (and generate buzz). Such Advance Review Copies, or ARCs (and their electronic equivalents eARCs) are often sent out looking for bug reports or pain points ("Hey this unnecessary rape scene isn't cool and doesn't even advance the plot") and some indication as to what sort of readers will most like it, which is often really helpful when a work doesn't fall neatly into a particular subgenre or subverts a subgenres tropes and cliches (you can think of this as the publisher getting data on "people who like X also like Y" or "This is Space Opera for people who don't like Space Opera!"). This stage is more about priming (ARCs go to reviewers as well) and optimizing the top of the funnel than doing major iteration on the book itself, and collecting endorsements from other authors or celebrities to feature on the cover, but detailed feedback if provided won't be discarded, though it will be filtered by the editor.

So, you see, within the constraints of novel publishing and the long lead times to get books printed (12 months from when the editor accepts the manuscript!) Novels actually do incorporate a lot of feedback and iteration, just not the sort of micro optimization that is the hallmark of A/B tests.

Sounds interesting but can you get enough readers to help you do the testing?

Another thing is that I don't think you can isolate basic parts like you can with software since there needs to be lots of writing to convey an idea. The time needed to do it would increase by a large factor when you try to break up to test it.

I'd like to to see a/b testing for book outlines? Or book elevator pitches?

We expect software to be updated with improvements to its functionality, whereas a novel is non-functional. (I'm not quite sure what non-functional means...)

My other reply is that A/B testing might be said to be accomplished in consultation with an editor.

Novels are considered creative works, not products. And there are plenty of formula based novels. A/B testing isn't going to improve anything that's intentionally mediocre.

Is art meant to be min-maxed?

Perhaps a publisher might think so, but I would suggest the appeal is the unique POV and voice of the author and the choices they make.

You should have A/B tested this question in your brain prior to writing it …

It’s called a “choose your own adventure” novel