Why are private/hidden methods so popular in OOP APIs?

I think you need to clarify what you mean by "private". Are you referring to making methods private?

When you make a method public, you are making a promise to users that you won't make any breaking changes to that API without a major version upgrade. Once something is out there, you can't take it back. So if you have some functionality that isn't meant to be used by the outside world but is used to serve your API, then you wouldn't want to expose that and tie your hands in the future.

I know the answer, thanks to watching a recent video by Richard Feldman on why functional programming isn’t popular yet. I recommend watching it for some interesting talk about programming languages in general, it’s on YouTube at https://m.youtube.com/watch?v=QyJZzq0v7Z4

Anyway what privates give you is what everyone wants from a programming language: modularity.

We want to be able to expose an api to some code and for it to have some inner workings that we don’t see because:

1. If we know they can’t be touched we have some implicit guarantees about our program. E.g. the mouse pointer cannot go off screen even if we try to make a call to do that.

2. It is simpler for a programmer to understand the api which might be 1% of the size of the module itself with all private’s.

3. I can change my private’s without breaking your calls. Essentially a minor version change in Semvar parlance instead of a major breaking change.

4. Make impossible state impossible! By guarding the state.

All these things require modularity, and OO uses classes and private members to provide that modularity.

Elm uses modules and the exposing keyword (or lack of!) as a contrast.

JS modules are similar with a lack of an export allowing you to hide something. That something you did export can use for state or functionality. There is an old IIFE trick that works this way also.

Making everything public is a bad idea, as you miss out on modularity advantages. Not sure how Ruby hides things as I haven’t used it for a while but I’m sure there is a way.

And finally reflection in OO to access private’s is almost always a sin, unless you are make a debugging tool or similar. Canny? No way!

The primary reason is of course encapsulation. Many things you don't want to leave publically accessible. For instance if you're creating a thread safe api, you need to be mindful of references/mutexes/locks which can easily be enforced with wrappers. If you expose the underlying objects directly you could easily dead lock, corrupt memory, create race conditions, etc. It also greatly simplifies testing and guarantees/provability since you can guard against access patterns or certain inputs or states.

Personally, I have run into enough horror caused by mutating a global state that I write code which is quite strict about what it exposes. If you're a python dev, I'd suggest reading this (posted on hn recently but I can't find the hn link currently): https://instagram-engineering.com/python-at-scale-strict-mod...

I get your sentiment of the overused "hidden"/"private" stuff that seems like people following some sort of decades-old ritual and then adding getter/setter, where it should have been all transparent, knowingly that the module/package will eventually be running in a single process.

"private"/"hidden" attrs/methods help limiting the knowledge needed by new developers jumping into and building on top of existing code base.

Concealing complexity is necessary, but can be done with other methods such as HOF, selective exports, etc.

This is called interface-first design paradigm where your module is "serving" other module by doing its job without exposing its complexity.

Disclaimer: I'm mostly use TypeScript in a functional/data oriented design manner. Not a big fan of OOP.

This is all about the Unix philosophy of ‘do one thing, and do it well’. I used to be a python programmer before I moved to java and a significantly larger stack. The public interfaces of different classes made writing new code so so much easier. In fact, I think python should use this paradigm if only for the purpose of productivity.

The issue is not consent or privacy, it's promises.

An API is an "application programmatic interface" and the idea of an interface is that it's making a promise or some kind of contract.

Every precocious teenager thinks they're a consenting adult, but they don't really understand how fucking works until they've actually made a baby.

So you publish some code that does a thing, and however many consumers are exploring every sweaty nook and cranny of your interfaces, and not only do you not remember because cocaine is a hell of a drug, but now you've got a small legion of dependents you aren't even aware of.

Even if you're using a language like Java that lets you impose the bureaucracy of encapsulation, there are always various unintended consequences to the use of your API.

(Besides making assumptions about state, while the API might have all sorts of encapsulation, it still throws exceptions. And those exceptions might wrap internal exceptions, so a consumer trying to respond to an error condition might slide the condom off and investigate those internal exceptions to identify their specific error.)

There are two major reasons to claim a thing is private:

1. If your code breaks when I make a change, you can yell at me, but I can point at the fine print and tell my boss, "nah, it's swidi's fault".

2. It might even get you to nag me, "I need feature X to do a thing" and then I'm aware you're using feature X.

I'm working on a language to make APIs work better[1] and you should tell me if I'm fucking crazy or not.

Don't take any of this seriously, I'm quite inebriated at the moment.

[1]: https://tenet-lang.org/

To quote something from Pragmatic Programmer (1999):

"Functions should act like a contract. Functions should do no more and no less than what they're expected to. If they fail to meet the contract, they should report it."

You can reason about your code more effectively when you can see more precisely what function calls are made on an object or class of objects, and from where. Making a method private limits that surface area.

The more public interfaces you have, the harder it is to improve things.

"Implementation Detail" is a fundamental concept in software engineering.

If we don't formalise something as fundamental as that in tooling, we're pretty much giving up on tooling.