With FAANG turnover-why don't we know about actual recommendation algos?
Recommendation algorithms of Facebook, Google, YouTube are still a black box to us. We don't know what exact features are the input, how much are they weighted and so on.
But the turnover of software engineers is quite substantial, they come and go from these companies. How is it even possible to keep this stuff secret, then?
For most folks outside of the above companies I can understand why know-how would be kept secret after people are gone from the employer: it's quite boring and of little interest to anyone, really. No benefit or thrill if you divulge anything.
The twitter code (with whatever caveats) is publicly available. What's a concise summary of how their "algorithm" works?
At this point I expect many of the algorithms have been trained not designed and thus nobody knows for sure what they are doing. And even before that it's a really complex topic that probably can't be easily summarized.
People generally value their financial wellbeing and trustworthiness more than Internet points. It is also likely that no one person knows the algo, and fully describing the algo would likely require theft of IP.
Nice try, Internal HR Counsel.
> For most folks outside of the above companies I can understand why know-how would be kept secret after people are gone from the employer: it's quite boring and of little interest to anyone, really. No benefit or thrill if you divulge anything.
Why would you expect it to be any different here? There's not going to be anything mind-blowing, earth-shattering. They're very big and complex and likely continually tweaked.
Go look in arxiv.org; what I see is a huge amount of research in China and India, you might think there is a “missile gap” in e-commerce. The real action is in things like TikTok and Temu, Amazon and Google are legacy companies like AMC, Ford and GM were in the 1970s.
These aren't "algorithms" in the traditional sense of that word. They are complex, convoluted systems with hundreds of moving parts/services owned by dozens to hundreds of teams. There's no one individual who has a total understanding of the whole thing, and only a few that could speak to it at a high level. Most folks are busy optimizing and improving the microservices their team owns and such. The few folks with the high-level understanding are typically senior enough where divulging this info is neither particularly thrilling nor worth any potential risk.
And yes, recommendation engines are not nearly as exciting as the conspiracy theories around them.
There’s not much need for turnover, plenty of companies publish quite a bit on how their systems work. You can look for recsys papers from your favorite company if you want. The Netflix recommendation systems workshop is also good and has many industry talks.
The reality is that there’s a lot of (what is essentially) matrix factorization, and not much of anything nefarious or very interesting to most people who are not recommendation system engineers.
Well Google, YouTube and FB engineers should be embarrassed because their "algorithms" suck.