How bad is the xz hack?

I could have been much worse, but was identified last minute. Debian, Ubuntu and Fedora had the code only in their testing channels, and overall only .deb or .rpm based distros with systemd have been affected (see https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78b... ). We have to make sure open source maintainers are not underpaid, and stop the madness of global commons exploitation.

The real question should be how many more similar things there are in the entire Linux codebase I would bet good money there are more that haven't been detected.

Every intelligence service in the world wants one or more. Many can hire truly bright programmers who will contribute to Linux over years, doing nothing nefarious at all, to build a trustworthy profile. and even contribute meaningful and positive features.

Then, maybe 2 of 3 trusted profiles make tiny changes as part of a much larger valid contribution. each one a tiny adjustment, each one not particularly interesting from a security point of view, unless you look at all 3 at the same time, and of course they were not added at the same time.

I am quite certain they have people who are working at Apple and Microsoft as well with the same mission.

Does anyone know if distros using the systemd-notify patch are considering not linking to libsystemd and instead implementing the notify-protocol themselves?

As this would only be like 10 lines of code, instead of linking a huge library that will in turn pull in another bunch of libs, it seems like it should reduce attack surface.

It seems it was discovered in the last minute, close to ending up in ubuntu 24.04 LTS and derivates.

Luckily most users was unaffected unless they used a rolling release distro.

Finally, we still dont have a full analysis so its not known if systemd-enabled sshd is the only attack vector.

I think the really scary part is the particular modus operandi that this demonstrates, that is easily repeatable. A malicious actor builds up a reputation in the open source space and eventually takes ownership of the release process at which point a backdoor is inserted.

There is absolutely nothing that would suggest this hasn't happened before or will not happen again.

20 on a scale of 10. It's a remote code execution backdoor against the whole world... How bad can it be?

What I find curious is, everybody is acting like, yeah, we found that one attempt! But there gotta be more. And this is only open source. At Google, Meta, Microsoft there gotta be the same forces at play.

So yeah, congrats to everyone involved discovering this, excellent work, really! But how sure can we be, that there was no other successful attempt?

In practice, it's a non-issue for anyone using a stable version of a major Linux distro, such as Ubuntu 22.04 LTS or RHEL/Alma/Rocky 9. The liblzma in those distros were cut and frozen long before our suspect began to have serious involvement in its development. (Not sure about Amazon Linux. Arch doesn't seem to be affected.)

The more worrying question is, what else was compromised by these kinds of people, and for how long?

I too have been trying to find out about the hack.

There has been a lot of interesting analysis of the code, which has a feeling of the awe and sophistication of stuxnet yet on the scale of “this could be the work of a single motivated coder … like one of us”.

But what I haven’t found yet, and would like leads on, is what ther thinking is about the attack in a wider context…

What is the current thinking on:

1. Was the malicious code inserted by the new contributor, or was this likely a compromised account?

2. If the new contributor did it, did they become an xz maintainer with the aim of acting maliciously?

3. If the new contributor was malicious, why take so long to attack and why do it in xz? Or are they also attacking lots of other systems, perhaps under other names?

4. And what can we do about these attacks? How can we build a system that isn’t ultimately brought crashing down by one or two bad actors?

5. Small technical detail I haven’t spotted in writeups: was the attack commands signed in some way so only the attacker can use it and the world can go hunting for a smoking gun cert that matches the attacker?

This is probably the worst backdoor in history in the open source sphere, if not ever.

Certainly the worst of my career.

I feel total sympathy for the maintainer if they 100% didn’t know what was going on.

It’s also unfortunate that the maintainer will be associated with the lack of judgement. But they will get through it. Probably time to pass the torch fully now though.

This is just a small number of committers and one vuln but by its execution suggests there’s more to be discovered and what’s now on those machines this vuln could execute upon and what’s it done to machines on those networks.

Similar exploit vector in Swoole (binary blobs ran under the guise of "testing") and exactly why it was forked for Open Swoole: https://old.reddit.com/r/PHP/comments/1b9acx3/article_intro_...

Thankfully this only effects Swoole users, not all of PHP.

Not as bad as the fallout is going to be. Already some people are calling for mandatory identification when contributing to open source projects. This will, of course, have a chilling effect on FOSS contributions, while it seems unlikely (to me) that this would improve security in any meaningful manner.

It was caught just in time so the “badness” in terms of “what do I need to do to mitigate this right now” is thankfully pretty low.

But IMO the real “bad” here is that it calls into question the entire model of volunteer open source contribution, which underpins a crazy amount of the tech world. No easy answers there.

It is still pretty early but yeah, I suppose you could look at it as “-1 day” vulnerability… or a back door which goes back to my comment about it being still pretty early. Who added this code? And who is auditing it?

It was somehow astonishing to me that the code which is actually shipped (tarball) is not the very same code as the one in the repo. Somehow renders code reviews useless. Seems to be a common practice anyway.

And what us regular folks who run Ubuntu LTS on VMs need to do, if anything.

So, if say, I'm using MX Linux (semi-rolling release) based on debain stable, using sysVinit instead of systemd, I should be ok related to this issue?

perfect 10 on severity, probably close 10 on impact. Caught before it was widely exploited (it got noticed because it just rolled out, in an attempt to get it into the release window for 24.04 etc) but it’s essentially RCE on all Debian, Ubuntu, and fedora openssh servers.

How does this affect Mac users who had the vulnerability?

tl;dr. SSH though VPN only. Sleep tight.

Makes me wonder if there’s going to be a push for KYC (know your committer) in the open source world.