Would I see significant performance improvements (e.g. compilation speeds) by moving to M3/M4?
Let's say that I do not care about energy efficiency/battery life.
https://www.cpubenchmark.net/laptop.html
It seems like the Apple Silicon chips are generally faster single-core, but a bit slower multi-core. In real life, it will probably depend on how well optimized your compiler (or whatever software) is for Apple Silicon. And native Linux on Apple hardware isn't mature yet, so if you need to virtualize it, it'll be even slower.
The advantage of the Macs are generally efficiency (performance per watt, meaning longer battery life and less heat and noise) and build quality, and well, you get a Mac (if you like macOS). If you don't care and just need something to crank out FLOPs, IMO I'd probably just build a desktop and run the workload there and remote into it from a cheap laptop of any sort.
If you do get a Thinkpad though, read up on their thermal management and try to get one that doesn't throttle so easily. The X1s I had usually had lower voltage CPUs (less performance) and would throttle even those under load, so they weren't good for long sustained CPU usage. Fine for business use or the occasional `npm install`, but if you're running multi-hour workloads, it will very likely overheat and throttle down. Not sure if their workstation models (the T models and others) are better in this regard; they are generally larger and not as squished together (read: hopefully better cooling/more fans).
Not to mention the architecture: Apple Silicon performs better in some tasks (e.g. running Android emulators) not because they are faster, but because they are ARM.
PS: I have a MacBook Air, but in terms of performance purely there are better options for the same price here.
An 11 year old Apple isn't in the same state. An M* processor is a gimmick for today's needs; it is a niche platform solution that's reactionary to niche needs of today; that's not a design for the future. Hence M1, M2, M2, M4, M88, etc. The Apple OS will not support decade old hardware, it's not in the business model. You have a fine computer for today that is designed to be disposable. Like buying a plastic stool instead of a wooden one. The plastic one may be lighter, in a variety of colours, your friends might go for them, and it could lie about being privacy focused, but in the end it's a plastic stool not a wooden one.
Apple Silicon processors are generally much faster than equivalent AMD and Intel laptop processors.
Which OS matters more to you? If Linux, then use Intel. If macOS, then use Apple Silicon. Linux runs well enough on M1 processors, but you should not buy an M1 for Linux alone. Linux does not yet run on the newest Apple Silicon processors.
If you are using something like Docker on macOS that will also kill your performance even more.