How to get into OS/systems programming in 2023?

Question

I graduated from college about decade+ ago, took few OS & systems programming courses, but didn't find them interesting back then. However, now, more than a decade later, I'm finding myself bored at my cookie cutter C#/Asp.net/SQL(+ cloud) job. What would be your suggestions getting into OS/System programming in 2023 ? For starters, I'm thinking about learning assembly, which I never learned.

_448 · Accepted Answer

* Start with C programming language. K&R book is still the best, even though outdated.
* Take a course like Nand2Tetris (freely available online)
* Read a good OS book, e.g. The Design of the UNIX Operating System by Maurice Bach or any Andrew Tanenbaum book on the subject.
* Search for "unix systems programming" or "linux systems programming" and you will find some fantastic free books online.
* Read the free Linux Device Driver book. It is outdated but will still provide enough information to help you self navigate Linux driver code.
* Visit https://wiki.osdev.org/Expanded_Main_Page
* And as they say, "Read the source, Luke"; read the Linux source code. First you will have to identify which subsystem you are interested in, and then start browsing that.

markus_zhang · Answer

I'm following the OSTEP website and the book:
https://pages.cs.wisc.edu/~remzi/OSTEP/
In the first step they ask you to build a simple shell with redirection and parallel execution which seems to be accessible to me (I don't have CS background).
MIT also has their own OS courses exposed to audiences:
https://pdos.csail.mit.edu/6.S081/2020/

1letterunixname · Answer

I would learn C and x86/arm assembly first enough to bootstrap.
It's usually good to learn how to write ld linker scripts and how to make something bootable from grub.
Then, I would look Rust because it's easier to create correct no_std kernels.
EDIT: Link spew turned into a gist https://gist.github.com/5a1b94e8fde45e37c55c5a13d97c9b3f

HeyLaughingBoy · Answer

As a hobby or as a career switch?I've been programming embedded systems for the majority of my career and if you were to tell me that you just learned assembly, I'd wonder why you were wasting time like that. The last time I did more than glance at assembly language would have been around 1999.If it's for a hobby, grab an arduino clone board and go for it! I'd suggest something ESP32-based though.

gladiatr72 · Answer

I haven't been an active BSD operator for several years (the old hardware died), but the lack of corporate influence in the DragonFly/Free/Net/OpenBSD might be a space to check out. They've got non-trivial standards for what gets excepted into their source trees, but a significant number of contributors to those projects are professional or (at the very least) lifelong systems coders. OpenBSD would be the best choice if you're into what's current in the hardened systems arena. NetBSD has historically been the OS that runs on nearly every general computing platform that can still boot (I thought I had read something about Net sunsetting support for the vax architecture, but I just checked--still listed on their tier 2 list)

eimrine · Answer

Nobody (except of hardware guys and compiler writers) needs to learn Asm, but definitely you are got to get a skill of reading it. There are a lot of asms (one per every computing architerture plus several possible syntaxes). Of course you need to be an expert in C, in my understanding this is way more important than Asm. One of the easiest way of starting system programming is to build some electronic toy based on microcontroller and write some code for it to do something useful. Bonus points will be earned if you will manage to pack some complicated behaviour so heavily that your controller will be having zero free memory.

hnthrowaway0328 · Answer

One thing I found pretty useful is to port very early Linux command line programs to modern platforms.I just "ported" (nerfed the number of switches) the 1991 version of `ls.c` to modern Ubuntu and learned a lot of about `stat()`, `lstat()`, the `struct state`, file types in Linux and some programming tricks. It only has 1.2K LoC so it's easy to sort out the logic and write your own from scratch if needed.

mikewarot · Answer

I'd dig into genode[1], which is a capability based operating system. You'll likely see an upsurge in interest in capability based systems in the next decade.[1] https://genode.org/

sonjat · Answer

You can try working on the Linux kernel, perhaps with the kernel janitors project (https://code.google.com/archive/p/kernel-janitors/ ). I recommend hanging out on the mailing list a little while to get some idea of what to do, as well as reading the wikis on that page.

brapru · Answer

Also worth mentioning SerenityOS https://github.com/SerenityOS/serenity. It&rsquo;s a very friendly environment and easy to get involved.

phendrenad2 · Answer

Assembly isn't very fun, but maybe reading an old book on assembly for an old 8- or 16-bit cpu will satisfy your curiosity.

How to get into OS/systems programming in 2023?

I'd dig into genode[1], which is a capability based operating system. You'll likely see an upsurge in interest in capability based systems in the next decade.
[1] https://genode.org/

You can try working on the Linux kernel, perhaps with the kernel janitors project (https://code.google.com/archive/p/kernel-janitors/ ). I recommend hanging out on the mailing list a little while to get some idea of what to do, as well as reading the wikis on that page.

Also worth mentioning SerenityOS https://github.com/SerenityOS/serenity. It’s a very friendly environment and easy to get involved.

Assembly isn't very fun, but maybe reading an old book on assembly for an old 8- or 16-bit cpu will satisfy your curiosity.